From karen.kinnear at oracle.com Wed Jun 10 13:54:59 2015 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 10 Jun 2015 09:54:59 -0400 Subject: RFR: Proposed jimage refresh for JDK9 In-Reply-To: References: <8B2EAD6D-BA19-4C5A-9DD6-15BD00D2191F@oracle.com> <8A4E1C90-1B23-41BB-BD1C-B01AFCE04291@oracle.com> <555B69DD.3010308@oracle.com> <5D669F63-4F9C-4508-9585-F509582BC97E@oracle.com> <8B159D12-1682-42DD-9BF9-5FA218E710FA@oracle.com> <55646CDE.5020106@oracle.com> <5564809D.2050200@oracle.com> <55648254.2060401@oracle.com> <5564A76C.5020500@oracle.com> Message-ID: Folks, We have concerns with the current jimage design relative to the JVM interfaces. We need to investigate alternative approaches here, including a libjimage.so analogous to libzip.so that would be called directly from both the JDK and the jvm. Some have suggested having most of the logic be in java. We are concerned about checking in JVM_* interfaces that will should go away soon and concerned about licensees and other java vendors adopting them making them harder to remove the longer they are there. Jim has explained that it is critical to get these changes in quickly so that the IDEs can get the updated jrtfs APIs, and that it is a huge amount of work to decouple the API changes from the jimage changes or to make the requested jimage interface changes. Assuming it is critical for these changes to go into JDK9 immediately rather than fixing the design issues first - We propose adding the JVM_* interfaces as deprecated when they go in. We would like commitment from Jim and those setting priorities for the jimage work that the changed design and JVM_* interfaces will be in the next set of changes coming from his team. If this is a problem due to priorities or staffing, can you please talk to us about that now? thanks, Karen On May 27, 2015, at 8:49 AM, Jim Laskey (Oracle) wrote: > [Have been OOTO] > > I have a cunning plan. However, getting this jimage refresh in play has me really bogged down (there is always one more thing -- curse you Jobs.) > > I do plan on moving the bulk of the jimage JVM_ calls to the JDK. The callbacks into the VM will be reduced to a few calls (open, close, read, read_compressed) and will be VM style correct. > > Cheers, > > ? Jim > > > > > >> On May 26, 2015, at 2:03 PM, Coleen Phillimore wrote: >> >> >> >> On 5/26/15 10:25 AM, Alan Bateman wrote: >>> >>> On 26/05/2015 15:18, Coleen Phillimore wrote: >>>> : >>>> >>>>> >>>>> In any case, I don't think the JVM functions are a supported interface so we shouldn't need a CCC. >>>> >>>> We need CCC to remove the JVM functions so I assume we need one to add them. >>> Okay although JVM_* functions have never been a supported interface (to my knowledge anyway). >>> >> These are the current rules, which assume that even though unsupported, these interfaces are known to our customers and licensees. >>> >>>>> >>>>> Also I think we need to see where this is going longer term, my preference would be to move these JVM_* functions out of the VM and put the jimage support in its own libjimage.so. I realize this requires mmap and other support that might be tied a bit to VM options but I think we should at least explore it. >>>> >>>> I think this functionality should be in Java. >>> We might be talking different issues here, I assume you need a minimal native implementation to startup. >> >> Yes. It seems wasteful to have the JDK code call back to the JVM for this, and then we have new JVM functions to support (and have to file CCC requests if we remove them). >> >> Thanks, >> Coleen >> >>> >>> -Alan >> > From james.laskey at oracle.com Thu Jun 18 00:08:54 2015 From: james.laskey at oracle.com (Jim Laskey (Oracle)) Date: Wed, 17 Jun 2015 21:08:54 -0300 Subject: RFR: JDK-8080511 - Refresh of jimage support Message-ID: https://bugs.openjdk.java.net/browse/JDK-8080511 This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools Details: - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 - A new archive abstraction for class/resource sources. - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) - JNI support for jimage into hotspot. - White box tests written to exercise native jimage support. From karen.kinnear at oracle.com Thu Jun 18 14:59:06 2015 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Thu, 18 Jun 2015 10:59:06 -0400 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: References: Message-ID: <23CC0B1E-BDE7-4E48-8A22-B0C19AA74AF8@oracle.com> Code review for the hotspot component: Thank you for the cleanups to jvm.cpp to make them JVM_ENTRY and fix thread_state. If you pass the testing and Lois ok's the hotspot code, then I am ok with checking in the hotspot code as is. Comments for next round of changes: 1. mutexLocker.cpp - new lock should say "false" not "true" for whether the VMThread should block 2. os.hpp restartable_read_at is added but I don't see it used 3. imageFile.cpp We've been asked to always used the counted form of strcpy - i.e. strncpy to ensure we never have buffer overruns. e.g. 311, 437, ... 4. jvm.cpp - seems odd that the interface would need to know if the resource was compressed or uncompressed. In the next round of API design, seems like the java code would pass in the uncompressedAddress and let the internals handle buffering the compressed data and doing the uncompressing transparently. 5. imageDecompressor.hpp get_decompressor I think you've mixed your exception handling approaches. You have a CHECK_NULL for new_symbol call, which means it will throw an exception if it returns NULL. Then you check for HAS_PENDING_EXCEPTION which is the logic you would use if did not have the CHECK, but then you throw the exception away. If you want to throw an exception from get_decompressor you should pass in the last argument as TRAPS macro which will give you the THREAD. see macros in exceptions.hpp So the way it is right now, you will throw an exception. You have a choice - you can throw an exception pass in TRAPS to get_decompressor, keep the CHECK_NULL and remove the if HAS_PENDING_EXCEPTION. - this is the recommended approach Alternative - change CHECK_NULL to pass in THREAD. Keep the rest. Given you are planning to take this out of the vm - this would make more sense. thanks, Karen On Jun 17, 2015, at 8:08 PM, Jim Laskey (Oracle) wrote: > https://bugs.openjdk.java.net/browse/JDK-8080511 > > This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. > > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools > > > Details: > > - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. > - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 > - A new archive abstraction for class/resource sources. > - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) > - JNI support for jimage into hotspot. > - White box tests written to exercise native jimage support. > From lois.foltan at oracle.com Thu Jun 18 17:55:18 2015 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 18 Jun 2015 13:55:18 -0400 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <23CC0B1E-BDE7-4E48-8A22-B0C19AA74AF8@oracle.com> References: <23CC0B1E-BDE7-4E48-8A22-B0C19AA74AF8@oracle.com> Message-ID: <55830606.50500@oracle.com> On 6/18/2015 10:59 AM, Karen Kinnear wrote: > Code review for the hotspot component: > > Thank you for the cleanups to jvm.cpp to make them JVM_ENTRY and fix thread_state. > > If you pass the testing and Lois ok's the hotspot code, then I am ok with checking in the > hotspot code as is. I have reviewed and am okay with checking in the hotspot code as is once testing is complete. One comment inlined below. Lois > > Comments for next round of changes: > > 1. mutexLocker.cpp - new lock should say "false" not "true" for whether the VMThread should block > > 2. os.hpp > restartable_read_at is added but I don't see it used > > 3. imageFile.cpp > We've been asked to always used the counted form of strcpy - i.e. strncpy to ensure we never > have buffer overruns. e.g. 311, 437, ... > > 4. jvm.cpp - seems odd that the interface would need to know if the resource was compressed > or uncompressed. In the next round of API design, seems like the java code would pass in > the uncompressedAddress and let the internals handle buffering the compressed data and > doing the uncompressing transparently. > > 5. imageDecompressor.hpp > get_decompressor > I think you've mixed your exception handling approaches. You have a CHECK_NULL for new_symbol > call, which means it will throw an exception if it returns NULL. Then you check for HAS_PENDING_EXCEPTION > which is the logic you would use if did not have the CHECK, but then you throw the exception away. > If you want to throw an exception from get_decompressor you should pass in the last argument as TRAPS macro > which will give you the THREAD. > > see macros in exceptions.hpp > > So the way it is right now, you will throw an exception. > > You have a choice - you can throw an exception > pass in TRAPS to get_decompressor, keep the CHECK_NULL and remove the if HAS_PENDING_EXCEPTION. > - this is the recommended approach > > Alternative - change CHECK_NULL to pass in THREAD. Keep the rest. Given you are planning to take this > out of the vm - this would make more sense. Whatever mechanism you choose to use in get_decompressor, please also use for image_decompressor_init( )/createSymbol() within imageDecompressor.cpp as well. > > thanks, > Karen > > > > On Jun 17, 2015, at 8:08 PM, Jim Laskey (Oracle) wrote: > >> https://bugs.openjdk.java.net/browse/JDK-8080511 >> >> This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. >> >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools >> >> >> Details: >> >> - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. >> - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 >> - A new archive abstraction for class/resource sources. >> - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) >> - JNI support for jimage into hotspot. >> - White box tests written to exercise native jimage support. >> From mandy.chung at oracle.com Thu Jun 18 18:48:25 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Jun 2015 11:48:25 -0700 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: References: Message-ID: <5E615ED4-B352-4C71-9F67-273BC08BBD37@oracle.com> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top modules.xml looks fine. test/lib/sun/hotspot/WhiteBox.java This adds the native entry points. Where are they implemented? It?s not obvious to me. http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools the jdeps change looks fine. http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk/ This is a huge patch that I have to rely on the extensive testing to catch potential issues. I tried running the jrt-fs tests using JDK 8 with jrt-fs.jar that work well. Some comments and mostly minor. make/gensrc/Gensrc-jdk.dev.gmk - you could revert this one make/src/classes/build/tools/module/ImageBuilder.java Nit: line 366, 382 - left over commented lines some indentation/formatting nits line 386-397 jdk/tools/jimage/TaskHelper.java 43 static final long serialVersionUID = 8765093759964640721L; // ## re-generate jdk/tools/jimage/JImageTask.java The start year is replaced in the copyright header. You should add the end year instead. test/java/nio/Buffer/LimitDirectMemory.sh test/java/nio/file/spi/SetDefaultProvider.java Why this test is ignored? Mandy > On Jun 17, 2015, at 5:08 PM, Jim Laskey (Oracle) wrote: > > https://bugs.openjdk.java.net/browse/JDK-8080511 > > This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. > > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools > > > Details: > > - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. > - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 > - A new archive abstraction for class/resource sources. > - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) > - JNI support for jimage into hotspot. > - White box tests written to exercise native jimage support. > From nipa at codefx.org Fri Jun 19 08:38:24 2015 From: nipa at codefx.org (Nicolai Parlog) Date: Fri, 19 Jun 2015 10:38:24 +0200 Subject: Multiple Versions vs "Module Hell" Message-ID: <5583D500.60603@codefx.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi! I am writing a series about Project Jigsaw[1] and am currently drafting an article about the planned features[2, 3]. One paragraph in particular made me wonder: "It is not necessary to support more than one version of a module within a single configuration."[4] At first sight this looks like it would create "Module Hell" instead of JAR Hell because instead of multiple JARs depending on different versions of the same JAR, we can now have multiple modules depending on different versions of the same module. I guess this is not so but I don't really see why not. I assume encapsulating the dependencies is the answer but it would be great to have a less speculative and vague explanation. Maybe someone on this list can help me out? Thanks in advance Nicolai [1] blog.codefx.org/java/dev/motivation-goals-project-jigsaw/ [2] openjdk.java.net/projects/jigsaw/goals-reqs/ [3] http://openjdk.java.net/projects/jigsaw/spec/reqs/02 [4] http://openjdk.java.net/projects/jigsaw/goals-reqs/03#multiple-versions - -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJVg9T/AAoJEMo7rS6czNUJxIgP/AwvojqRU2prRebBKKPCmxce ZwdozRccbV0ubmGOmezm15JTRThMNWzW6TOHoua5m2i+Iuv5G0CNsAeJI9PoDiS1 gC0TCOR3A0rTmtZZJcgqXsFDum7XFIpBLGz9BQEdyO9vjYRkbKHMI03TG2ITwz4H O9FJ/TtXxFALFPaF+yZ7jJbMfh9XFOlcNKBQwxJow+6N5xwuQdKQhv4Si7UbysRO 8yZsAd2p8PsbpXrV0qVhHA2VaCSGoORt6xhNOAhFnBGH1m+3cyvjCPFZdvU4PttV 0+2Tbcn+fh1emU7JJ7ChsW1rnTRpVDg4BYnb21cntHweqpMEIRSYlpYon2ZzTQee SWZwZ2CGhTWTGNXx4hfJjYmlA88vtHPMkK4h2cAIO6z0gF/3Hv83T2wa80zgpADk YMVvhOZUrHXmhaq5WP/CB1xn6RADuVv66WN0qVEhI5HNIowZ8rSuFPuE8T/ezFTi kMociRdTpeE2YJz9y1bsiQ1j5zIptOXxvp9rplufmMww3sAOc3pdlLXjb1zoSwpk SezNcW1SVMO74QMqjKXuaZXMiMrC2GU9ttrbUHRTPufTY3ggCaC+4RY1CTHgKnWL 09Xolp/F9HVoS7Odhml21fLSUzq/7Z6usO4OAL4KFjY/4Q7o9a+gm8N1orrCAb+N 1bRa20FV33h38n4N12Bh =lvRd -----END PGP SIGNATURE----- From david.lloyd at redhat.com Fri Jun 19 13:17:32 2015 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 19 Jun 2015 08:17:32 -0500 Subject: Multiple Versions vs "Module Hell" In-Reply-To: <5583D500.60603@codefx.org> References: <5583D500.60603@codefx.org> Message-ID: <5584166C.5070902@redhat.com> It really comes down to the definition of "more than one version of a module". What is the difference between more than one version of a module, and to different modules which happen to have very similar code in them? On 06/19/2015 03:38 AM, Nicolai Parlog wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi! > > I am writing a series about Project Jigsaw[1] and am currently > drafting an article about the planned features[2, 3]. > > One paragraph in particular made me wonder: > > "It is not necessary to support more than one version of a module > within a single configuration."[4] > > At first sight this looks like it would create "Module Hell" instead > of JAR Hell because instead of multiple JARs depending on different > versions of the same JAR, we can now have multiple modules depending > on different versions of the same module. > > I guess this is not so but I don't really see why not. I assume > encapsulating the dependencies is the answer but it would be great to > have a less speculative and vague explanation. Maybe someone on this > list can help me out? > > Thanks in advance > Nicolai > > > [1] blog.codefx.org/java/dev/motivation-goals-project-jigsaw/ > [2] openjdk.java.net/projects/jigsaw/goals-reqs/ > [3] http://openjdk.java.net/projects/jigsaw/spec/reqs/02 > [4] > http://openjdk.java.net/projects/jigsaw/goals-reqs/03#multiple-versions > > > > - -- > > PGP Key: > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > Web: > http://codefx.org > a blog about software development > http://do-foss.de > Free and Open Source Software for the City of Dortmund > > Twitter: > https://twitter.com/nipafx > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.22 (MingW32) > > iQIcBAEBAgAGBQJVg9T/AAoJEMo7rS6czNUJxIgP/AwvojqRU2prRebBKKPCmxce > ZwdozRccbV0ubmGOmezm15JTRThMNWzW6TOHoua5m2i+Iuv5G0CNsAeJI9PoDiS1 > gC0TCOR3A0rTmtZZJcgqXsFDum7XFIpBLGz9BQEdyO9vjYRkbKHMI03TG2ITwz4H > O9FJ/TtXxFALFPaF+yZ7jJbMfh9XFOlcNKBQwxJow+6N5xwuQdKQhv4Si7UbysRO > 8yZsAd2p8PsbpXrV0qVhHA2VaCSGoORt6xhNOAhFnBGH1m+3cyvjCPFZdvU4PttV > 0+2Tbcn+fh1emU7JJ7ChsW1rnTRpVDg4BYnb21cntHweqpMEIRSYlpYon2ZzTQee > SWZwZ2CGhTWTGNXx4hfJjYmlA88vtHPMkK4h2cAIO6z0gF/3Hv83T2wa80zgpADk > YMVvhOZUrHXmhaq5WP/CB1xn6RADuVv66WN0qVEhI5HNIowZ8rSuFPuE8T/ezFTi > kMociRdTpeE2YJz9y1bsiQ1j5zIptOXxvp9rplufmMww3sAOc3pdlLXjb1zoSwpk > SezNcW1SVMO74QMqjKXuaZXMiMrC2GU9ttrbUHRTPufTY3ggCaC+4RY1CTHgKnWL > 09Xolp/F9HVoS7Odhml21fLSUzq/7Z6usO4OAL4KFjY/4Q7o9a+gm8N1orrCAb+N > 1bRa20FV33h38n4N12Bh > =lvRd > -----END PGP SIGNATURE----- > -- - DML From Alan.Bateman at oracle.com Fri Jun 19 13:27:49 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Jun 2015 14:27:49 +0100 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <5E615ED4-B352-4C71-9F67-273BC08BBD37@oracle.com> References: <5E615ED4-B352-4C71-9F67-273BC08BBD37@oracle.com> Message-ID: <558418D5.4020609@oracle.com> On 18/06/2015 19:48, Mandy Chung wrote: > : > > test/java/nio/Buffer/LimitDirectMemory.sh > test/java/nio/file/spi/SetDefaultProvider.java > Why this test is ignored? > > For LimitDirectMemory.sh then the test needs to be adjusted to take account of the direct buffers that the jimage code is created. We can do this after Jim's push if it makes things easier. The SetDefaultProvider.java test is more tricky. I suggested to Jim that he exclude it temporarily because interposing on the default provider with this version of jimage code will lead to recursive initialization issues. Once we get to one jimage file and replacing the ext/app class loaders then it will become a non-issue and we liberate the test then. -Alan From mandy.chung at oracle.com Fri Jun 19 14:26:50 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 19 Jun 2015 07:26:50 -0700 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <558418D5.4020609@oracle.com> References: <5E615ED4-B352-4C71-9F67-273BC08BBD37@oracle.com> <558418D5.4020609@oracle.com> Message-ID: <06B43F38-88C4-4396-B199-CD97705AB3C2@oracle.com> > On Jun 19, 2015, at 6:27 AM, Alan Bateman wrote: > > > > On 18/06/2015 19:48, Mandy Chung wrote: >> : >> >> test/java/nio/Buffer/LimitDirectMemory.sh >> test/java/nio/file/spi/SetDefaultProvider.java >> Why this test is ignored? >> >> > For LimitDirectMemory.sh then the test needs to be adjusted to take account of the direct buffers that the jimage code is created. We can do this after Jim's push if it makes things easier. > > The SetDefaultProvider.java test is more tricky. I suggested to Jim that he exclude it temporarily because interposing on the default provider with this version of jimage code will lead to recursive initialization issues. Once we get to one jimage file and replacing the ext/app class loaders then it will become a non-issue and we liberate the test then. It?s fine to exclude them. Jean-Francois - can you file a bug and add the bug number next to @ignore? Mandy From paul.sandoz at oracle.com Fri Jun 19 14:39:38 2015 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 19 Jun 2015 16:39:38 +0200 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: References: Message-ID: <165AE6EA-8FB7-4715-8D76-477692B63C95@oracle.com> On Jun 18, 2015, at 2:08 AM, Jim Laskey (Oracle) wrote: > https://bugs.openjdk.java.net/browse/JDK-8080511 > > This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. > > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk I eyeballed the JDK changes. Just two thing: make/src/classes/build/tools/module/ModuleArchive.java 90 @Override 91 public Stream entries() { 92 List entries = new ArrayList<>(); 93 try { 94 if (classes != null) 95 Files.walk(classes) 96 .sorted() 97 .filter(p -> !Files.isDirectory(p) 98 && !classes.relativize(p).toString().startsWith("_the.") 99 && !classes.relativize(p).toString().equals("javac_state")) 100 .map(p -> toEntry(p, classes, EntryType.CLASS_OR_RESOURCE)) 101 .forEach(entries::add); 102 if (cmds != null) 103 Files.walk(cmds) 104 .filter(p -> !Files.isDirectory(p)) 105 .map(p -> toEntry(p, cmds, EntryType.NATIVE_CMD)) 106 .forEach(entries::add); 107 if (libs != null) 108 Files.walk(libs) 109 .filter(p -> !Files.isDirectory(p)) 110 .map(p -> toEntry(p, libs, EntryType.NATIVE_LIB)) 111 .forEach(entries::add); 112 if (configs != null) 113 Files.walk(configs) 114 .filter(p -> !Files.isDirectory(p)) 115 .map(p -> toEntry(p, configs, EntryType.CONFIG)) 116 .forEach(entries::add); 117 } catch (IOException ioe) { 118 throw new UncheckedIOException(ioe); 119 } 120 return entries.stream(); 121 } You can use collect(toList()) In general the contract for Archive.entries probably has to say the stream needs to be closed after use since it might cover lazy I/O based resources, so callers will need to use a try-with-resources block. Paul. > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools > > > Details: > > - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. > - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 > - A new archive abstraction for class/resource sources. > - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) > - JNI support for jimage into hotspot. > - White box tests written to exercise native jimage support. > From nipa at codefx.org Sat Jun 20 11:59:22 2015 From: nipa at codefx.org (Nicolai Parlog) Date: Sat, 20 Jun 2015 13:59:22 +0200 Subject: Multiple Versions vs "Module Hell" In-Reply-To: <5584166C.5070902@redhat.com> References: <5583D500.60603@codefx.org> <5584166C.5070902@redhat.com> Message-ID: <5585559A.9060400@codefx.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi David, thank you for your answer. It was a very good one as it nudged me into the right direction but still made me think. ;) Here's what I came up with: > What is the difference between more than one version of a module, > and to different modules which happen to have very similar code in > them? Good point. Since "[m]ultiple modules may export packages of the same name"[1] and the JVM "must prevent code from accessing classes and interfaces [...] in packages whose containing modules are not required by the module containing the code", we have everything we need. Now we just have to act as if the two versions are two different modules and we're done. If the runtime does not support this directly, we could still do it manually by creating a new module for each version, which simply reexports all published code, and redirect the dependencies to those modules instead of the original one. Thanks ... Nicolai [1] http://openjdk.java.net/projects/jigsaw/spec/reqs/02#exports [2] http://openjdk.java.net/projects/jigsaw/spec/reqs/02#encapsulation On 19.06.2015 15:17, David M. Lloyd wrote: > It really comes down to the definition of "more than one version of > a module". What is the difference between more than one version of > a module, and to different modules which happen to have very > similar code in them? > > On 06/19/2015 03:38 AM, Nicolai Parlog wrote: Hi! > > I am writing a series about Project Jigsaw[1] and am currently > drafting an article about the planned features[2, 3]. > > One paragraph in particular made me wonder: > > "It is not necessary to support more than one version of a module > within a single configuration."[4] > > At first sight this looks like it would create "Module Hell" > instead of JAR Hell because instead of multiple JARs depending on > different versions of the same JAR, we can now have multiple > modules depending on different versions of the same module. > > I guess this is not so but I don't really see why not. I assume > encapsulating the dependencies is the answer but it would be great > to have a less speculative and vague explanation. Maybe someone on > this list can help me out? > > Thanks in advance Nicolai > > > [1] blog.codefx.org/java/dev/motivation-goals-project-jigsaw/ [2] > openjdk.java.net/projects/jigsaw/goals-reqs/ [3] > http://openjdk.java.net/projects/jigsaw/spec/reqs/02 [4] > http://openjdk.java.net/projects/jigsaw/goals-reqs/03#multiple-version s > > > > >> > - -- PGP Key: http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 Web: http://codefx.org a blog about software development http://do-foss.de Free and Open Source Software for the City of Dortmund Twitter: https://twitter.com/nipafx -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJVhVWaAAoJEMo7rS6czNUJBlsQAJtV7NJOtTVTFW95Do/UE6RU W0QaFo0egxyTaXGz9/nQkECR1V3lgFnw2Evhh3n6hkv8McwZmXsS9tbAJqJ8fH/A pRMioLMLoFUU9LGmPTJSo8uUinGH3+NKWytqOQ8bgrsEFg08+GFV6ZHdB9JDaQOh htDpW/4/7v786g7rHgVQ41yOD3tuPbvvHkCKjA8lKKAMX/+QPNOOC+003/x6MzR7 EAqTCEQB2+Vr3vx6cfpgYLwvv5VEPxL3jczWi3V62gTVKNYXR2iMSNfggoOCQytt KsQ3W8yDBGMbppr+zhGPwIWgcrjjUK0J8pF/HS3Ogc+upmrpDJK6HlFZJO/TG/eo KUqScJa4aHilGv4iZ3KXANsnQqLQzg3XOcdIc4ApVmP0arPGX/q2ukH8qWkcVKqS CHTo9dX4cCyl6kGVEZWP7gergiQ1YIgQHj4q4I7F5lJMY+WIpHEYkuy83oAEujQa ueZ5/HjtKD8XqeXr+tONKqJyIiqT7OmMSX0fZ9dkSoTDZ9t4AXRlViabSCbL+NqM +j8Wp1MjVxDuKlk8Al5JBHT7QKRmi2qWUGR1BNK5z7nLAiTVNVLEJdppqbU1cnoN 1ESb5ED9J9Cmh3oCzg3NHyPNaaG5a2/DVhpxU5vSdjh+R+y3ceRUs0qYMyGliPwt cApKM/dH6vQlFasAw6XU =/Nwq -----END PGP SIGNATURE----- From peter.firmstone at zeus.net.au Mon Jun 22 12:32:40 2015 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Mon, 22 Jun 2015 22:32:40 +1000 Subject: Multiple Versions vs "Module Hell" In-Reply-To: <5585559A.9060400@codefx.org> References: <5583D500.60603@codefx.org> <5584166C.5070902@redhat.com> <5585559A.9060400@codefx.org> Message-ID: <55880068.3090507@zeus.net.au> Considering Java allows interfaces to evolve, with default methods, would it be beneficial to provide the ability in the jigsaw api to create API modules, with the intent of selecting common public api for backward compatible evolution, while also allowing different implementation modules, for whatever reason, to co-exist in the same runtime (loaded by different ClassLoaders). Clients of the api may then be able to access the underlying module directly to access extending features; it encourages a common api for maximum sharing and compatibility. Just a thought. Cheers, Peter. On 20/06/2015 9:59 PM, Nicolai Parlog wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi David, > > thank you for your answer. It was a very good one as it nudged me into > the right direction but still made me think. ;) > > Here's what I came up with: > >> What is the difference between more than one version of a module, >> and to different modules which happen to have very similar code in >> them? > Good point. Since "[m]ultiple modules may export packages of the same > name"[1] and the JVM "must prevent code from accessing classes and > interfaces [...] in packages whose containing modules are not required > by the module containing the code", we have everything we need. > > Now we just have to act as if the two versions are two different > modules and we're done. If the runtime does not support this directly, > we could still do it manually by creating a new module for each > version, which simply reexports all published code, and redirect the > dependencies to those modules instead of the original one. > > Thanks ... Nicolai > > > [1] http://openjdk.java.net/projects/jigsaw/spec/reqs/02#exports > [2] http://openjdk.java.net/projects/jigsaw/spec/reqs/02#encapsulation > > > > On 19.06.2015 15:17, David M. Lloyd wrote: >> It really comes down to the definition of "more than one version of >> a module". What is the difference between more than one version of >> a module, and to different modules which happen to have very >> similar code in them? >> >> On 06/19/2015 03:38 AM, Nicolai Parlog wrote: Hi! >> >> I am writing a series about Project Jigsaw[1] and am currently >> drafting an article about the planned features[2, 3]. >> >> One paragraph in particular made me wonder: >> >> "It is not necessary to support more than one version of a module >> within a single configuration."[4] >> >> At first sight this looks like it would create "Module Hell" >> instead of JAR Hell because instead of multiple JARs depending on >> different versions of the same JAR, we can now have multiple >> modules depending on different versions of the same module. >> >> I guess this is not so but I don't really see why not. I assume >> encapsulating the dependencies is the answer but it would be great >> to have a less speculative and vague explanation. Maybe someone on >> this list can help me out? >> >> Thanks in advance Nicolai >> >> >> [1] blog.codefx.org/java/dev/motivation-goals-project-jigsaw/ [2] >> openjdk.java.net/projects/jigsaw/goals-reqs/ [3] >> http://openjdk.java.net/projects/jigsaw/spec/reqs/02 [4] >> http://openjdk.java.net/projects/jigsaw/goals-reqs/03#multiple-version > s >> >> >> > - -- > > PGP Key: > http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 > > Web: > http://codefx.org > a blog about software development > http://do-foss.de > Free and Open Source Software for the City of Dortmund > > Twitter: > https://twitter.com/nipafx > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.22 (MingW32) > > iQIcBAEBAgAGBQJVhVWaAAoJEMo7rS6czNUJBlsQAJtV7NJOtTVTFW95Do/UE6RU > W0QaFo0egxyTaXGz9/nQkECR1V3lgFnw2Evhh3n6hkv8McwZmXsS9tbAJqJ8fH/A > pRMioLMLoFUU9LGmPTJSo8uUinGH3+NKWytqOQ8bgrsEFg08+GFV6ZHdB9JDaQOh > htDpW/4/7v786g7rHgVQ41yOD3tuPbvvHkCKjA8lKKAMX/+QPNOOC+003/x6MzR7 > EAqTCEQB2+Vr3vx6cfpgYLwvv5VEPxL3jczWi3V62gTVKNYXR2iMSNfggoOCQytt > KsQ3W8yDBGMbppr+zhGPwIWgcrjjUK0J8pF/HS3Ogc+upmrpDJK6HlFZJO/TG/eo > KUqScJa4aHilGv4iZ3KXANsnQqLQzg3XOcdIc4ApVmP0arPGX/q2ukH8qWkcVKqS > CHTo9dX4cCyl6kGVEZWP7gergiQ1YIgQHj4q4I7F5lJMY+WIpHEYkuy83oAEujQa > ueZ5/HjtKD8XqeXr+tONKqJyIiqT7OmMSX0fZ9dkSoTDZ9t4AXRlViabSCbL+NqM > +j8Wp1MjVxDuKlk8Al5JBHT7QKRmi2qWUGR1BNK5z7nLAiTVNVLEJdppqbU1cnoN > 1ESb5ED9J9Cmh3oCzg3NHyPNaaG5a2/DVhpxU5vSdjh+R+y3ceRUs0qYMyGliPwt > cApKM/dH6vQlFasAw6XU > =/Nwq > -----END PGP SIGNATURE----- > From niftiness at gmail.com Mon Jun 22 21:46:19 2015 From: niftiness at gmail.com (Tim Boudreau) Date: Mon, 22 Jun 2015 17:46:19 -0400 Subject: Multiple Versions vs "Module Hell" In-Reply-To: <5583D500.60603@codefx.org> References: <5583D500.60603@codefx.org> Message-ID: On Fri, Jun 19, 2015 at 4:38 AM, Nicolai Parlog wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi! > > I am writing a series about Project Jigsaw[1] and am currently > drafting an article about the planned features[2, 3]. > > One paragraph in particular made me wonder: > > "It is not necessary to support more than one version of a module > within a single configuration."[4] > > At first sight this looks like it would create "Module Hell" instead > of JAR Hell because instead of multiple JARs depending on different > versions of the same JAR, we can now have multiple modules depending > on different versions of the same module. > > I guess this is not so but I don't really see why not. I assume > encapsulating the dependencies is the answer but it would be great to > have a less speculative and vague explanation. Maybe someone on this > list can help me out? > (reposting this to the list since I inadvertently replied only to Nicolai) The key phrase is "It is not necessary". That doesn't mean "it is prohibited", just that a module system is not *required* to support doing that to pass the test of being a faithful implementation of the module system spec. Many existing module systems do - for example, NetBeans module system (very similar to OSGi but created several years earlier) allows that where it can prove objects from library A-v1 can't possibly get mixed up with objects from A-v2. It's the sort of thing that's useful in large, modular applications that are composed from parts that are dynamically loaded. But it's just pointless overhead in a small, self-contained application that's never going to have those sorts of problems to solve. So it makes sense, in defining the scope of the spec, that there's no *requirement *to support that scenario. There's no prohibition either. As long as there are ways to hook into the module discovery / loading process, anybody who wants that can implement it, without it needing to be dictated by the spec. -Tim From martijnverburg at gmail.com Tue Jun 23 09:34:41 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Tue, 23 Jun 2015 10:34:41 +0100 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? Message-ID: Hi all, As part of the Adoption Group we're planning on holding a Hackday in London for the July/Aug timeframe, primarily as an education/awareness piece around Jigsaw (there's lots to catch up on). Is there anything in particular you'd like a group of 'users'/developers to look at? I assume basing this off the latest jdk9 forest will be more stable than the jigsaw forest. Cheers, Martijn From Alan.Bateman at oracle.com Tue Jun 23 11:10:04 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Jun 2015 12:10:04 +0100 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: References: Message-ID: <55893E8C.5050600@oracle.com> On 23/06/2015 10:34, Martijn Verburg wrote: > Hi all, > > As part of the Adoption Group we're planning on holding a Hackday in London > for the July/Aug timeframe, primarily as an education/awareness piece > around Jigsaw (there's lots to catch up on). > > Is there anything in particular you'd like a group of 'users'/developers to > look at? I assume basing this off the latest jdk9 forest will be more > stable than the jigsaw forest. > I would expect/hope that JSR 376 will be further along by then so there may be more to talk about and play with at this event. In the mean-time there is a lot to discuss and prepare. We've been warning (and javac has emitted warnings) for many years that JDK-internal APIs will not be accessible out of the box so anything that reduces or eliminates dependences on internal APIs in popular libraries will be a big help. It might be fun to use jdeps on the say the 20 most popular libraries to get a feel for the problem. We've had the initial changes for JEP 220 in JDK 9 for about 6 months. This is the JEP that changes the structure of the run-time images. This has implications for tools (mostly) that have been used to directly accessing rt.jar and other internal files in the legacy images. JEP 220 comes with a supported interface for accessing classes and resources in the run-time image so that may be worth spending time on. There is a refresh of this in flight and it should be in by the time you have this event. It might be fun to try out the top 20 tools and plugins to see if they run with JDK 9 and maybe even hack on one or two of them to use the FileSystem API. Another idea is to look at the module graph in JEP 200, or better still, the most up to date graph of modules in the JDK 9 forests. One idea is to hack on make/Images.gmk in the top-level repository to force the JDK build to produce run-time with a subset of the modules. The primitive image building tool that this make file runs will be replaced by a linker tool in time but there is enough to hack on to get a feel for the modules and what might run (or not run) with a small set of modules. The module graph is essentially an API so any feedback or usage of that API would be useful too. -Alan From martijnverburg at gmail.com Tue Jun 23 11:28:30 2015 From: martijnverburg at gmail.com (Martijn Verburg) Date: Tue, 23 Jun 2015 12:28:30 +0100 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: <55893E8C.5050600@oracle.com> References: <55893E8C.5050600@oracle.com> Message-ID: Hi Alan, That's brilliant, we'll set up those three broad areas and see if we can get small groups to work on each. Cheers, Martijn On 23 June 2015 at 12:10, Alan Bateman wrote: > On 23/06/2015 10:34, Martijn Verburg wrote: > >> Hi all, >> >> As part of the Adoption Group we're planning on holding a Hackday in >> London >> for the July/Aug timeframe, primarily as an education/awareness piece >> around Jigsaw (there's lots to catch up on). >> >> Is there anything in particular you'd like a group of 'users'/developers >> to >> look at? I assume basing this off the latest jdk9 forest will be more >> stable than the jigsaw forest. >> >> I would expect/hope that JSR 376 will be further along by then so there > may be more to talk about and play with at this event. > > In the mean-time there is a lot to discuss and prepare. We've been warning > (and javac has emitted warnings) for many years that JDK-internal APIs will > not be accessible out of the box so anything that reduces or eliminates > dependences on internal APIs in popular libraries will be a big help. It > might be fun to use jdeps on the say the 20 most popular libraries to get a > feel for the problem. > > We've had the initial changes for JEP 220 in JDK 9 for about 6 months. > This is the JEP that changes the structure of the run-time images. This has > implications for tools (mostly) that have been used to directly accessing > rt.jar and other internal files in the legacy images. JEP 220 comes with a > supported interface for accessing classes and resources in the run-time > image so that may be worth spending time on. There is a refresh of this in > flight and it should be in by the time you have this event. It might be fun > to try out the top 20 tools and plugins to see if they run with JDK 9 and > maybe even hack on one or two of them to use the FileSystem API. > > Another idea is to look at the module graph in JEP 200, or better still, > the most up to date graph of modules in the JDK 9 forests. One idea is to > hack on make/Images.gmk in the top-level repository to force the JDK build > to produce run-time with a subset of the modules. The primitive image > building tool that this make file runs will be replaced by a linker tool in > time but there is enough to hack on to get a feel for the modules and what > might run (or not run) with a small set of modules. The module graph is > essentially an API so any feedback or usage of that API would be useful too. > > -Alan > From jean-francois.denise at oracle.com Tue Jun 23 12:03:23 2015 From: jean-francois.denise at oracle.com (Jean-Francois Denise) Date: Tue, 23 Jun 2015 14:03:23 +0200 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <23CC0B1E-BDE7-4E48-8A22-B0C19AA74AF8@oracle.com> References: <23CC0B1E-BDE7-4E48-8A22-B0C19AA74AF8@oracle.com> Message-ID: <9DE4FC35-D75D-49A5-BFDF-336610D6396A@oracle.com> Hi Karen, On 18 Jun 2015, at 16:59, Karen Kinnear wrote: > Code review for the hotspot component: > > Thank you for the cleanups to jvm.cpp to make them JVM_ENTRY and fix thread_state. > You are welcome. > If you pass the testing and Lois ok's the hotspot code, then I am ok with checking in the > hotspot code as is. > Thanks. > Comments for next round of changes: > > 1. mutexLocker.cpp - new lock should say "false" not "true" for whether the VMThread should block Done, along with not-leaf. > > 2. os.hpp > restartable_read_at is added but I don't see it used Removed. > > 3. imageFile.cpp > We've been asked to always used the counted form of strcpy - i.e. strncpy to ensure we never > have buffer overruns. e.g. 311, 437, ... > > 4. jvm.cpp - seems odd that the interface would need to know if the resource was compressed > or uncompressed. In the next round of API design, seems like the java code would pass in > the uncompressedAddress and let the internals handle buffering the compressed data and > doing the uncompressing transparently. > > 5. imageDecompressor.hpp > get_decompressor > I think you've mixed your exception handling approaches. You have a CHECK_NULL for new_symbol > call, which means it will throw an exception if it returns NULL. Then you check for HAS_PENDING_EXCEPTION > which is the logic you would use if did not have the CHECK, but then you throw the exception away. > If you want to throw an exception from get_decompressor you should pass in the last argument as TRAPS macro > which will give you the THREAD. > > see macros in exceptions.hpp > > So the way it is right now, you will throw an exception. > > You have a choice - you can throw an exception > pass in TRAPS to get_decompressor, keep the CHECK_NULL and remove the if HAS_PENDING_EXCEPTION. > - this is the recommended approach > > Alternative - change CHECK_NULL to pass in THREAD. Keep the rest. Given you are planning to take this > out of the vm - this would make more sense. > Thanks, we will implement 3/4/5 in the next round. > thanks, > Karen > > > > On Jun 17, 2015, at 8:08 PM, Jim Laskey (Oracle) wrote: > >> https://bugs.openjdk.java.net/browse/JDK-8080511 >> >> This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. >> >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools >> >> >> Details: >> >> - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. >> - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 >> - A new archive abstraction for class/resource sources. >> - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) >> - JNI support for jimage into hotspot. >> - White box tests written to exercise native jimage support. >> > From jean-francois.denise at oracle.com Tue Jun 23 12:03:53 2015 From: jean-francois.denise at oracle.com (Jean-Francois Denise) Date: Tue, 23 Jun 2015 14:03:53 +0200 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <55830606.50500@oracle.com> References: <23CC0B1E-BDE7-4E48-8A22-B0C19AA74AF8@oracle.com> <55830606.50500@oracle.com> Message-ID: Hi Lois, On 18 Jun 2015, at 19:55, Lois Foltan wrote: > > On 6/18/2015 10:59 AM, Karen Kinnear wrote: >> Code review for the hotspot component: >> >> Thank you for the cleanups to jvm.cpp to make them JVM_ENTRY and fix thread_state. >> >> If you pass the testing and Lois ok's the hotspot code, then I am ok with checking in the >> hotspot code as is. > > I have reviewed and am okay with checking in the hotspot code as is once testing is complete. One comment inlined below. > Lois > >> >> Comments for next round of changes: >> >> 1. mutexLocker.cpp - new lock should say "false" not "true" for whether the VMThread should block >> >> 2. os.hpp >> restartable_read_at is added but I don't see it used >> >> 3. imageFile.cpp >> We've been asked to always used the counted form of strcpy - i.e. strncpy to ensure we never >> have buffer overruns. e.g. 311, 437, ... >> >> 4. jvm.cpp - seems odd that the interface would need to know if the resource was compressed >> or uncompressed. In the next round of API design, seems like the java code would pass in >> the uncompressedAddress and let the internals handle buffering the compressed data and >> doing the uncompressing transparently. >> >> 5. imageDecompressor.hpp >> get_decompressor >> I think you've mixed your exception handling approaches. You have a CHECK_NULL for new_symbol >> call, which means it will throw an exception if it returns NULL. Then you check for HAS_PENDING_EXCEPTION >> which is the logic you would use if did not have the CHECK, but then you throw the exception away. >> If you want to throw an exception from get_decompressor you should pass in the last argument as TRAPS macro >> which will give you the THREAD. >> >> see macros in exceptions.hpp >> >> So the way it is right now, you will throw an exception. >> >> You have a choice - you can throw an exception >> pass in TRAPS to get_decompressor, keep the CHECK_NULL and remove the if HAS_PENDING_EXCEPTION. >> - this is the recommended approach >> >> Alternative - change CHECK_NULL to pass in THREAD. Keep the rest. Given you are planning to take this >> out of the vm - this would make more sense. > > Whatever mechanism you choose to use in get_decompressor, please also use for image_decompressor_init( )/createSymbol() within imageDecompressor.cpp as well. OK, well noted. Thanks. JF > >> >> thanks, >> Karen >> >> >> >> On Jun 17, 2015, at 8:08 PM, Jim Laskey (Oracle) wrote: >> >>> https://bugs.openjdk.java.net/browse/JDK-8080511 >>> >>> This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. >>> >>> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top >>> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk >>> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot >>> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools >>> >>> >>> Details: >>> >>> - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. >>> - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 >>> - A new archive abstraction for class/resource sources. >>> - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) >>> - JNI support for jimage into hotspot. >>> - White box tests written to exercise native jimage support. >>> > From jean-francois.denise at oracle.com Tue Jun 23 12:05:08 2015 From: jean-francois.denise at oracle.com (Jean-Francois Denise) Date: Tue, 23 Jun 2015 14:05:08 +0200 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <5E615ED4-B352-4C71-9F67-273BC08BBD37@oracle.com> References: <5E615ED4-B352-4C71-9F67-273BC08BBD37@oracle.com> Message-ID: On 18 Jun 2015, at 20:48, Mandy Chung wrote: > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top > modules.xml looks fine. > > test/lib/sun/hotspot/WhiteBox.java > This adds the native entry points. Where are they implemented? It?s not obvious to me. The native is in hotspot repo vm/prims > > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools > the jdeps change looks fine. > > http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk/ > This is a huge patch that I have to rely on the extensive testing to catch potential issues. I tried running the jrt-fs tests using JDK 8 with jrt-fs.jar that work well. Some comments and mostly minor. > > make/gensrc/Gensrc-jdk.dev.gmk > - you could revert this one Done > > make/src/classes/build/tools/module/ImageBuilder.java > Nit: line 366, 382 - left over commented lines > some indentation/formatting nits line 386-397 Done > > jdk/tools/jimage/TaskHelper.java > 43 static final long serialVersionUID = 8765093759964640721L; // ## re-generate Removed the comment > > jdk/tools/jimage/JImageTask.java > The start year is replaced in the copyright header. You should add the end year instead. Done > > test/java/nio/Buffer/LimitDirectMemory.sh > test/java/nio/file/spi/SetDefaultProvider.java > Why this test is ignored? Done, JDK bug logged. > > Mandy Thanks. JF > >> On Jun 17, 2015, at 5:08 PM, Jim Laskey (Oracle) wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8080511 >> >> This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. >> >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools >> >> >> Details: >> >> - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. >> - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 >> - A new archive abstraction for class/resource sources. >> - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) >> - JNI support for jimage into hotspot. >> - White box tests written to exercise native jimage support. >> > From jean-francois.denise at oracle.com Tue Jun 23 12:05:59 2015 From: jean-francois.denise at oracle.com (Jean-Francois Denise) Date: Tue, 23 Jun 2015 14:05:59 +0200 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <06B43F38-88C4-4396-B199-CD97705AB3C2@oracle.com> References: <5E615ED4-B352-4C71-9F67-273BC08BBD37@oracle.com> <558418D5.4020609@oracle.com> <06B43F38-88C4-4396-B199-CD97705AB3C2@oracle.com> Message-ID: <87A58F30-067C-4DAC-B7F7-4F59AFD52F55@oracle.com> Hi Mandy, I logged https://bugs.openjdk.java.net/browse/JDK-8129343 Thanks. JF On 19 Jun 2015, at 16:26, Mandy Chung wrote: > >> On Jun 19, 2015, at 6:27 AM, Alan Bateman wrote: >> >> >> >> On 18/06/2015 19:48, Mandy Chung wrote: >>> : >>> >>> test/java/nio/Buffer/LimitDirectMemory.sh >>> test/java/nio/file/spi/SetDefaultProvider.java >>> Why this test is ignored? >>> >>> >> For LimitDirectMemory.sh then the test needs to be adjusted to take account of the direct buffers that the jimage code is created. We can do this after Jim's push if it makes things easier. >> >> The SetDefaultProvider.java test is more tricky. I suggested to Jim that he exclude it temporarily because interposing on the default provider with this version of jimage code will lead to recursive initialization issues. Once we get to one jimage file and replacing the ext/app class loaders then it will become a non-issue and we liberate the test then. > > It?s fine to exclude them. Jean-Francois - can you file a bug and add the bug number next to @ignore? > > Mandy > From jean-francois.denise at oracle.com Tue Jun 23 12:10:11 2015 From: jean-francois.denise at oracle.com (Jean-Francois Denise) Date: Tue, 23 Jun 2015 14:10:11 +0200 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <165AE6EA-8FB7-4715-8D76-477692B63C95@oracle.com> References: <165AE6EA-8FB7-4715-8D76-477692B63C95@oracle.com> Message-ID: <69B55DAD-6E30-4703-B2B0-4A5EEC598149@oracle.com> Hi Paul, On 19 Jun 2015, at 16:39, Paul Sandoz wrote: > > On Jun 18, 2015, at 2:08 AM, Jim Laskey (Oracle) wrote: > >> https://bugs.openjdk.java.net/browse/JDK-8080511 >> >> This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. >> >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk > > I eyeballed the JDK changes. Just two thing: > > make/src/classes/build/tools/module/ModuleArchive.java > > 90 @Override > 91 public Stream entries() { > 92 List entries = new ArrayList<>(); > 93 try { > 94 if (classes != null) > 95 Files.walk(classes) > 96 .sorted() > 97 .filter(p -> !Files.isDirectory(p) > 98 && !classes.relativize(p).toString().startsWith("_the.") > 99 && !classes.relativize(p).toString().equals("javac_state")) > 100 .map(p -> toEntry(p, classes, EntryType.CLASS_OR_RESOURCE)) > 101 .forEach(entries::add); > 102 if (cmds != null) > 103 Files.walk(cmds) > 104 .filter(p -> !Files.isDirectory(p)) > 105 .map(p -> toEntry(p, cmds, EntryType.NATIVE_CMD)) > 106 .forEach(entries::add); > 107 if (libs != null) > 108 Files.walk(libs) > 109 .filter(p -> !Files.isDirectory(p)) > 110 .map(p -> toEntry(p, libs, EntryType.NATIVE_LIB)) > 111 .forEach(entries::add); > 112 if (configs != null) > 113 Files.walk(configs) > 114 .filter(p -> !Files.isDirectory(p)) > 115 .map(p -> toEntry(p, configs, EntryType.CONFIG)) > 116 .forEach(entries::add); > 117 } catch (IOException ioe) { > 118 throw new UncheckedIOException(ioe); > 119 } > 120 return entries.stream(); > 121 } > > You can use collect(toList()) ==> OK collect used. In addition, filter first then sort, tryWithResource for 4 Files stream. > > In general the contract for Archive.entries probably has to say the stream needs to be closed after use since it might cover lazy I/O based resources, so callers will need to use a try-with-resources block. ==> Added a note in javadoc, implemented explicit close for non lazy streams, added missing tryWithResource. Added a comment on what should be done in ModuleArchive to keep laziness. > > Paul. Thanks. > >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot >> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools >> >> >> Details: >> >> - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. >> - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 >> - A new archive abstraction for class/resource sources. >> - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) >> - JNI support for jimage into hotspot. >> - White box tests written to exercise native jimage support. >> > From vitalyd at gmail.com Tue Jun 23 13:43:02 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 23 Jun 2015 09:43:02 -0400 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: <55893E8C.5050600@oracle.com> References: <55893E8C.5050600@oracle.com> Message-ID: Hi Alan, Regarding use of internal APIs - what is the plan for Unsafe? I recall some discussion about possibly making it a proper public API, but don't recall seeing anything finalized. Thanks sent from my phone On Jun 23, 2015 7:10 AM, "Alan Bateman" wrote: > On 23/06/2015 10:34, Martijn Verburg wrote: > >> Hi all, >> >> As part of the Adoption Group we're planning on holding a Hackday in >> London >> for the July/Aug timeframe, primarily as an education/awareness piece >> around Jigsaw (there's lots to catch up on). >> >> Is there anything in particular you'd like a group of 'users'/developers >> to >> look at? I assume basing this off the latest jdk9 forest will be more >> stable than the jigsaw forest. >> >> I would expect/hope that JSR 376 will be further along by then so there > may be more to talk about and play with at this event. > > In the mean-time there is a lot to discuss and prepare. We've been warning > (and javac has emitted warnings) for many years that JDK-internal APIs will > not be accessible out of the box so anything that reduces or eliminates > dependences on internal APIs in popular libraries will be a big help. It > might be fun to use jdeps on the say the 20 most popular libraries to get a > feel for the problem. > > We've had the initial changes for JEP 220 in JDK 9 for about 6 months. > This is the JEP that changes the structure of the run-time images. This has > implications for tools (mostly) that have been used to directly accessing > rt.jar and other internal files in the legacy images. JEP 220 comes with a > supported interface for accessing classes and resources in the run-time > image so that may be worth spending time on. There is a refresh of this in > flight and it should be in by the time you have this event. It might be fun > to try out the top 20 tools and plugins to see if they run with JDK 9 and > maybe even hack on one or two of them to use the FileSystem API. > > Another idea is to look at the module graph in JEP 200, or better still, > the most up to date graph of modules in the JDK 9 forests. One idea is to > hack on make/Images.gmk in the top-level repository to force the JDK build > to produce run-time with a subset of the modules. The primitive image > building tool that this make file runs will be replaced by a linker tool in > time but there is enough to hack on to get a feel for the modules and what > might run (or not run) with a small set of modules. The module graph is > essentially an API so any feedback or usage of that API would be useful too. > > -Alan > From Alan.Bateman at oracle.com Tue Jun 23 14:19:49 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Jun 2015 15:19:49 +0100 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: References: <55893E8C.5050600@oracle.com> Message-ID: <55896B05.2080209@oracle.com> On 23/06/2015 14:43, Vitaly Davidovich wrote: > > Hi Alan, > > Regarding use of internal APIs - what is the plan for Unsafe? I > recall some discussion about possibly making it a proper public API, > but don't recall seeing anything finalized. > Check out Paul Sandoz's slides from JavaOne last year as this has the list of JEPs and efforts that are under way to reduce Unsafe usage over time: https://oracleus.activeevents.com/2014/connect/fileDownload/session/661E477DEFA4719C7A48FEF082013CC4%2FCON5150_Sandoz-j1-2014-unsafe-CON5150.pdf -Alan From vitalyd at gmail.com Tue Jun 23 14:33:24 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 23 Jun 2015 10:33:24 -0400 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: <55896B05.2080209@oracle.com> References: <55893E8C.5050600@oracle.com> <55896B05.2080209@oracle.com> Message-ID: Yup, I've seen that but that depends on a few JEPs and large undertakings that aren't part of java 9 release, AFAIK. sent from my phone On Jun 23, 2015 10:19 AM, "Alan Bateman" wrote: > > > On 23/06/2015 14:43, Vitaly Davidovich wrote: > >> >> Hi Alan, >> >> Regarding use of internal APIs - what is the plan for Unsafe? I recall >> some discussion about possibly making it a proper public API, but don't >> recall seeing anything finalized. >> >> Check out Paul Sandoz's slides from JavaOne last year as this has the > list of JEPs and efforts that are under way to reduce Unsafe usage over > time: > > > https://oracleus.activeevents.com/2014/connect/fileDownload/session/661E477DEFA4719C7A48FEF082013CC4%2FCON5150_Sandoz-j1-2014-unsafe-CON5150.pdf > > -Alan > > From mark.reinhold at oracle.com Tue Jun 23 15:32:49 2015 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 23 Jun 2015 08:32:49 -0700 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: References: , <55893E8C.5050600@oracle.com>, , <55896B05.2080209@oracle.com>, Message-ID: <20150623083249.604405@eggemoggin.niobe.net> 2015/6/23 7:33 -0700, vitalyd at gmail.com: > Yup, I've seen that but that depends on a few JEPs and large undertakings > that aren't part of java 9 release, AFAIK. Some are, but some aren't. It will be possible to continue using sun.misc.Unsafe in JDK 9, but to do so you'll have to use a new command-line flag, or the reflection API, and (as always) such usage is unsupported and at your own risk. - Mark From vitalyd at gmail.com Tue Jun 23 15:45:50 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 23 Jun 2015 11:45:50 -0400 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: <20150623083249.604405@eggemoggin.niobe.net> References: <55893E8C.5050600@oracle.com> <55896B05.2080209@oracle.com> <20150623083249.604405@eggemoggin.niobe.net> Message-ID: Yes, but until all the "safe" replacements are in place and vetted (e.g. performance is on par with Unsafe, same functionality is available, etc), I don't see the point of making it even more annoying to grab hold of. The people who are using it will continue using it until the replacements are available, and this is just going to annoy them. Personally, I'd keep it at javac complaining about its use during compilation, and only add additional flag/burden once you're ready for people to move away from it entirely. On Tue, Jun 23, 2015 at 11:32 AM, wrote: > 2015/6/23 7:33 -0700, vitalyd at gmail.com: > > Yup, I've seen that but that depends on a few JEPs and large undertakings > > that aren't part of java 9 release, AFAIK. > > Some are, but some aren't. > > It will be possible to continue using sun.misc.Unsafe in JDK 9, but to > do so you'll have to use a new command-line flag, or the reflection API, > and (as always) such usage is unsupported and at your own risk. > > - Mark > From jean-francois.denise at oracle.com Tue Jun 23 16:08:52 2015 From: jean-francois.denise at oracle.com (Jean-Francois Denise) Date: Tue, 23 Jun 2015 18:08:52 +0200 Subject: RFR: JDK-8080511 - Refresh of jimage support In-Reply-To: <69B55DAD-6E30-4703-B2B0-4A5EEC598149@oracle.com> References: <165AE6EA-8FB7-4715-8D76-477692B63C95@oracle.com> <69B55DAD-6E30-4703-B2B0-4A5EEC598149@oracle.com> Message-ID: <487562C8-9F9D-43B3-AF21-D99368852DDB@oracle.com> Hi, an updated webrev that takes into account reviews. In addition the hotspot tests have been reworked following Christian Tornqvist suggestions. top: http://cr.openjdk.java.net/~jfdenise/hs-rt-jimage.3/webrev-top/ langtools: http://cr.openjdk.java.net/~jfdenise/hs-rt-jimage.3/webrev-langtools/ hotspot: http://cr.openjdk.java.net/~jfdenise/hs-rt-jimage.3/webrev-hotspot/ jdk: http://cr.openjdk.java.net/~jfdenise/hs-rt-jimage.3/webrev-jdk/ Testing: JCK(s) tests are passing (ran only on Mac OS) Hotspot tests are passing (All platforms) Java PIT tests are passing (All platforms). Ran once on https://bugs.openjdk.java.net/browse/JDK-8129592 Thanks. JF On 23 Jun 2015, at 14:10, Jean-Francois Denise wrote: > Hi Paul, > > On 19 Jun 2015, at 16:39, Paul Sandoz wrote: > >> >> On Jun 18, 2015, at 2:08 AM, Jim Laskey (Oracle) wrote: >> >>> https://bugs.openjdk.java.net/browse/JDK-8080511 >>> >>> This is an long overdue refresh of the jimage support in the JDK9-dev repo. This includes native support for reading jimage files, improved jrt-fs (java runtime file system) support for retrieving modules and packages from the runtime, and improved performance for langtools in the presence of jrt-fs. >>> >>> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-top >>> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-jdk >> >> I eyeballed the JDK changes. Just two thing: >> >> make/src/classes/build/tools/module/ModuleArchive.java >> >> 90 @Override >> 91 public Stream entries() { >> 92 List entries = new ArrayList<>(); >> 93 try { >> 94 if (classes != null) >> 95 Files.walk(classes) >> 96 .sorted() >> 97 .filter(p -> !Files.isDirectory(p) >> 98 && !classes.relativize(p).toString().startsWith("_the.") >> 99 && !classes.relativize(p).toString().equals("javac_state")) >> 100 .map(p -> toEntry(p, classes, EntryType.CLASS_OR_RESOURCE)) >> 101 .forEach(entries::add); >> 102 if (cmds != null) >> 103 Files.walk(cmds) >> 104 .filter(p -> !Files.isDirectory(p)) >> 105 .map(p -> toEntry(p, cmds, EntryType.NATIVE_CMD)) >> 106 .forEach(entries::add); >> 107 if (libs != null) >> 108 Files.walk(libs) >> 109 .filter(p -> !Files.isDirectory(p)) >> 110 .map(p -> toEntry(p, libs, EntryType.NATIVE_LIB)) >> 111 .forEach(entries::add); >> 112 if (configs != null) >> 113 Files.walk(configs) >> 114 .filter(p -> !Files.isDirectory(p)) >> 115 .map(p -> toEntry(p, configs, EntryType.CONFIG)) >> 116 .forEach(entries::add); >> 117 } catch (IOException ioe) { >> 118 throw new UncheckedIOException(ioe); >> 119 } >> 120 return entries.stream(); >> 121 } >> >> You can use collect(toList()) > > ==> OK collect used. In addition, filter first then sort, tryWithResource for 4 Files stream. > >> >> In general the contract for Archive.entries probably has to say the stream needs to be closed after use since it might cover lazy I/O based resources, so callers will need to use a try-with-resources block. > > ==> Added a note in javadoc, implemented explicit close for non lazy streams, added missing tryWithResource. Added a comment on what should be done in ModuleArchive to keep laziness. >> >> Paul. > > Thanks. > >> >>> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-hotspot >>> http://cr.openjdk.java.net/~jlaskey/hs-rt-jimage/webrev-langtools >>> >>> >>> Details: >>> >>> - jrt-fs provides access, via the nio FileSystem API, to the classes in a .jimage file, organized by module or by package. >>> - Shared code for jimage support converted to native. Currently residing in hotspot, but will migrate to it?s own jdk library https://bugs.openjdk.java.net/browse/JDK-8087181 >>> - A new archive abstraction for class/resource sources. >>> - java based implementation layer for jimage reading to allow backport to JDK8 (jrt-fs.jar - IDE support.) >>> - JNI support for jimage into hotspot. >>> - White box tests written to exercise native jimage support. From mark.reinhold at oracle.com Tue Jun 23 16:12:19 2015 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 23 Jun 2015 09:12:19 -0700 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: References: , <55893E8C.5050600@oracle.com>, , <55896B05.2080209@oracle.com>, , <20150623083249.604405@eggemoggin.niobe.net>, Message-ID: <20150623091219.950515@eggemoggin.niobe.net> 2015/6/23 8:45 -0700, vitalyd at gmail.com: > Yes, but until all the "safe" replacements are in place and vetted (e.g. > performance is on par with Unsafe, same functionality is available, etc), I > don't see the point of making it even more annoying to grab hold of. The > people who are using it will continue using it until the replacements are > available, and this is just going to annoy them. That's precisely the point. sun.misc.Unsafe and its ilk will go away one day. In preparation for that, making it a bit harder to use will motivate its current users to consider whether they really do need to use it -- some do, but some don't. If you absolutely do need it then now is the time to start looking at the alternatives in development, and contribute to those efforts in order to make sure that your needs are met. Paul's work on variable handles (JEP 193 [1]), e.g., is far enough along that feedback would be useful. Making sun.misc.Unsafe harder to use will also help the many users who unknowingly depend upon this unsupported API, via libraries which do depend upon it, to become aware of that dependence. They can then either seek alternatives or ask the maintainers of those libraries to do so. - Mark [1] http://openjdk.java.net/jeps/193 From vitalyd at gmail.com Tue Jun 23 16:27:16 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 23 Jun 2015 12:27:16 -0400 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: <20150623091219.950515@eggemoggin.niobe.net> References: <55893E8C.5050600@oracle.com> <55896B05.2080209@oracle.com> <20150623083249.604405@eggemoggin.niobe.net> <20150623091219.950515@eggemoggin.niobe.net> Message-ID: > > sun.misc.Unsafe and its ilk will go away one day. In preparation for > that, making it a bit harder to use will motivate its current users to > consider whether they really do need to use it -- some do, but some > don't. I think there's already sufficient noise about use of Unsafe that there's no need for yet another inconvenience -- I don't think this class is flying under the radar in this regard? If you absolutely do need it then now is the time to start looking at > the alternatives in development, and contribute to those efforts in > order to make sure that your needs are met. Paul's work on variable > handles (JEP 193 [1]), e.g., is far enough along that feedback would > be useful. Sure, but that's orthogonal to putting up more red tape in java 9 when replacements aren't in place yet. If you guys want more involvement in the development/validation of replacements, that needs to happen now anyway and not wait until java 9. Making sun.misc.Unsafe harder to use will also help the many users who > unknowingly depend upon this unsupported API, via libraries which do > depend upon it, to become aware of that dependence. They can then > either seek alternatives or ask the maintainers of those libraries to > do so. I get what you're after here, but I'm not sure it's going to have the desired effect; if a user of a lib that uses Unsafe has to now change their launch commands to add a flag, they're not going to be irritated by the lib author. Also, the library author may rightly point out that s/he can't move away from Unsafe yet because some of the replacement feature(s) aren't in place yet. This type of thing would carry more weight if Unsafe is fully replaceable in terms of perf and functionality when these roadblocks are put up. On Tue, Jun 23, 2015 at 12:12 PM, wrote: > 2015/6/23 8:45 -0700, vitalyd at gmail.com: > > Yes, but until all the "safe" replacements are in place and vetted (e.g. > > performance is on par with Unsafe, same functionality is available, > etc), I > > don't see the point of making it even more annoying to grab hold of. The > > people who are using it will continue using it until the replacements are > > available, and this is just going to annoy them. > > That's precisely the point. > > sun.misc.Unsafe and its ilk will go away one day. In preparation for > that, making it a bit harder to use will motivate its current users to > consider whether they really do need to use it -- some do, but some > don't. > > If you absolutely do need it then now is the time to start looking at > the alternatives in development, and contribute to those efforts in > order to make sure that your needs are met. Paul's work on variable > handles (JEP 193 [1]), e.g., is far enough along that feedback would > be useful. > > Making sun.misc.Unsafe harder to use will also help the many users who > unknowingly depend upon this unsupported API, via libraries which do > depend upon it, to become aware of that dependence. They can then > either seek alternatives or ask the maintainers of those libraries to > do so. > > - Mark > > > [1] http://openjdk.java.net/jeps/193 > From forax at univ-mlv.fr Tue Jun 23 16:34:01 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 23 Jun 2015 18:34:01 +0200 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: <20150623091219.950515@eggemoggin.niobe.net> References: , <55893E8C.5050600@oracle.com>, , <55896B05.2080209@oracle.com>, , <20150623083249.604405@eggemoggin.niobe.net>, <20150623091219.950515@eggemoggin.niobe.net> Message-ID: <55898A79.9070204@univ-mlv.fr> Hi Mark, On 06/23/2015 06:12 PM, mark.reinhold at oracle.com wrote: > 2015/6/23 8:45 -0700, vitalyd at gmail.com: >> Yes, but until all the "safe" replacements are in place and vetted (e.g. >> performance is on par with Unsafe, same functionality is available, etc), I >> don't see the point of making it even more annoying to grab hold of. The >> people who are using it will continue using it until the replacements are >> available, and this is just going to annoy them. I've never seen a project that use Unsafe without a good reason (off-heap, more fine grained concurrency primitives, faster serialization, etc). People will move from unsafe iff there is a good public replacement API. 'Educating' smart people, the ones that build businesses on top of this API, by trying to use their users against them is in my opinion a stupid move. There will be no replacement for Unsafe in 9, it's a shame but we are not ready, we should wait 10 have a good migation plan and push the red button. But you already know that. regards, R?mi > That's precisely the point. > > sun.misc.Unsafe and its ilk will go away one day. In preparation for > that, making it a bit harder to use will motivate its current users to > consider whether they really do need to use it -- some do, but some > don't. > > If you absolutely do need it then now is the time to start looking at > the alternatives in development, and contribute to those efforts in > order to make sure that your needs are met. Paul's work on variable > handles (JEP 193 [1]), e.g., is far enough along that feedback would > be useful. > > Making sun.misc.Unsafe harder to use will also help the many users who > unknowingly depend upon this unsupported API, via libraries which do > depend upon it, to become aware of that dependence. They can then > either seek alternatives or ask the maintainers of those libraries to > do so. > > - Mark > > > [1] http://openjdk.java.net/jeps/193 From jeroen at sumatra.nl Wed Jun 24 08:04:49 2015 From: jeroen at sumatra.nl (Jeroen Frijters) Date: Wed, 24 Jun 2015 08:04:49 +0000 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: <20150623091219.950515@eggemoggin.niobe.net> References: , <55893E8C.5050600@oracle.com>, , <55896B05.2080209@oracle.com>, , <20150623083249.604405@eggemoggin.niobe.net>, <20150623091219.950515@eggemoggin.niobe.net> Message-ID: Hi Mark, I have a passionate dislike for Unsafe, but I have to agree with Vitaly and R?mi. Until there are alternatives, it makes no sense to add more annoyances. Regards, Jeroen > -----Original Message----- > From: jigsaw-dev [mailto:jigsaw-dev-bounces at openjdk.java.net] On Behalf > Of mark.reinhold at oracle.com > Sent: Tuesday, June 23, 2015 18:12 > To: Vitaly Davidovich > Cc: jigsaw-dev at openjdk.java.net; adoption-discuss at openjdk.java.net > Subject: Re: Jigsaw Hackday in London - Anything in particular you want > us to look at? > > 2015/6/23 8:45 -0700, vitalyd at gmail.com: > > Yes, but until all the "safe" replacements are in place and vetted > (e.g. > > performance is on par with Unsafe, same functionality is available, > > etc), I don't see the point of making it even more annoying to grab > > hold of. The people who are using it will continue using it until the > > replacements are available, and this is just going to annoy them. > > That's precisely the point. > > sun.misc.Unsafe and its ilk will go away one day. In preparation for > that, making it a bit harder to use will motivate its current users to > consider whether they really do need to use it -- some do, but some > don't. > > If you absolutely do need it then now is the time to start looking at > the alternatives in development, and contribute to those efforts in > order to make sure that your needs are met. Paul's work on variable > handles (JEP 193 [1]), e.g., is far enough along that feedback would be > useful. > > Making sun.misc.Unsafe harder to use will also help the many users who > unknowingly depend upon this unsupported API, via libraries which do > depend upon it, to become aware of that dependence. They can then > either seek alternatives or ask the maintainers of those libraries to do > so. > > - Mark > > > [1] http://openjdk.java.net/jeps/193 From richard.warburton at gmail.com Mon Jun 29 12:43:25 2015 From: richard.warburton at gmail.com (Richard Warburton) Date: Mon, 29 Jun 2015 13:43:25 +0100 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: <55912886.9070101@tglman.com> References: <55893E8C.5050600@oracle.com> <55896B05.2080209@oracle.com> <20150623083249.604405@eggemoggin.niobe.net> <20150623091219.950515@eggemoggin.niobe.net> <55912886.9070101@tglman.com> Message-ID: Hi, This is a good point actually. I have also used that class before as it seems to be the only way to hook those signals in a commandline app. I appreciate its a difficult case to deal with since you cant safely run Java code inside a signal handler callback because signal handler callback code must be reentrant safe but it feels like having some kind of Java level hook for this kind of signal would be useful. Even if it is asynchronous. regards, Richard On 29 Jun 2015 12:15, "Tglman" wrote: > Hi All, > > I am a library developer, and in some cases in the project i work on is > used Unsafe, and we already planning to replace it with other solutions. > anyway while i was trying to test my project on jdk9 i found that also > other api we use are removed. > In my spefic case we use also 'sun.misc.SignalHandler', is this api > going to be available in future following the same approach used for > sun.misc.Unsafe ? > > Is it there any replacement for handling not shutdown/kill/interrupt > signals ? > (In my specific case we catch also SIGTRAP) > > Thank You > > Emanuele > > > > > On 23/06/15 17:12, mark.reinhold at oracle.com wrote: > > 2015/6/23 8:45 -0700, vitalyd at gmail.com: > >> Yes, but until all the "safe" replacements are in place and vetted (e.g. > >> performance is on par with Unsafe, same functionality is available, > etc), I > >> don't see the point of making it even more annoying to grab hold of. > The > >> people who are using it will continue using it until the replacements > are > >> available, and this is just going to annoy them. > > That's precisely the point. > > > > sun.misc.Unsafe and its ilk will go away one day. In preparation for > > that, making it a bit harder to use will motivate its current users to > > consider whether they really do need to use it -- some do, but some > > don't. > > > > If you absolutely do need it then now is the time to start looking at > > the alternatives in development, and contribute to those efforts in > > order to make sure that your needs are met. Paul's work on variable > > handles (JEP 193 [1]), e.g., is far enough along that feedback would > > be useful. > > > > Making sun.misc.Unsafe harder to use will also help the many users who > > unknowingly depend upon this unsupported API, via libraries which do > > depend upon it, to become aware of that dependence. They can then > > either seek alternatives or ask the maintainers of those libraries to > > do so. > > > > - Mark > > > > > > [1] http://openjdk.java.net/jeps/193 > > From mandy.chung at oracle.com Mon Jun 29 16:58:43 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 29 Jun 2015 09:58:43 -0700 Subject: Jigsaw Hackday in London - Anything in particular you want us to look at? In-Reply-To: References: <55893E8C.5050600@oracle.com> <55896B05.2080209@oracle.com> <20150623083249.604405@eggemoggin.niobe.net> <20150623091219.950515@eggemoggin.niobe.net> <55912886.9070101@tglman.com> Message-ID: FYI. The RFE concerning sun.misc.SignalHandler is: https://bugs.openjdk.java.net/browse/JDK-8087286 Mandy > On Jun 29, 2015, at 5:43 AM, Richard Warburton wrote: > > Hi, > > This is a good point actually. I have also used that class before as it > seems to be the only way to hook those signals in a commandline app. > > I appreciate its a difficult case to deal with since you cant safely run > Java code inside a signal handler callback because signal handler callback > code must be reentrant safe but it feels like having some kind of Java > level hook for this kind of signal would be useful. Even if it is > asynchronous. > > regards, > > Richard > On 29 Jun 2015 12:15, "Tglman" wrote: > >> Hi All, >> >> I am a library developer, and in some cases in the project i work on is >> used Unsafe, and we already planning to replace it with other solutions. >> anyway while i was trying to test my project on jdk9 i found that also >> other api we use are removed. >> In my spefic case we use also 'sun.misc.SignalHandler', is this api >> going to be available in future following the same approach used for >> sun.misc.Unsafe ? >> >> Is it there any replacement for handling not shutdown/kill/interrupt >> signals ? >> (In my specific case we catch also SIGTRAP) >> >> Thank You >> >> Emanuele >> >> >> >> >> On 23/06/15 17:12, mark.reinhold at oracle.com wrote: >>> 2015/6/23 8:45 -0700, vitalyd at gmail.com: >>>> Yes, but until all the "safe" replacements are in place and vetted (e.g. >>>> performance is on par with Unsafe, same functionality is available, >> etc), I >>>> don't see the point of making it even more annoying to grab hold of. >> The >>>> people who are using it will continue using it until the replacements >> are >>>> available, and this is just going to annoy them. >>> That's precisely the point. >>> >>> sun.misc.Unsafe and its ilk will go away one day. In preparation for >>> that, making it a bit harder to use will motivate its current users to >>> consider whether they really do need to use it -- some do, but some >>> don't. >>> >>> If you absolutely do need it then now is the time to start looking at >>> the alternatives in development, and contribute to those efforts in >>> order to make sure that your needs are met. Paul's work on variable >>> handles (JEP 193 [1]), e.g., is far enough along that feedback would >>> be useful. >>> >>> Making sun.misc.Unsafe harder to use will also help the many users who >>> unknowingly depend upon this unsupported API, via libraries which do >>> depend upon it, to become aware of that dependence. They can then >>> either seek alternatives or ask the maintainers of those libraries to >>> do so. >>> >>> - Mark >>> >>> >>> [1] http://openjdk.java.net/jeps/193 >> >>