From serguei.spitsyn at oracle.com Sat Oct 1 00:02:33 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 30 Sep 2016 17:02:33 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <101e4a9e-2e91-6415-9dd9-0d57522ce7f6@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <101e4a9e-2e91-6415-9dd9-0d57522ce7f6@oracle.com> Message-ID: <7e1ab158-b01e-1e08-d276-0b3c4acd14e0@oracle.com> On 9/30/16 16:57, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > > Just wanted to share a couple of comments I have so far. > > > http://cr.openjdk.java.net/~coleenp/8164921.01/webrev/src/share/vm/memory/metaspace.cpp.frames.html > > > 302 debug_only(GrowableArray* _all_blocks); > 855 DEBUG_ONLY(_all_blocks = new (ResourceObj::C_HEAP, mtClass) > GrowableArray(100, true)); 863 DEBUG_ONLY(delete > _all_blocks;) 891 > DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); 908 > DEBUG_ONLY(_all_blocks->remove((Metablock*)free_block)); 922 > DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); > > Not clear, why different macros debug_only() is used at L302. Could > it be DEBUG_ONLY as well? > > 866 void BlockFreelist::return_block(MetaWord* p, size_t word_size) { > 867 Metablock* free_chunk = ::new (p) Metablock(word_size); > 868 if (word_size < SmallBlocks::small_block_min_size()) { > 869 // Dark matter > 870 return; 871 } else if (word_size < > SmallBlocks::small_block_max_size()) { > > In case of dark matter the free_chunk is leaked. It is better to > rearrange the fragment to something like this: > > void BlockFreelist::return_block(MetaWord* p, size_t word_size) { > if (word_size < SmallBlocks::small_block_min_size()) { > return; // Dark matter > } > Metablock* free_chunk = ::new (p) Metablock(word_size); if (word_size > < SmallBlocks::small_block_max_size()) { > > 886 MetaWord* BlockFreelist::get_block(size_t word_size) { > 887 if (word_size < SmallBlocks::small_block_max_size() && > small_blocks()->list_at(word_size).count() > 0) { > . . . > 892 return new_block; > 893 } > 894 > 895 if (word_size < BlockFreelist::min_size()) { > 896 // Dark matter. Too small for dictionary. > 897 return NULL; > 898 } > > It'd be better to reorder the fragments 887-893 and 895-898. Thanks, > Serguei The formatting got broken at the end. :( I guess, it is better to return to plain text mode in my emails. Thanks, Serguei > On 9/30/16 12:02, Coleen Phillimore wrote: >> Summary: Return Metablocks smaller than dictionary's dark matter. >> This change contributed by Jon Masamitsu and myself. To reclaim >> "dark matter" this change adds an array of small blocks by size, >> created lazily, to return Metablocks smaller than the >> BinaryTreeDictionary entry's minimum size. This change also fixed a >> bug in small object double free and adds debugging code to check for >> this case. With this change, the submitted test case runs >> indefinitely. Also passed rbt tier 1-5 testing. open webrev at >> http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >> https://bugs.openjdk.java.net/browse/JDK-8164921 Thanks, Coleen and Jon From dmitry.samersoff at oracle.com Sat Oct 1 07:26:26 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Sat, 1 Oct 2016 10:26:26 +0300 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <4efb2443-9ade-fb56-2016-f46bb5b2eb57@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <62fcf1e8-de12-5f25-5de2-9b9db39a0ca1@oracle.com> <4efb2443-9ade-fb56-2016-f46bb5b2eb57@oracle.com> Message-ID: <5dd2e6f0-55ee-aeb2-e534-e116316ef6ca@oracle.com> Coleen, > Or something but the loops seem simple enough to me. OK. Thank you for explanation. -Dmitry On 2016-10-01 01:44, Coleen Phillimore wrote: > > > On 9/30/16 3:46 PM, Dmitry Samersoff wrote: >> Coleen, >> >> My $.02 >> >> 1. >> metaspace.cpp:279 we have assert(word_size >= _small_block_min_size, >> ...) so at 868 and 895 >> if (word_size < SmallBlocks::small_block_min_size()) { >> probably should be asserts as well >> >> >> 2. >> It might be better to rewrite loops at metaspace.cpp 261,268 >> for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { >> as >> for (uint i = 0; >> i < (_small_block_max_size - _small_block_min_size); ++i) { >> ... > > for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { > _small_lists[i - _small_block_min_size].set_size(i); > } > > > In this loop I want 'i' to set the size. The index of small_lists > starts at 0 but the size is the size of the blocks. This might make > more sense like: > > for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { > int k = i - _small_block_min_size; // zero based addressing > _small_lists[k].set_size(i); > } > > Or something but the loops seem simple enough to me. > > Coleen >> >> -Dmitry >> >> On 2016-09-30 22:02, Coleen Phillimore wrote: >>> Summary: Return Metablocks smaller than dictionary's dark matter. >>> >>> This change contributed by Jon Masamitsu and myself. To reclaim "dark >>> matter" this change adds an array of small blocks by size, created >>> lazily, to return Metablocks smaller than the BinaryTreeDictionary >>> entry's minimum size. This change also fixed a bug in small object >>> double free and adds debugging code to check for this case. >>> >>> With this change, the submitted test case runs indefinitely. Also >>> passed rbt tier 1-5 testing. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >>> >>> Thanks, >>> Coleen and Jon >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From kim.barrett at oracle.com Sat Oct 1 21:29:21 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 1 Oct 2016 17:29:21 -0400 Subject: RFR: 8166862: CMS needs klass_or_null_acquire Message-ID: Please review this change to the CMS collector to replace uses of klass_or_null with klass_or_null_acquire where needed. [Reviewing on hotspot-dev; only GC code is involved, but some non-GC folks have expressed interest.] All non-assert uses of klass_or_null are being replaced with the acquire variant. In most cases, the size is being accessed soon after, and the acquire is needed to ensure order of klass and possible inline size access. In a few cases, the need for acquire is less immediately obvious: - CompatibleFreeListSpace::block_is_obj - Size access is not lexically apparent, but callers access the size or other data ordered with the klass. - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: These perform other operations that need to be properly ordered wrto the klass access. Removed a couple of explict acquire fences that are made redundant by the use of klass_or_null_acquire. The latter is preferable, as it permits the use of platform-specific load_acquire implementations that might be better than separate load + acquire fence. Removed a stale comment about permgen and is_parsable. CR: https://bugs.openjdk.java.net/browse/JDK-8166862 Webrev: http://cr.openjdk.java.net/~kbarrett/8166862/webrev.00/ Testing: JPRT From kirill.zhaldybin at oracle.com Mon Oct 3 09:14:09 2016 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Mon, 3 Oct 2016 12:14:09 +0300 Subject: RFR(S) : 8166925 : several native TESTs should be changed to TEST_VM In-Reply-To: References: Message-ID: <57F22161.9060908@oracle.com> Igor, looks good to me. Thank you for fixing it. Regards, Kirill On 30.09.2016 18:34, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8166925/webrev.00 >> 109 lines changed: 0 ins; 0 del; 109 mod; > > Hi all, > > could you please review this patch which fixes type of tests? the fix is basically 's/TEST/TEST_VM/?. > > there are several tests which depend on initialized JVM, but do not declare it (by using TEST_VM macros). here is the list of those tests along w/ observations where/when they need JVM: > > 1. all arrayOopDesc tests use arrayOopDesc::header_size_in_bytes which depends on a jvm state, it reads HeapWordSize and UseCompressedClassPointers and furthermore has an assert which checks that their values haven't changed. > 2. all utilities/json tests use tty variable (as JSON::_st field), tty initialization is a port of jvm initialization. > 3. SymbolTable.temp_new_symbol calls JavaThread::current() > 4. LogFileOutput tests use ResourceMark which calls JavaThread::current() > 5. LogDecorations.level, uptime, tags and timestamps tests directly or indirectly call os::javaTimeNanos() which assumes that some structures, e.g. Bsd::_timebase_info, are inited > 6. LogConfigurationTest tests require ResourceMark or os::javaTimeNanos > > JBS: https://bugs.openjdk.java.net/browse/JDK-8166925 > webrev: http://cr.openjdk.java.net/~iignatyev/8166925/webrev.00 > > Thanks, > ? Igor > From igor.ignatyev at oracle.com Mon Oct 3 09:22:28 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 3 Oct 2016 12:22:28 +0300 Subject: RFR(S) : 8166925 : several native TESTs should be changed to TEST_VM In-Reply-To: <57F22161.9060908@oracle.com> References: <57F22161.9060908@oracle.com> Message-ID: <9DD40934-F87A-4FF7-A2DB-8375B724B3D1@oracle.com> Kirill, thank you for review. Cheers, ? Igor > On Oct 3, 2016, at 12:14 PM, Kirill Zhaldybin wrote: > > Igor, > > looks good to me. > Thank you for fixing it. > > Regards, Kirill > > On 30.09.2016 18:34, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8166925/webrev.00 >>> 109 lines changed: 0 ins; 0 del; 109 mod; >> >> Hi all, >> >> could you please review this patch which fixes type of tests? the fix is basically 's/TEST/TEST_VM/?. >> >> there are several tests which depend on initialized JVM, but do not declare it (by using TEST_VM macros). here is the list of those tests along w/ observations where/when they need JVM: >> >> 1. all arrayOopDesc tests use arrayOopDesc::header_size_in_bytes which depends on a jvm state, it reads HeapWordSize and UseCompressedClassPointers and furthermore has an assert which checks that their values haven't changed. >> 2. all utilities/json tests use tty variable (as JSON::_st field), tty initialization is a port of jvm initialization. >> 3. SymbolTable.temp_new_symbol calls JavaThread::current() >> 4. LogFileOutput tests use ResourceMark which calls JavaThread::current() >> 5. LogDecorations.level, uptime, tags and timestamps tests directly or indirectly call os::javaTimeNanos() which assumes that some structures, e.g. Bsd::_timebase_info, are inited >> 6. LogConfigurationTest tests require ResourceMark or os::javaTimeNanos >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8166925 >> webrev: http://cr.openjdk.java.net/~iignatyev/8166925/webrev.00 >> >> Thanks, >> ? Igor >> > From thomas.schatzl at oracle.com Mon Oct 3 09:25:39 2016 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 03 Oct 2016 11:25:39 +0200 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: References: Message-ID: <1475486739.2702.10.camel@oracle.com> Hi, On Sat, 2016-10-01 at 17:29 -0400, Kim Barrett wrote: > Please review this change to the CMS collector to replace uses of > klass_or_null with klass_or_null_acquire where needed. > > [Reviewing on hotspot-dev; only GC code is involved, but some non-GC > folks have expressed interest.] > > All non-assert uses of klass_or_null are being replaced with the > acquire variant.??In most cases, the size is being accessed soon > after, and the acquire is needed to ensure order of klass and > possible inline size access.??In a few cases, the need for acquire is > less immediately obvious: > > - CompatibleFreeListSpace::block_is_obj - Size access is not > lexically apparent, but callers access the size or other data ordered > with the klass. > > - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: > ? These perform other operations that need to be properly ordered > wrto the klass access. > > Removed a couple of explict acquire fences that are made redundant by > the use of klass_or_null_acquire.??The latter is preferable, as it > permits the use of platform-specific load_acquire implementations > that might be better than separate load + acquire fence. > > Removed a stale comment about permgen and is_parsable. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166862 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166862/webrev.00/ ? looks good. Thomas From kirill.zhaldybin at oracle.com Mon Oct 3 09:30:40 2016 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Mon, 3 Oct 2016 12:30:40 +0300 Subject: RFR(S) : 8166129 : hitting vmassert during gtest execution doesn't generate core and hs_err files In-Reply-To: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> References: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> Message-ID: <57F22540.8050508@oracle.com> Igor, Looks good to me! Regards, Kirill On 30.09.2016 19:06, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ >> 54 lines changed: 31 ins; 14 del; 9 mod; > > Hi all, > > could you please review the patch which enables core dumping and hs_err file generation in case of vmassert, guarantee and the like are hit during gtest tests execution? > > before this fix, we always initialize a jvm w/ -XX:+SuppressFatalErrorMessage and -XX:-CreateCoredumpOnCrash, now we do so only if we run assert tests, tests which we expect to hit an vmassert. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8166129 > webrev: http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ > > Thanks, > ? Igor > From igor.ignatyev at oracle.com Mon Oct 3 09:51:47 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 3 Oct 2016 12:51:47 +0300 Subject: RFR(S) : 8166129 : hitting vmassert during gtest execution doesn't generate core and hs_err files In-Reply-To: <57F22540.8050508@oracle.com> References: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> <57F22540.8050508@oracle.com> Message-ID: Hi Kirill, thank you for review. Cheers, ? Igor > On Oct 3, 2016, at 12:30 PM, Kirill Zhaldybin wrote: > > Igor, > > Looks good to me! > > Regards, Kirill > > On 30.09.2016 19:06, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ >>> 54 lines changed: 31 ins; 14 del; 9 mod; >> >> Hi all, >> >> could you please review the patch which enables core dumping and hs_err file generation in case of vmassert, guarantee and the like are hit during gtest tests execution? >> >> before this fix, we always initialize a jvm w/ -XX:+SuppressFatalErrorMessage and -XX:-CreateCoredumpOnCrash, now we do so only if we run assert tests, tests which we expect to hit an vmassert. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8166129 >> webrev: http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ >> >> Thanks, >> ? Igor >> > From edward.nevill at gmail.com Mon Oct 3 11:30:30 2016 From: edward.nevill at gmail.com (Edward Nevill) Date: Mon, 03 Oct 2016 12:30:30 +0100 Subject: Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <25E205BE-9EE9-488A-9D08-DED365288D7F@oracle.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <1475231521.14313.29.camel@gmail.com> <25E205BE-9EE9-488A-9D08-DED365288D7F@oracle.com> Message-ID: <1475494230.2320.5.camel@gmail.com> Hi Bob, The 32-bit ARM early access binaries are client only. I only see the problems with the server JIT. All the best, Ed. On Fri, 2016-09-30 at 10:03 -0400, Bob Vandette wrote: > Ed, > > Could you try the 32-bit ARM binaries from the early access page to > see if the > G1GC problem reproduces?? > > https://jdk9.java.net/download/ > > Bob. From erik.joelsson at oracle.com Mon Oct 3 13:53:44 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 3 Oct 2016 15:53:44 +0200 Subject: RFR: JDK-8164120: The minimal VM should be stripped using --strip-unneeded In-Reply-To: <95711c11-662c-20be-8cd6-5b2cbe601c24@oracle.com> References: <95711c11-662c-20be-8cd6-5b2cbe601c24@oracle.com> Message-ID: (adding hotspot-dev) On 2016-10-03 15:52, Erik Joelsson wrote: > Hello, > > Please review this small change, which adds --strip-unneeded when > stripping the minimal VM on Linux, which reduces size on x86 builds > with around 1.2MB. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8164120 > > Webrev: http://cr.openjdk.java.net/~erikj/8164120/webrev.hotspot.01/ > > /Erik > From mikael.gerdin at oracle.com Mon Oct 3 13:58:36 2016 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 3 Oct 2016 15:58:36 +0200 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: <1475486739.2702.10.camel@oracle.com> References: <1475486739.2702.10.camel@oracle.com> Message-ID: Hi, On 2016-10-03 11:25, Thomas Schatzl wrote: > Hi, > > On Sat, 2016-10-01 at 17:29 -0400, Kim Barrett wrote: >> Please review this change to the CMS collector to replace uses of >> klass_or_null with klass_or_null_acquire where needed. >> >> [Reviewing on hotspot-dev; only GC code is involved, but some non-GC >> folks have expressed interest.] >> >> All non-assert uses of klass_or_null are being replaced with the >> acquire variant. In most cases, the size is being accessed soon >> after, and the acquire is needed to ensure order of klass and >> possible inline size access. In a few cases, the need for acquire is >> less immediately obvious: >> >> - CompatibleFreeListSpace::block_is_obj - Size access is not >> lexically apparent, but callers access the size or other data ordered >> with the klass. >> >> - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: >> These perform other operations that need to be properly ordered >> wrto the klass access. >> >> Removed a couple of explict acquire fences that are made redundant by >> the use of klass_or_null_acquire. The latter is preferable, as it >> permits the use of platform-specific load_acquire implementations >> that might be better than separate load + acquire fence. >> >> Removed a stale comment about permgen and is_parsable. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8166862 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8166862/webrev.00/ > > looks good. Looks good to me as well. /Mikael > > Thomas > From jesper.wilhelmsson at oracle.com Mon Oct 3 14:04:05 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 3 Oct 2016 16:04:05 +0200 Subject: RFR(xs): JDK-8167026 - Quarantine TestDaemonThread.java Message-ID: <74375cbd-b5f7-dc9d-2813-20686cea2bfe@oracle.com> Hi, Please review this quarantine of a test that fails when run by JPRT. The change will be pushed directly to jdk9/dev. Bug: https://bugs.openjdk.java.net/browse/JDK-8167001 Webrev: http://cr.openjdk.java.net/~jwilhelm/8167026/webrev.00/ Thanks, /Jesper From dmitry.samersoff at oracle.com Mon Oct 3 14:11:26 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Mon, 3 Oct 2016 17:11:26 +0300 Subject: RFR(xs): JDK-8167026 - Quarantine TestDaemonThread.java In-Reply-To: <74375cbd-b5f7-dc9d-2813-20686cea2bfe@oracle.com> References: <74375cbd-b5f7-dc9d-2813-20686cea2bfe@oracle.com> Message-ID: <8b199d39-0990-3147-5847-b6db93ec1cd9@oracle.com> Jesper, Looks good for me! -Dmitry On 2016-10-03 17:04, Jesper Wilhelmsson wrote: > Hi, > > Please review this quarantine of a test that fails when run by JPRT. > The change will be pushed directly to jdk9/dev. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8167001 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8167026/webrev.00/ > > Thanks, > /Jesper -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From jesper.wilhelmsson at oracle.com Mon Oct 3 14:22:25 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 3 Oct 2016 16:22:25 +0200 Subject: RFR(xs): JDK-8167026 - Quarantine TestDaemonThread.java In-Reply-To: <8b199d39-0990-3147-5847-b6db93ec1cd9@oracle.com> References: <74375cbd-b5f7-dc9d-2813-20686cea2bfe@oracle.com> <8b199d39-0990-3147-5847-b6db93ec1cd9@oracle.com> Message-ID: <184a6771-7225-f1c2-fc85-3d26ffe3062b@oracle.com> Thanks Dmitry! /Jesper Den 3/10/16 kl. 16:11, skrev Dmitry Samersoff: > Jesper, > > Looks good for me! > > -Dmitry > > On 2016-10-03 17:04, Jesper Wilhelmsson wrote: >> Hi, >> >> Please review this quarantine of a test that fails when run by JPRT. >> The change will be pushed directly to jdk9/dev. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8167001 >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8167026/webrev.00/ >> >> Thanks, >> /Jesper > > From coleen.phillimore at oracle.com Mon Oct 3 15:46:01 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 3 Oct 2016 11:46:01 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <101e4a9e-2e91-6415-9dd9-0d57522ce7f6@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <101e4a9e-2e91-6415-9dd9-0d57522ce7f6@oracle.com> Message-ID: <2dbb46a7-7608-97ec-cf8a-2abbf6e755dd@oracle.com> Serguei, thank you for looking at this. On 9/30/16 7:57 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > > Just wanted to share a couple of comments I have so far. > > > http://cr.openjdk.java.net/~coleenp/8164921.01/webrev/src/share/vm/memory/metaspace.cpp.frames.html > > 302 debug_only(GrowableArray* _all_blocks); > 855 DEBUG_ONLY(_all_blocks = new (ResourceObj::C_HEAP, mtClass) > GrowableArray(100, true)); 863 DEBUG_ONLY(delete > _all_blocks;) 891 > DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); 908 > DEBUG_ONLY(_all_blocks->remove((Metablock*)free_block)); 922 > DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); > Not clear, why different macros debug_only() is used at L302. > Could it be DEBUG_ONLY as well? Yes, good catch. I thought I had recompiled after adding the DEBUG_ONLY, but I must not have. It should be DEBUG_ONLY. > 866 void BlockFreelist::return_block(MetaWord* p, size_t word_size) { > 867 Metablock* free_chunk = ::new (p) Metablock(word_size); > 868 if (word_size < SmallBlocks::small_block_min_size()) { > 869 // Dark matter > 870 return; 871 } else if (word_size < > SmallBlocks::small_block_max_size()) { > In case of dark matter the free_chunk is leaked. It is better to > rearrange the fragment to something like this: > void BlockFreelist::return_block(MetaWord* p, size_t word_size) { > if (word_size < SmallBlocks::small_block_min_size()) { > return; // Dark matter > } > Metablock* free_chunk = ::new (p) Metablock(word_size); if (word_size > < SmallBlocks::small_block_max_size()) { Okay, yes, this seems like it would be a bug since SmallBlocks::small_block_min_size() should equal sizeof Metablock in words, but since we align up to sizeof Metablock during allocation, we won't ever hit this. But I'll change as suggested. > 886 MetaWord* BlockFreelist::get_block(size_t word_size) { > 887 if (word_size < SmallBlocks::small_block_max_size() && > small_blocks()->list_at(word_size).count() > 0) { > . . . > 892 return new_block; > 893 } > 894 > 895 if (word_size < BlockFreelist::min_size()) { > 896 // Dark matter. Too small for dictionary. > 897 return NULL; > 898 } > It'd be better to reorder the fragments 887-893 and 895-898. Thanks, > Serguei The order is important because we first try for something in the small_blocks array, then try for something in the dictionary. BlockFreelist::min_size() is the size of a minimal dictionary entry, not the same as SmallBlocks::small_block_min_size(). So the code is, look for something in the small blocks (3-11 words) but if the size is too small for the dictionary (12 words), return. Then the block is allocated from the Metachunk and not the small_blocks/dictionary of returned blocks. Thank you for starting your review. Please let me know when you have further comments. Coleen > On 9/30/16 12:02, Coleen Phillimore wrote: >> Summary: Return Metablocks smaller than dictionary's dark matter. >> This change contributed by Jon Masamitsu and myself. To reclaim >> "dark matter" this change adds an array of small blocks by size, >> created lazily, to return Metablocks smaller than the >> BinaryTreeDictionary entry's minimum size. This change also fixed a >> bug in small object double free and adds debugging code to check for >> this case. With this change, the submitted test case runs >> indefinitely. Also passed rbt tier 1-5 testing. open webrev at >> http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >> https://bugs.openjdk.java.net/browse/JDK-8164921 Thanks, Coleen and Jon From coleen.phillimore at oracle.com Mon Oct 3 15:47:56 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 3 Oct 2016 11:47:56 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <5dd2e6f0-55ee-aeb2-e534-e116316ef6ca@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <62fcf1e8-de12-5f25-5de2-9b9db39a0ca1@oracle.com> <4efb2443-9ade-fb56-2016-f46bb5b2eb57@oracle.com> <5dd2e6f0-55ee-aeb2-e534-e116316ef6ca@oracle.com> Message-ID: <8a9c6e63-589c-023f-85b2-67b8d39ddace@oracle.com> On 10/1/16 3:26 AM, Dmitry Samersoff wrote: > Coleen, > >> Or something but the loops seem simple enough to me. > OK. Thank you for explanation. Thank you for you comments and for looking at the code. Coleen > > -Dmitry > > On 2016-10-01 01:44, Coleen Phillimore wrote: >> >> On 9/30/16 3:46 PM, Dmitry Samersoff wrote: >>> Coleen, >>> >>> My $.02 >>> >>> 1. >>> metaspace.cpp:279 we have assert(word_size >= _small_block_min_size, >>> ...) so at 868 and 895 >>> if (word_size < SmallBlocks::small_block_min_size()) { >>> probably should be asserts as well >>> >>> >>> 2. >>> It might be better to rewrite loops at metaspace.cpp 261,268 >>> for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { >>> as >>> for (uint i = 0; >>> i < (_small_block_max_size - _small_block_min_size); ++i) { >>> ... >> for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { >> _small_lists[i - _small_block_min_size].set_size(i); >> } >> >> >> In this loop I want 'i' to set the size. The index of small_lists >> starts at 0 but the size is the size of the blocks. This might make >> more sense like: >> >> for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { >> int k = i - _small_block_min_size; // zero based addressing >> _small_lists[k].set_size(i); >> } >> >> Or something but the loops seem simple enough to me. >> >> Coleen >>> -Dmitry >>> >>> On 2016-09-30 22:02, Coleen Phillimore wrote: >>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>> >>>> This change contributed by Jon Masamitsu and myself. To reclaim "dark >>>> matter" this change adds an array of small blocks by size, created >>>> lazily, to return Metablocks smaller than the BinaryTreeDictionary >>>> entry's minimum size. This change also fixed a bug in small object >>>> double free and adds debugging code to check for this case. >>>> >>>> With this change, the submitted test case runs indefinitely. Also >>>> passed rbt tier 1-5 testing. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >>>> >>>> Thanks, >>>> Coleen and Jon > From vladimir.kozlov at oracle.com Mon Oct 3 19:17:04 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Oct 2016 12:17:04 -0700 Subject: hs-comp is closed for pushes Message-ID: According to proposed dates for "Merging jdk9/hs-comp with jdk9/hs" we are closing hs-comp/ for pushes except new nightly bugs fixes. We want to get clean Nightly testing results in preparation for merge jdk9/hs-comp with jdk9/hs on Friday, October 7th. Thanks, Vladimir From kim.barrett at oracle.com Mon Oct 3 19:56:04 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Oct 2016 15:56:04 -0400 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: References: <1475486739.2702.10.camel@oracle.com> Message-ID: > On Oct 3, 2016, at 9:58 AM, Mikael Gerdin wrote: > On 2016-10-03 11:25, Thomas Schatzl wrote: >> Hi, >> >> On Sat, 2016-10-01 at 17:29 -0400, Kim Barrett wrote: >>> Please review this change to the CMS collector to replace uses of >>> klass_or_null with klass_or_null_acquire where needed. >>> >>> [?] >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8166862 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8166862/webrev.00/ >> >> looks good. > > Looks good to me as well. > /Mikael > >> >> Thomas Thanks, Thomas and Mikael. From kim.barrett at oracle.com Mon Oct 3 20:11:06 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Oct 2016 16:11:06 -0400 Subject: RFR(S) : 8166129 : hitting vmassert during gtest execution doesn't generate core and hs_err files In-Reply-To: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> References: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> Message-ID: > On Sep 30, 2016, at 12:06 PM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ >> 54 lines changed: 31 ins; 14 del; 9 mod; > > Hi all, > > could you please review the patch which enables core dumping and hs_err file generation in case of vmassert, guarantee and the like are hit during gtest tests execution? > > before this fix, we always initialize a jvm w/ -XX:+SuppressFatalErrorMessage and -XX:-CreateCoredumpOnCrash, now we do so only if we run assert tests, tests which we expect to hit an vmassert. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8166129 > webrev: http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ > > Thanks, > ? Igor Does this change eliminate the need for this (IMO misplaced) bit of code in the vmassert macro? if (is_executing_unit_tests()) { \ report_assert_msg(__VA_ARGS__); \ } \ From vladimir.kozlov at oracle.com Mon Oct 3 20:31:54 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Oct 2016 13:31:54 -0700 Subject: ATTENTION: hs-comp is CLOSED for pushes!!! Message-ID: <24a59760-c99d-4c58-21a2-4d2c5c2b6cff@oracle.com> I have to repeat. Please, wait until October 7th with pushes. Or push into jdk9/hs. According to proposed dates for "Merging jdk9/hs-comp with jdk9/hs" we are closing hs-comp/ for pushes except new nightly bugs fixes. We want to get clean Nightly testing results in preparation for merge jdk9/hs-comp with jdk9/hs on Friday, October 7th. Thanks, Vladimir From igor.ignatyev at oracle.com Mon Oct 3 20:33:34 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 3 Oct 2016 23:33:34 +0300 Subject: RFR(S) : 8166129 : hitting vmassert during gtest execution doesn't generate core and hs_err files In-Reply-To: References: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> Message-ID: <4E0B1A2D-CBF9-4FCF-8F91-A81E632703FA@oracle.com> Kim, > Does this change eliminate the need for this (IMO misplaced) bit of code in the vmassert macro? unfortunately not, since we still disable fatal error reporting for assert tests, we need to have that code to support TEST_VM_ASSERT_MSG tests ? assert tests which check that expected assert message is printed out. I agree that bit of code does not really look nice, but I?d prefer to work on that issue separately. Thanks, ? Igor > On Oct 3, 2016, at 11:11 PM, Kim Barrett wrote: > >> On Sep 30, 2016, at 12:06 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ >>> 54 lines changed: 31 ins; 14 del; 9 mod; >> >> Hi all, >> >> could you please review the patch which enables core dumping and hs_err file generation in case of vmassert, guarantee and the like are hit during gtest tests execution? >> >> before this fix, we always initialize a jvm w/ -XX:+SuppressFatalErrorMessage and -XX:-CreateCoredumpOnCrash, now we do so only if we run assert tests, tests which we expect to hit an vmassert. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8166129 >> webrev: http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ >> >> Thanks, >> ? Igor > > Does this change eliminate the need for this (IMO misplaced) bit of code in the vmassert macro? > > if (is_executing_unit_tests()) { \ > report_assert_msg(__VA_ARGS__); \ > } \ > From vladimir.kozlov at oracle.com Mon Oct 3 21:49:40 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Oct 2016 14:49:40 -0700 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> Message-ID: <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> Hi Volker, Can you prepare combined patch (or set of patches) based on latest reviews together with s390 code as it will be in final push? We want to run it through our pre-integration testing to verify that it does not have problems. Thanks, Vladimir On 9/29/16 11:25 AM, Vladimir Kozlov wrote: > You need to wait when Mark (OpenJDK Lead) move it to Candidate (or > other) state: > > http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html > > Vladimir > > On 9/29/16 9:55 AM, Volker Simonis wrote: >> Hi Vladimir, >> >> thanks a lot for reviewing and endorsing the JEP. >> >> I've linked all the relevant issues to the JEP (they all have a link >> to a webrev) and change the state to "Submitted". >> >> There's just one more small shared change we need for the port for >> which we haven't opened a bug now because we are still working on >> simplifying it. The current version looks as follows: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-constant_table_offset.patch >> >> >> What are the next steps? Should I add a "jdk9-fc-request" label to t >> he JEP and add a corresponding "FC Extension Request" comment to it? >> Or will this be done automatically once I move it to "Candidate"? >> >> Is there anything left to do before I can move it to "Candidate" state? >> >> Thanks a lot and best regards, >> Volker >> >> >> >> >> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov >> wrote: >>> On 9/27/16 10:49 AM, Volker Simonis wrote: >>>> >>>> Hi, >>>> >>>> can you please review and endorse the following draft JEP for the >>>> integration of the Linux/s390x port into the jkd9 master repository: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8166730 >>> >>> >>> Good. >>> Add links to webrevs in a comment. It will help to get umbrella FC >>> extension >>> approval. >>> >>>> >>>> As detailed in the JEP, the Linux/s390x requires very few shared >>>> changes and we therefore don't foresee any impact on the existing >>>> platforms at all. Following you can find a short description of the >>>> planned changes: >>>> >>>> hotspot: >>>> ======= >>>> >>>> Out for review: >>>> 8166560: [s390] Basic enablement of s390 port. >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ >>>> >>>> Reviewed: >>>> 8166562: C2: Suppress relocations in scratch emit. >>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/ >>>> >>>> Will send RFR soon (depends on 8166560): >>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. >>>> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01 >>> >>> >>> Wrong link. >>> >>> Thanks, >>> Vladimir >>> >>> >>>> >>>> We are still investigating the need of these shared changes: >>>> >>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000011-pass_PC_to_retAddrOffset.patch >>>> >>>> >>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000016-constant_table_offset.patch >>>> >>>> >>>> And finally the patch with the s390x-only platform files. We are still >>>> editing these to get them into OpenJdk style and shape. >>>> Hotspot passes most jck, jtreg and spec tests with these. >>>> >>>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000101-zFiles.patch >>>> >>>> >>>> top-level repository: >>>> =============== >>>> >>>> The following is just adding some s390x specific compiler flags to >>>> flags.m4 >>>> 8166800: [s390] Top-level build changes required for Linux/s390x >>>> https://bugs.openjdk.java.net/browse/JDK-8166800 >>>> >>>> jdk repository: >>>> ============ >>>> >>>> This one just adds a new jvm.cfg file for s390x >>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x >>>> https://bugs.openjdk.java.net/browse/JDK-8166801 >>>> >>>> >>>> And finally we plan to do one more change which fixes the jtreg test >>>> on Linux/s390x. But this is mainly for the correct detection of the >>>> platform and for excluding the tests which are not appropriate for >>>> s390x. >>>> >>>> Thank you and best regards, >>>> Volker >>>> >>> From serguei.spitsyn at oracle.com Mon Oct 3 23:11:07 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 3 Oct 2016 16:11:07 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <2dbb46a7-7608-97ec-cf8a-2abbf6e755dd@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <101e4a9e-2e91-6415-9dd9-0d57522ce7f6@oracle.com> <2dbb46a7-7608-97ec-cf8a-2abbf6e755dd@oracle.com> Message-ID: <53a8ecf1-59fd-02ff-f329-8cd49de29286@oracle.com> Coleen, On 10/3/16 08:46, Coleen Phillimore wrote: > > Serguei, thank you for looking at this. > > On 9/30/16 7:57 PM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> >> Just wanted to share a couple of comments I have so far. >> >> >> http://cr.openjdk.java.net/~coleenp/8164921.01/webrev/src/share/vm/memory/metaspace.cpp.frames.html >> >> 302 debug_only(GrowableArray* _all_blocks); >> 855 DEBUG_ONLY(_all_blocks = new (ResourceObj::C_HEAP, mtClass) >> GrowableArray(100, true)); 863 DEBUG_ONLY(delete >> _all_blocks;) 891 >> DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); 908 >> DEBUG_ONLY(_all_blocks->remove((Metablock*)free_block)); 922 >> DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); >> Not clear, why different macros debug_only() is used at L302. >> Could it be DEBUG_ONLY as well? > Yes, good catch. I thought I had recompiled after adding the > DEBUG_ONLY, but I must not have. It should be DEBUG_ONLY. >> 866 void BlockFreelist::return_block(MetaWord* p, size_t word_size) { >> 867 Metablock* free_chunk = ::new (p) Metablock(word_size); >> 868 if (word_size < SmallBlocks::small_block_min_size()) { >> 869 // Dark matter >> 870 return; 871 } else if (word_size < >> SmallBlocks::small_block_max_size()) { >> In case of dark matter the free_chunk is leaked. It is better to >> rearrange the fragment to something like this: >> void BlockFreelist::return_block(MetaWord* p, size_t word_size) { >> if (word_size < SmallBlocks::small_block_min_size()) { >> return; // Dark matter >> } >> Metablock* free_chunk = ::new (p) Metablock(word_size); if (word_size >> < SmallBlocks::small_block_max_size()) { > Okay, yes, this seems like it would be a bug since > SmallBlocks::small_block_min_size() should equal sizeof Metablock in > words, but since we align up to sizeof Metablock during allocation, we > won't ever hit this. But I'll change as suggested. I agree, we won't hit it. But it is still better to make it right. Thank you for the update! >> 886 MetaWord* BlockFreelist::get_block(size_t word_size) { >> 887 if (word_size < SmallBlocks::small_block_max_size() && >> small_blocks()->list_at(word_size).count() > 0) { >> . . . >> 892 return new_block; >> 893 } >> 894 >> 895 if (word_size < BlockFreelist::min_size()) { >> 896 // Dark matter. Too small for dictionary. >> 897 return NULL; >> 898 } >> It'd be better to reorder the fragments 887-893 and 895-898. >> Thanks, Serguei > The order is important because we first try for something in the > small_blocks array, then try for something in the dictionary. > BlockFreelist::min_size() is the size of a minimal dictionary entry, > not the same as SmallBlocks::small_block_min_size(). So the code is, > look for something in the small blocks (3-11 words) but if the size is > too small for the dictionary (12 words), return. Then the block is > allocated from the Metachunk and not the small_blocks/dictionary of > returned blocks. Thank you for starting your review. Please let me > know when you have further comments. Coleen Ok, thank you for explanation. But then the additional check is needed before the L886: if (word_size < SmallBlocks::small_block_min_size()) { return; } Otherwise, we can hit the assert in the list_at(): 278 FreeList& list_at(size_t word_size) { 279 assert(word_size >= _small_block_min_size, "There are no metaspace objects less than %u words", _small_block_min_size); 280 return _small_lists[word_size - _small_block_min_size]; 281 } I do not have any other comments. Reviewed. Thanks, Serguei >> On 9/30/16 12:02, Coleen Phillimore wrote: >>> Summary: Return Metablocks smaller than dictionary's dark matter. >>> This change contributed by Jon Masamitsu and myself. To reclaim >>> "dark matter" this change adds an array of small blocks by size, >>> created lazily, to return Metablocks smaller than the >>> BinaryTreeDictionary entry's minimum size. This change also fixed >>> a bug in small object double free and adds debugging code to check >>> for this case. With this change, the submitted test case runs >>> indefinitely. Also passed rbt tier 1-5 testing. open webrev at >>> http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>> https://bugs.openjdk.java.net/browse/JDK-8164921 Thanks, Coleen and Jon From kim.barrett at oracle.com Tue Oct 4 02:22:49 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Oct 2016 22:22:49 -0400 Subject: RFR(S) : 8166129 : hitting vmassert during gtest execution doesn't generate core and hs_err files In-Reply-To: <4E0B1A2D-CBF9-4FCF-8F91-A81E632703FA@oracle.com> References: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> <4E0B1A2D-CBF9-4FCF-8F91-A81E632703FA@oracle.com> Message-ID: > On Oct 3, 2016, at 4:33 PM, Igor Ignatyev wrote: > > Kim, > >> Does this change eliminate the need for this (IMO misplaced) bit of code in the vmassert macro? > unfortunately not, since we still disable fatal error reporting for assert tests, we need to have that code to support TEST_VM_ASSERT_MSG tests ? assert tests which check that expected assert message is printed out. I agree that bit of code does not really look nice, but I?d prefer to work on that issue separately. Thanks for the explanation. Is there a CR for that? As for this change for 8166129, looks good. From david.holmes at oracle.com Tue Oct 4 07:16:25 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Oct 2016 17:16:25 +1000 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> Message-ID: <34f41756-7ac7-684e-c896-9b8babf8f419@oracle.com> Hi Martin, On 30/09/2016 8:03 PM, Martin Walsh wrote: > On 23/09/2016 14:34, Martin Walsh wrote: >> On 20/09/2016 08:42, Erik Joelsson wrote: >>> Hello, >>> >>> >>> On 2016-09-20 03:28, David Holmes wrote: >>>> Hi Martin, >>>> >>>> Build changes must be reviewed by the build team - now cc'd >>>> >>> Thanks for forwarding David, I can't keep up with all the lists to find >>> these unless posted to build-dev. >>>> On 20/09/2016 12:16 AM, Martin Walsh wrote: >>>>> Could I get a code review for the following bug: >>>>> >>>>> JDK-8165482 java in ldoms, with cpu-arch=generic has problems >>>>> >>>>> Webrev of the changes is available here: >>>>> >>>>> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ >>>> >>>> What is the devinfo library? Is it part of the normal Solaris >>>> installation, or does it need to be installed specifically? Is it >>>> available in our official build toolkits? >>> I did a bit of digging. It's part "system/libraries" so should be pretty >>> standard. That package is in the devkit and I verified that >>> libdevinfo.so is there too. >>> >>> Configure change looks fine. Just remember to also push the closed >>> generated-configure.sh as David said. >>> >>> /Erik >>>> >>>> Will checking the prom prior to using kstat change any of the values >>>> we currently see? (other than the generic case being fixed of course). >> >> It shouldn't, as SPARC64 seems to be reserved for Fujitsu SPARC >> machines. However, to air on the side of caution I will investigate >> further and follow-up shortly. > > So, after further deliberation I have decided that although utilising > the PROM may work on some occasions, it is not the correct solution. > > All Oracle SPARC LDOMS use the "sun4-cpu" string as the CPU > implementation, therefore I think the best short term fix is to add an > additional match that checks for this string. Long term, this could do > with a re-write, but that is a JDK10 project. > > This updated fix also means there are no build changes. > > Updated the webrev accordingly. > > http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ These new changes seem okay to me. Only one nit in src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp 467 } 468 } 469 } 470 } the indent on line 469 is wrong. Thanks, David ----- > Thanks, > > Martin > From david.holmes at oracle.com Tue Oct 4 07:56:15 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Oct 2016 17:56:15 +1000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: <4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap> References: <4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap> Message-ID: <41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com> Hi Goetz, On 28/09/2016 8:26 PM, Lindenmaier, Goetz wrote: > Hi, > > here a new webrev for this change: > http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr02/ > > I reverted the major reorderings in macros.hpp and os_linux.cpp. > David asked me to do so, and I guess it makes reviewing more simple. Thanks. That said ... > Also this fixes the issue spotted by David which actually was wrong. > The other renaming of ARM to ARM32 was correct, though, as > AARCH64 is defined in both ARM 64-bit ports, and is checked before. > So the existing case checking ARM is only reached if !LP64. > I documented this ... ... We actually have a bug with the Elf32_* defines in os_linux.cpp 1769 #elif (defined ARM) // ARM must come before AARCH64 because the closed 64-bit port requires so. 1770 static Elf32_Half running_arch_code=EM_ARM; Aarch64 should come first otherwise we will set the wrong value. I've been told it would only affect an error message but still ... can I ask you to fix this please. Thanks. --- src/share/vm/code/codeCache.cpp I'd prefer to just see something like: S390_ONLY(if (_heaps == NULL) return;) // May be true when NMT detail is used --- Otherwise seems okay. Thanks, David > Also I removed the change in mutex.hpp. Maybe we contribute > the full change this was part of, but independent of the s390 port. > > I withdraw the part of the change to jdk introducing jvm.cfg. Volker wants to > do a separate change for this. > > Best regards, > Goetz. > > > >> -----Original Message----- >> From: Volker Simonis [mailto:volker.simonis at gmail.com] >> Sent: Dienstag, 27. September 2016 19:58 >> To: David Holmes >> Cc: Lindenmaier, Goetz ; hotspot- >> dev at openjdk.java.net; core-libs-dev >> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. >> >> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes >> wrote: >>> Hi Goetz, >>> >>> I see a change not related directly to S390 ie change from ARM to ARM32 in >>> src/os/linux/vm/os_linux.cpp >>> >> >> The change looks a little confusing because Goetz reordered the ifdef >> cascades alphabetically (which I think is good). >> >> Besides that, the only real change not related to s390 is indeed the >> change from ARM to ARM32 which happend two times in the file. >> >> @Goetz: have you done this intentionally? >> >>> It will be a while before I can go through this in any detail. >>> >>> David >>> >>> >>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: >>>> >>>> Hi, >>>> >>>> This change is part of the s390 port. It contains some basic adaptions >>>> needed for a full hotspot port for linux s390x. >>>> >>>> It defines the required macros, platform names and includes. >>>> >>>> The s390 port calles CodeCache::contains() in current_frame(), which is >>>> used in NMT. As NMT already collects stack traces before the CodeCache >> is >>>> initialized, contains() needs a check for this. >>>> >>>> Wherever a row of platforms are listed, I sorted them alphabetically. >>>> >>>> The jdk requires the file jvm.cfg. >>>> >>>> Please review. I please need a sponsor. >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/hotspot.wr01/ >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ >>>> >>>> Best regards, >>>> Goetz. >>>> >>> From david.holmes at oracle.com Tue Oct 4 08:20:12 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Oct 2016 18:20:12 +1000 Subject: RFR: JDK-8164120: The minimal VM should be stripped using --strip-unneeded In-Reply-To: References: <95711c11-662c-20be-8cd6-5b2cbe601c24@oracle.com> Message-ID: Hi Erik, On 3/10/2016 11:53 PM, Erik Joelsson wrote: > (adding hotspot-dev) > > > On 2016-10-03 15:52, Erik Joelsson wrote: >> Hello, >> >> Please review this small change, which adds --strip-unneeded when >> stripping the minimal VM on Linux, which reduces size on x86 builds >> with around 1.2MB. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8164120 >> >> Webrev: http://cr.openjdk.java.net/~erikj/8164120/webrev.hotspot.01/ I'm confused as to which is the chicken and which is the egg: STRIPFLAGS := $(JVM_STRIPFLAGS) JVM_STRIPFLAGS := $(STRIPFLAGS) --strip-unneeded ?? Thanks, David >> >> /Erik >> > From martin.walsh at oracle.com Tue Oct 4 09:03:03 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Tue, 4 Oct 2016 10:03:03 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <34f41756-7ac7-684e-c896-9b8babf8f419@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> <34f41756-7ac7-684e-c896-9b8babf8f419@oracle.com> Message-ID: (dropped build-dev alias as there are no longer any build changes in the revised fix) On 04/10/2016 08:16, David Holmes wrote: > Hi Martin, >> All Oracle SPARC LDOMS use the "sun4-cpu" string as the CPU >> implementation, therefore I think the best short term fix is to add an >> additional match that checks for this string. Long term, this could do >> with a re-write, but that is a JDK10 project. >> >> This updated fix also means there are no build changes. >> >> Updated the webrev accordingly. >> >> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ > > These new changes seem okay to me. > > Only one nit in src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp > > 467 } > 468 } > 469 } > 470 } > > the indent on line 469 is wrong. Fixed. Thanks David. I have updated the webrev and the patch attached to the bug. Would anybody be willing to sponsor this changeset? Thanks, Martin From david.holmes at oracle.com Tue Oct 4 09:04:46 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Oct 2016 19:04:46 +1000 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: References: Message-ID: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> Hi Kim, On 2/10/2016 7:29 AM, Kim Barrett wrote: > Please review this change to the CMS collector to replace uses of > klass_or_null with klass_or_null_acquire where needed. > > [Reviewing on hotspot-dev; only GC code is involved, but some non-GC > folks have expressed interest.] > > All non-assert uses of klass_or_null are being replaced with the > acquire variant. In most cases, the size is being accessed soon > after, and the acquire is needed to ensure order of klass and possible > inline size access. In a few cases, the need for acquire is less > immediately obvious: > > - CompatibleFreeListSpace::block_is_obj - Size access is not lexically > apparent, but callers access the size or other data ordered with the > klass. > > - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: > These perform other operations that need to be properly ordered wrto > the klass access. Can you point me to the release-store that is being paired with these load-acquires please? Superficially this all seems fine and can't have any correctness issues, it is just a matter of where acquire semantics are actually needed. And it is hard to tell that without understanding exactly how the different read-paths may execute relative to the write-path. Thanks, David > Removed a couple of explict acquire fences that are made redundant by > the use of klass_or_null_acquire. The latter is preferable, as it > permits the use of platform-specific load_acquire implementations that > might be better than separate load + acquire fence. > > Removed a stale comment about permgen and is_parsable. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166862 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166862/webrev.00/ > > Testing: > JPRT > From david.holmes at oracle.com Tue Oct 4 09:06:18 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Oct 2016 19:06:18 +1000 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> <34f41756-7ac7-684e-c896-9b8babf8f419@oracle.com> Message-ID: On 4/10/2016 7:03 PM, Martin Walsh wrote: > (dropped build-dev alias as there are no longer any build changes in the > revised fix) > > On 04/10/2016 08:16, David Holmes wrote: >> Hi Martin, > >>> All Oracle SPARC LDOMS use the "sun4-cpu" string as the CPU >>> implementation, therefore I think the best short term fix is to add an >>> additional match that checks for this string. Long term, this could do >>> with a re-write, but that is a JDK10 project. >>> >>> This updated fix also means there are no build changes. >>> >>> Updated the webrev accordingly. >>> >>> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ >> >> These new changes seem okay to me. >> >> Only one nit in src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp >> >> 467 } >> 468 } >> 469 } >> 470 } >> >> the indent on line 469 is wrong. > > > Fixed. Thanks David. I have updated the webrev and the patch attached > to the bug. > > Would anybody be willing to sponsor this changeset? I can sponsor, but we still need a second review of this version. Thanks, David > Thanks, > > Martin > From erik.helin at oracle.com Tue Oct 4 09:42:52 2016 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 4 Oct 2016 11:42:52 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> Message-ID: <20161004094252.GL20056@ehelin.jrpg.bea.com> On 2016-09-30, Leonid Mesnik wrote: > Hi > > I think that it would be better to split this test into 4 tests. Currently > this test is going to be skipped if any specific GCare set. Or it could even > fail if any G1 specific options is set without setting G1 explicitly. > > So there is no way to run this with additional G1/CMS/ParGC specific options > now. Ok, I've split the test into four "tests" (they just run GCBasher with different GCs). Please see new patches: - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ Thanks, Erik > Leonid > > > On 28.09.2016 18:01, Erik Helin wrote: > >Hi all, > > > >this patch adds a new GC stress test called GCBasher. GCBasher builds up > >large (well, for some definiton of large) object graphs by figuring out > >the relations between classes in the JDK. The test usually stresses the > >GC quite a lot, especially when run with a smaller heap. > > > >The changes in the top-level repository are for JPRT. JPRT will now run > >the jtreg test GCBasher instead of the old version. > > > >Enhancement: > >https://bugs.openjdk.java.net/browse/JDK-8166790 > > > >Webrev: > >- hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > >- top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > > > >Testing: > >- Running the new jtreg test locally on Linux x86-64: > > $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > > > > (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > >- JPRT > > > >Thanks, > >Erik > From leonid.mesnik at oracle.com Tue Oct 4 09:47:20 2016 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 4 Oct 2016 12:47:20 +0300 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20161004094252.GL20056@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> Message-ID: <8135132f-ba05-117e-a5a4-9efe594fd281@oracle.com> Very nice. Thank you for implementing this. Leonid On 04.10.2016 12:42, Erik Helin wrote: > On 2016-09-30, Leonid Mesnik wrote: >> Hi >> >> I think that it would be better to split this test into 4 tests. Currently >> this test is going to be skipped if any specific GCare set. Or it could even >> fail if any G1 specific options is set without setting G1 explicitly. >> >> So there is no way to run this with additional G1/CMS/ParGC specific options >> now. > Ok, I've split the test into four "tests" (they just run GCBasher with > different GCs). Please see new patches: > > - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ > > Thanks, > Erik > >> Leonid >> >> >> On 28.09.2016 18:01, Erik Helin wrote: >>> Hi all, >>> >>> this patch adds a new GC stress test called GCBasher. GCBasher builds up >>> large (well, for some definiton of large) object graphs by figuring out >>> the relations between classes in the JDK. The test usually stresses the >>> GC quite a lot, especially when run with a smaller heap. >>> >>> The changes in the top-level repository are for JPRT. JPRT will now run >>> the jtreg test GCBasher instead of the old version. >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8166790 >>> >>> Webrev: >>> - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ >>> - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ >>> >>> Testing: >>> - Running the new jtreg test locally on Linux x86-64: >>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java >>> >>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) >>> - JPRT >>> >>> Thanks, >>> Erik From mikael.gerdin at oracle.com Tue Oct 4 12:32:09 2016 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 4 Oct 2016 14:32:09 +0200 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> Message-ID: <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: > Summary: Return Metablocks smaller than dictionary's dark matter. > > This change contributed by Jon Masamitsu and myself. To reclaim "dark > matter" this change adds an array of small blocks by size, created > lazily, to return Metablocks smaller than the BinaryTreeDictionary > entry's minimum size. This change also fixed a bug in small object > double free and adds debugging code to check for this case. > > With this change, the submitted test case runs indefinitely. Also > passed rbt tier 1-5 testing. > > open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8164921 I'd prefer it if SmallBlocks didn't expose its implementation by returning its FreeLists by reference, could you change it to have * return_chunk() * get_chunk() * num_chunks(word_size) and get rid of list_at? -- For how long do you plan to keep BlockFreelist::_all_blocks? I see that it's debug only but I fear that it could case problems with running out of native memory in our internal testing of debug builds. BlockFreelist::min_size() is a bit of a misnomer since it returns the minimum size of blocks to be put on the BlockTreeDictionary, not the minimum size of blocks which are reusable. Is there any particular reason behind _small_blocks being lazy allocated and _dictionary not? I would prefer if BlockFreelist::return_block would perform the checks in reverse order instead of having a return inside the first if block, something like if (word_size > small_block_max_size) { dict()->return_chunk(c) } else if (word_size > small_block_min_size) { small_blocks()->return_chunk(c) } else { // dark matter } For BlockFreelist::get_block I realize that the code is a bit more complex but this also raises a few questions. * When we allocate from the dictionary we search for a block which is at least as large as we ask for and split it, returning whatever was left back to the free list. We don't search for "large enough" blocks from the small blocks manager, is that intentional or just to keep the code simple (I like simple)? In the last part of get_block where we return the unused part of a block retrieved from the dictionary uses compares with BlockFreelist::min_size() which, as I mentioned above, is the min size for blocks in the dictionary, not the min size for blocks to be reusable. I think this code can just call return_block unconditionally for any nonzero value of "unused" and let return_block deal with dark matter. Are the changes to Method and ConstMethod the "bug fix to small object double free"? Is the problem that when a method was freed its annotations were deallocated as well? Could the annotations fields in the "old" ConstMethod be cleared instead so that the old annotation arrays could be kept or is that just needless complexity? Thanks /Mikael > > Thanks, > Coleen and Jon From vladimir.x.ivanov at oracle.com Tue Oct 4 13:13:51 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 4 Oct 2016 16:13:51 +0300 Subject: RFR(S) : 8166925 : several native TESTs should be changed to TEST_VM In-Reply-To: References: Message-ID: > http://cr.openjdk.java.net/~iignatyev/8166925/webrev.00 Reviewed. > there are several tests which depend on initialized JVM, but do not declare it (by using TEST_VM macros). here is the list of those tests along w/ observations where/when they need JVM: > > 1. all arrayOopDesc tests use arrayOopDesc::header_size_in_bytes which depends on a jvm state, it reads HeapWordSize and UseCompressedClassPointers and furthermore has an assert which checks that their values haven't changed. > 2. all utilities/json tests use tty variable (as JSON::_st field), tty initialization is a port of jvm initialization. > 3. SymbolTable.temp_new_symbol calls JavaThread::current() > 4. LogFileOutput tests use ResourceMark which calls JavaThread::current() > 5. LogDecorations.level, uptime, tags and timestamps tests directly or indirectly call os::javaTimeNanos() which assumes that some structures, e.g. Bsd::_timebase_info, are inited > 6. LogConfigurationTest tests require ResourceMark or os::javaTimeNanos Maybe add these comments to the tests? Best regards, Vladimir Ivanov From coleen.phillimore at oracle.com Tue Oct 4 14:31:48 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 4 Oct 2016 10:31:48 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <53a8ecf1-59fd-02ff-f329-8cd49de29286@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <101e4a9e-2e91-6415-9dd9-0d57522ce7f6@oracle.com> <2dbb46a7-7608-97ec-cf8a-2abbf6e755dd@oracle.com> <53a8ecf1-59fd-02ff-f329-8cd49de29286@oracle.com> Message-ID: <680d9007-421d-a855-bf2b-7844ebd2480c@oracle.com> On 10/3/16 7:11 PM, serguei.spitsyn at oracle.com wrote: > Coleen, > > > On 10/3/16 08:46, Coleen Phillimore wrote: >> >> Serguei, thank you for looking at this. >> >> On 9/30/16 7:57 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> >>> Just wanted to share a couple of comments I have so far. >>> >>> >>> http://cr.openjdk.java.net/~coleenp/8164921.01/webrev/src/share/vm/memory/metaspace.cpp.frames.html >>> >>> 302 debug_only(GrowableArray* _all_blocks); >>> 855 DEBUG_ONLY(_all_blocks = new (ResourceObj::C_HEAP, mtClass) >>> GrowableArray(100, true)); 863 DEBUG_ONLY(delete >>> _all_blocks;) 891 >>> DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); 908 >>> DEBUG_ONLY(_all_blocks->remove((Metablock*)free_block)); 922 >>> DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); >>> Not clear, why different macros debug_only() is used at L302. >>> Could it be DEBUG_ONLY as well? >> Yes, good catch. I thought I had recompiled after adding the >> DEBUG_ONLY, but I must not have. It should be DEBUG_ONLY. >>> 866 void BlockFreelist::return_block(MetaWord* p, size_t word_size) { >>> 867 Metablock* free_chunk = ::new (p) Metablock(word_size); >>> 868 if (word_size < SmallBlocks::small_block_min_size()) { >>> 869 // Dark matter >>> 870 return; 871 } else if (word_size < >>> SmallBlocks::small_block_max_size()) { >>> In case of dark matter the free_chunk is leaked. It is better to >>> rearrange the fragment to something like this: >>> void BlockFreelist::return_block(MetaWord* p, size_t word_size) { >>> if (word_size < SmallBlocks::small_block_min_size()) { >>> return; // Dark matter >>> } >>> Metablock* free_chunk = ::new (p) Metablock(word_size); if >>> (word_size < SmallBlocks::small_block_max_size()) { >> Okay, yes, this seems like it would be a bug since >> SmallBlocks::small_block_min_size() should equal sizeof Metablock in >> words, but since we align up to sizeof Metablock during allocation, >> we won't ever hit this. But I'll change as suggested. > > > I agree, we won't hit it. > But it is still better to make it right. > Thank you for the update! Okay, I fixed this as suggested. > > > > >>> 886 MetaWord* BlockFreelist::get_block(size_t word_size) { >>> 887 if (word_size < SmallBlocks::small_block_max_size() && >>> small_blocks()->list_at(word_size).count() > 0) { >>> . . . >>> 892 return new_block; >>> 893 } >>> 894 >>> 895 if (word_size < BlockFreelist::min_size()) { >>> 896 // Dark matter. Too small for dictionary. >>> 897 return NULL; >>> 898 } >>> It'd be better to reorder the fragments 887-893 and 895-898. >>> Thanks, Serguei >> The order is important because we first try for something in the >> small_blocks array, then try for something in the dictionary. >> BlockFreelist::min_size() is the size of a minimal dictionary entry, >> not the same as SmallBlocks::small_block_min_size(). So the code is, >> look for something in the small blocks (3-11 words) but if the size >> is too small for the dictionary (12 words), return. Then the block >> is allocated from the Metachunk and not the small_blocks/dictionary >> of returned blocks. Thank you for starting your review. Please let >> me know when you have further comments. Coleen > > > Ok, thank you for explanation. > But then the additional check is needed before the L886: > > if (word_size < SmallBlocks::small_block_min_size()) { > return; > } > > Otherwise, we can hit the assert in the list_at(): > > 278 FreeList& list_at(size_t word_size) { > 279 assert(word_size >= _small_block_min_size, "There are no metaspace > objects less than %u words", _small_block_min_size); > 280 return _small_lists[word_size - _small_block_min_size]; > 281 } I also added this as suggested. Thanks, Serguei! Coleen > I do not have any other comments. > Reviewed. > > > Thanks, > Serguei > > >>> On 9/30/16 12:02, Coleen Phillimore wrote: >>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>> "dark matter" this change adds an array of small blocks by size, >>>> created lazily, to return Metablocks smaller than the >>>> BinaryTreeDictionary entry's minimum size. This change also fixed >>>> a bug in small object double free and adds debugging code to check >>>> for this case. With this change, the submitted test case runs >>>> indefinitely. Also passed rbt tier 1-5 testing. open webrev at >>>> http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>>> https://bugs.openjdk.java.net/browse/JDK-8164921 Thanks, Coleen and >>>> Jon > From serguei.spitsyn at oracle.com Tue Oct 4 14:54:15 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 4 Oct 2016 07:54:15 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <680d9007-421d-a855-bf2b-7844ebd2480c@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <101e4a9e-2e91-6415-9dd9-0d57522ce7f6@oracle.com> <2dbb46a7-7608-97ec-cf8a-2abbf6e755dd@oracle.com> <53a8ecf1-59fd-02ff-f329-8cd49de29286@oracle.com> <680d9007-421d-a855-bf2b-7844ebd2480c@oracle.com> Message-ID: <9da39bb3-a69e-d40a-d68b-0a339c33ef0f@oracle.com> On 10/4/16 07:31, Coleen Phillimore wrote: > > > On 10/3/16 7:11 PM, serguei.spitsyn at oracle.com wrote: >> Coleen, >> >> >> On 10/3/16 08:46, Coleen Phillimore wrote: >>> >>> Serguei, thank you for looking at this. >>> >>> On 9/30/16 7:57 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Coleen, >>>> >>>> >>>> Just wanted to share a couple of comments I have so far. >>>> >>>> >>>> http://cr.openjdk.java.net/~coleenp/8164921.01/webrev/src/share/vm/memory/metaspace.cpp.frames.html >>>> >>>> 302 debug_only(GrowableArray* _all_blocks); >>>> 855 DEBUG_ONLY(_all_blocks = new (ResourceObj::C_HEAP, mtClass) >>>> GrowableArray(100, true)); 863 DEBUG_ONLY(delete >>>> _all_blocks;) 891 >>>> DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); 908 >>>> DEBUG_ONLY(_all_blocks->remove((Metablock*)free_block)); 922 >>>> DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); >>>> Not clear, why different macros debug_only() is used at L302. >>>> Could it be DEBUG_ONLY as well? >>> Yes, good catch. I thought I had recompiled after adding the >>> DEBUG_ONLY, but I must not have. It should be DEBUG_ONLY. >>>> 866 void BlockFreelist::return_block(MetaWord* p, size_t word_size) { >>>> 867 Metablock* free_chunk = ::new (p) Metablock(word_size); >>>> 868 if (word_size < SmallBlocks::small_block_min_size()) { >>>> 869 // Dark matter >>>> 870 return; 871 } else if (word_size < >>>> SmallBlocks::small_block_max_size()) { >>>> In case of dark matter the free_chunk is leaked. It is better to >>>> rearrange the fragment to something like this: >>>> void BlockFreelist::return_block(MetaWord* p, size_t word_size) { >>>> if (word_size < SmallBlocks::small_block_min_size()) { >>>> return; // Dark matter >>>> } >>>> Metablock* free_chunk = ::new (p) Metablock(word_size); if >>>> (word_size < SmallBlocks::small_block_max_size()) { >>> Okay, yes, this seems like it would be a bug since >>> SmallBlocks::small_block_min_size() should equal sizeof Metablock in >>> words, but since we align up to sizeof Metablock during allocation, >>> we won't ever hit this. But I'll change as suggested. >> >> >> I agree, we won't hit it. >> But it is still better to make it right. >> Thank you for the update! > > Okay, I fixed this as suggested. >> >> >> >> >>>> 886 MetaWord* BlockFreelist::get_block(size_t word_size) { >>>> 887 if (word_size < SmallBlocks::small_block_max_size() && >>>> small_blocks()->list_at(word_size).count() > 0) { >>>> . . . >>>> 892 return new_block; >>>> 893 } >>>> 894 >>>> 895 if (word_size < BlockFreelist::min_size()) { >>>> 896 // Dark matter. Too small for dictionary. >>>> 897 return NULL; >>>> 898 } >>>> It'd be better to reorder the fragments 887-893 and 895-898. >>>> Thanks, Serguei >>> The order is important because we first try for something in the >>> small_blocks array, then try for something in the dictionary. >>> BlockFreelist::min_size() is the size of a minimal dictionary entry, >>> not the same as SmallBlocks::small_block_min_size(). So the code is, >>> look for something in the small blocks (3-11 words) but if the size >>> is too small for the dictionary (12 words), return. Then the block >>> is allocated from the Metachunk and not the small_blocks/dictionary >>> of returned blocks. Thank you for starting your review. Please let >>> me know when you have further comments. Coleen >> >> >> Ok, thank you for explanation. >> But then the additional check is needed before the L886: >> >> if (word_size < SmallBlocks::small_block_min_size()) { >> return; >> } >> >> Otherwise, we can hit the assert in the list_at(): >> >> 278 FreeList& list_at(size_t word_size) { >> 279 assert(word_size >= _small_block_min_size, "There are no >> metaspace objects less than %u words", _small_block_min_size); >> 280 return _small_lists[word_size - _small_block_min_size]; >> 281 } > > I also added this as suggested. Thank you for updating the fix, Coleen! Serguei > > Thanks, Serguei! > Coleen >> I do not have any other comments. >> Reviewed. >> >> >> Thanks, >> Serguei >> >> >>>> On 9/30/16 12:02, Coleen Phillimore wrote: >>>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>>> "dark matter" this change adds an array of small blocks by size, >>>>> created lazily, to return Metablocks smaller than the >>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>> fixed a bug in small object double free and adds debugging code to >>>>> check for this case. With this change, the submitted test case >>>>> runs indefinitely. Also passed rbt tier 1-5 testing. open webrev >>>>> at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>>>> https://bugs.openjdk.java.net/browse/JDK-8164921 Thanks, Coleen >>>>> and Jon >> > From coleen.phillimore at oracle.com Tue Oct 4 16:15:55 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 4 Oct 2016 12:15:55 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> Message-ID: <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, Mikael Gerdin wrote: > Hi Coleen, > > On 2016-09-30 21:02, Coleen Phillimore wrote: >> Summary: Return Metablocks smaller than dictionary's dark matter. >> >> This change contributed by Jon Masamitsu and myself. To reclaim "dark >> matter" this change adds an array of small blocks by size, created >> lazily, to return Metablocks smaller than the BinaryTreeDictionary >> entry's minimum size. This change also fixed a bug in small object >> double free and adds debugging code to check for this case. >> >> With this change, the submitted test case runs indefinitely. Also >> passed rbt tier 1-5 testing. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 > > I'd prefer it if SmallBlocks didn't expose its implementation by > returning its FreeLists by reference, could you change it to have > * return_chunk() > * get_chunk() > * num_chunks(word_size) > > and get rid of list_at? Okay, I refactored this so small_chunks()->get_block() and return_block() are used rather than getting list_at. I didn't see where you got num_chunks, but list_at is hidden. > > -- > > For how long do you plan to keep BlockFreelist::_all_blocks? > I see that it's debug only but I fear that it could case problems with > running out of native memory in our internal testing of debug builds. I thought about taking it out, but it helped me find the double free bug. I think if we add new code to metadata and have to call deallocate_contents on it, we risk re-introducting these double free bugs. I could take it out. I don't think this gets that big but I'd hate to introduce some sort of OOM bug in our testing. > > BlockFreelist::min_size() is a bit of a misnomer since it returns the > minimum size of blocks to be put on the BlockTreeDictionary, not the > minimum size of blocks which are reusable. How about min_dictionary_size() ? > > Is there any particular reason behind _small_blocks being lazy > allocated and _dictionary not? We lazily create the BlockFreelists with this change. // Lazily create a block_freelist if (block_freelists() == NULL) { _block_freelists = new BlockFreelist(); } So the small_blocks are lazily created in the BlockFreelists but the dictionary is not. I guess if we're going to create the BlockFreelists here, we'll most likely need both and maybe small_blocks need not be lazily created. Was that your suggestion? My concern with this change was all the space used by the small_blocks() but if we're doing any deallocation within the metaspace, at least one of the things will be <12 words. I'll make small_blocks() not be lazily allocated since BlockFreelist are. These are pretty expensive, but should be limited to a few metaspaces. > > I would prefer if BlockFreelist::return_block would perform the checks > in reverse order instead of having a return inside the first if block, > something like > > if (word_size > small_block_max_size) { > dict()->return_chunk(c) > } else if (word_size > small_block_min_size) { > small_blocks()->return_chunk(c) > } else { > // dark matter > } Why? We don't want to cast Metablock into dark matter so check if word_size < small_block_min_size first. Metablock* free_chunk = ::new (p) Metablock(word_size); if (word_size < SmallBlocks::small_block_max_size()) { small_blocks()->return_chunk(word_size); } else { dictionary()->return_chunk(free_chunk); } > > > For BlockFreelist::get_block I realize that the code is a bit more > complex but this also raises a few questions. > * When we allocate from the dictionary we search for a block which is > at least as large as we ask for and split it, returning whatever was > left back to the free list. We don't search for "large enough" blocks > from the small blocks manager, is that intentional or just to keep the > code simple (I like simple)? I'm glad you asked about this so I could give background. It turns out that we deallocate metaspace items better this way. I had a version that did exactly what you said. It was a simple sorted linked list of returned blocks < min_dictionary_size (12) where get_block returned the first block where the item would fit. It had some best fit algorithm so if the block returned was a lot bigger, it wouldn't pick it. My implementation could get through 69300 retransformations before the list didn't work anymore (too many small block fragments of the wrong size) and metaspace was exhausted (metaspace was limited to 12M in this test). Jon's implementation ran this test indefinitely. So it's somewhat simple but it worked really well. > > In the last part of get_block where we return the unused part of a > block retrieved from the dictionary uses compares with > BlockFreelist::min_size() which, as I mentioned above, is the min size > for blocks in the dictionary, not the min size for blocks to be reusable. > I think this code can just call return_block unconditionally for any > nonzero value of "unused" and let return_block deal with dark matter. > yes, I'll make that change. > > > Are the changes to Method and ConstMethod the "bug fix to small object > double free"? Yes. > Is the problem that when a method was freed its annotations were > deallocated as well? Could the annotations fields in the "old" > ConstMethod be cleared instead so that the old annotation arrays could > be kept or is that just needless complexity? I'd rather copy the annotations, because I don't know how the old method, which could still be running, might use the annotations. I don't want to mess with that but I see your point. Coleen > > Thanks > /Mikael > > > >> >> Thanks, >> Coleen and Jon From goetz.lindenmaier at sap.com Tue Oct 4 16:48:53 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 4 Oct 2016 16:48:53 +0000 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> Message-ID: Hi Vladimir, This webrev contains all the changes to hotspot needed for the port: http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/ It includes http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/ http://cr.openjdk.java.net/~goetz/wr16/8166561-basic_C1C2_s390/webrev.01/ http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/ which are out for review. Further it includes the one change to relocate the pc-relative instructions where we didn't open a bug for yet, and the new s390-files. Altogether this passed all our tests that were running on the weekend on linuxs390. The s390-files though are not yet fully in shape, I'm still editing them to get rid of legacy stuff and SAP JVM specific code. E.g. all the code guarded by #ifdef SAPJVM will go away in the end. I hope to have the final versions by end of this week. Best regards, Goetz. > -----Original Message----- > From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] > On Behalf Of Vladimir Kozlov > Sent: Montag, 3. Oktober 2016 23:50 > To: Volker Simonis > Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; > build-dev ; HotSpot Open Source Developers > ; Java Core Libs dev at openjdk.java.net> > Subject: Re: RFR: JEP draft for Linux/s3990x port > > Hi Volker, > > Can you prepare combined patch (or set of patches) based on latest > reviews together with s390 code as it will be in final push? > > We want to run it through our pre-integration testing to verify that it > does not have problems. > > Thanks, > Vladimir > > On 9/29/16 11:25 AM, Vladimir Kozlov wrote: > > You need to wait when Mark (OpenJDK Lead) move it to Candidate (or > > other) state: > > > > http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html > > > > Vladimir > > > > On 9/29/16 9:55 AM, Volker Simonis wrote: > >> Hi Vladimir, > >> > >> thanks a lot for reviewing and endorsing the JEP. > >> > >> I've linked all the relevant issues to the JEP (they all have a link > >> to a webrev) and change the state to "Submitted". > >> > >> There's just one more small shared change we need for the port for > >> which we haven't opened a bug now because we are still working on > >> simplifying it. The current version looks as follows: > >> > >> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- > constant_table_offset.patch > >> > >> > >> What are the next steps? Should I add a "jdk9-fc-request" label to t > >> he JEP and add a corresponding "FC Extension Request" comment to it? > >> Or will this be done automatically once I move it to "Candidate"? > >> > >> Is there anything left to do before I can move it to "Candidate" state? > >> > >> Thanks a lot and best regards, > >> Volker > >> > >> > >> > >> > >> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov > >> wrote: > >>> On 9/27/16 10:49 AM, Volker Simonis wrote: > >>>> > >>>> Hi, > >>>> > >>>> can you please review and endorse the following draft JEP for the > >>>> integration of the Linux/s390x port into the jkd9 master repository: > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8166730 > >>> > >>> > >>> Good. > >>> Add links to webrevs in a comment. It will help to get umbrella FC > >>> extension > >>> approval. > >>> > >>>> > >>>> As detailed in the JEP, the Linux/s390x requires very few shared > >>>> changes and we therefore don't foresee any impact on the existing > >>>> platforms at all. Following you can find a short description of the > >>>> planned changes: > >>>> > >>>> hotspot: > >>>> ======= > >>>> > >>>> Out for review: > >>>> 8166560: [s390] Basic enablement of s390 port. > >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr01/ > >>>> > >>>> Reviewed: > >>>> 8166562: C2: Suppress relocations in scratch emit. > >>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > scratch_emit/webrev.01/ > >>>> > >>>> Will send RFR soon (depends on 8166560): > >>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. > >>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > scratch_emit/webrev.01 > >>> > >>> > >>> Wrong link. > >>> > >>> Thanks, > >>> Vladimir > >>> > >>> > >>>> > >>>> We are still investigating the need of these shared changes: > >>>> > >>>> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > 011-pass_PC_to_retAddrOffset.patch > >>>> > >>>> > >>>> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > 016-constant_table_offset.patch > >>>> > >>>> > >>>> And finally the patch with the s390x-only platform files. We are still > >>>> editing these to get them into OpenJdk style and shape. > >>>> Hotspot passes most jck, jtreg and spec tests with these. > >>>> > >>>> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > 101-zFiles.patch > >>>> > >>>> > >>>> top-level repository: > >>>> =============== > >>>> > >>>> The following is just adding some s390x specific compiler flags to > >>>> flags.m4 > >>>> 8166800: [s390] Top-level build changes required for Linux/s390x > >>>> https://bugs.openjdk.java.net/browse/JDK-8166800 > >>>> > >>>> jdk repository: > >>>> ============ > >>>> > >>>> This one just adds a new jvm.cfg file for s390x > >>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x > >>>> https://bugs.openjdk.java.net/browse/JDK-8166801 > >>>> > >>>> > >>>> And finally we plan to do one more change which fixes the jtreg test > >>>> on Linux/s390x. But this is mainly for the correct detection of the > >>>> platform and for excluding the tests which are not appropriate for > >>>> s390x. > >>>> > >>>> Thank you and best regards, > >>>> Volker > >>>> > >>> From goetz.lindenmaier at sap.com Tue Oct 4 16:52:04 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 4 Oct 2016 16:52:04 +0000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: <41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com> References: <4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap> <41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com> Message-ID: Hi David, I fixed the change accordingly: http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/ But do you really want me to incorporate a bugfix for ARM in the port? Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Dienstag, 4. Oktober 2016 09:56 > To: Lindenmaier, Goetz ; Volker Simonis > > Cc: hotspot-dev at openjdk.java.net; core-libs-dev dev at openjdk.java.net> > Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > > Hi Goetz, > > On 28/09/2016 8:26 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > here a new webrev for this change: > > http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr02/ > > > > I reverted the major reorderings in macros.hpp and os_linux.cpp. > > David asked me to do so, and I guess it makes reviewing more simple. > > Thanks. That said ... > > > Also this fixes the issue spotted by David which actually was wrong. > > The other renaming of ARM to ARM32 was correct, though, as > > AARCH64 is defined in both ARM 64-bit ports, and is checked before. > > So the existing case checking ARM is only reached if !LP64. > > I documented this ... > > ... We actually have a bug with the Elf32_* defines in os_linux.cpp > > 1769 #elif (defined ARM) // ARM must come before AARCH64 because the > closed 64-bit port requires so. > 1770 static Elf32_Half running_arch_code=EM_ARM; > > Aarch64 should come first otherwise we will set the wrong value. I've > been told it would only affect an error message but still ... can I ask > you to fix this please. Thanks. > > --- > src/share/vm/code/codeCache.cpp > > I'd prefer to just see something like: > > S390_ONLY(if (_heaps == NULL) return;) // May be true when NMT detail is > used > > --- > > Otherwise seems okay. > > Thanks, > David > > > Also I removed the change in mutex.hpp. Maybe we contribute > > the full change this was part of, but independent of the s390 port. > > > > I withdraw the part of the change to jdk introducing jvm.cfg. Volker wants > to > > do a separate change for this. > > > > Best regards, > > Goetz. > > > > > > > >> -----Original Message----- > >> From: Volker Simonis [mailto:volker.simonis at gmail.com] > >> Sent: Dienstag, 27. September 2016 19:58 > >> To: David Holmes > >> Cc: Lindenmaier, Goetz ; hotspot- > >> dev at openjdk.java.net; core-libs-dev > >> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > >> > >> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes > > >> wrote: > >>> Hi Goetz, > >>> > >>> I see a change not related directly to S390 ie change from ARM to ARM32 > in > >>> src/os/linux/vm/os_linux.cpp > >>> > >> > >> The change looks a little confusing because Goetz reordered the ifdef > >> cascades alphabetically (which I think is good). > >> > >> Besides that, the only real change not related to s390 is indeed the > >> change from ARM to ARM32 which happend two times in the file. > >> > >> @Goetz: have you done this intentionally? > >> > >>> It will be a while before I can go through this in any detail. > >>> > >>> David > >>> > >>> > >>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: > >>>> > >>>> Hi, > >>>> > >>>> This change is part of the s390 port. It contains some basic adaptions > >>>> needed for a full hotspot port for linux s390x. > >>>> > >>>> It defines the required macros, platform names and includes. > >>>> > >>>> The s390 port calles CodeCache::contains() in current_frame(), which is > >>>> used in NMT. As NMT already collects stack traces before the > CodeCache > >> is > >>>> initialized, contains() needs a check for this. > >>>> > >>>> Wherever a row of platforms are listed, I sorted them alphabetically. > >>>> > >>>> The jdk requires the file jvm.cfg. > >>>> > >>>> Please review. I please need a sponsor. > >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >> basic_s390/hotspot.wr01/ > >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/jdk.wr01/ > >>>> > >>>> Best regards, > >>>> Goetz. > >>>> > >>> From igor.ignatyev at oracle.com Tue Oct 4 18:02:19 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 4 Oct 2016 21:02:19 +0300 Subject: RFR(S) : 8166129 : hitting vmassert during gtest execution doesn't generate core and hs_err files In-Reply-To: References: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> <4E0B1A2D-CBF9-4FCF-8F91-A81E632703FA@oracle.com> Message-ID: <9E2482C6-5A56-43F1-A131-93B6EDE29229@oracle.com> > On Oct 4, 2016, at 5:22 AM, Kim Barrett wrote: > >> On Oct 3, 2016, at 4:33 PM, Igor Ignatyev wrote: >> >> Kim, >> >>> Does this change eliminate the need for this (IMO misplaced) bit of code in the vmassert macro? >> unfortunately not, since we still disable fatal error reporting for assert tests, we need to have that code to support TEST_VM_ASSERT_MSG tests ? assert tests which check that expected assert message is printed out. I agree that bit of code does not really look nice, but I?d prefer to work on that issue separately. > > Thanks for the explanation. Is there a CR for that? I?ve file a new one ? https://bugs.openjdk.java.net/browse/JDK-8167089 > > As for this change for 8166129, looks good. thank you for review ? Igor From igor.ignatyev at oracle.com Tue Oct 4 18:08:10 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 4 Oct 2016 21:08:10 +0300 Subject: RFR(S) : 8166925 : several native TESTs should be changed to TEST_VM In-Reply-To: References: Message-ID: <6D6A282D-170C-48B1-9A3A-ED780D6B3298@oracle.com> > On Oct 4, 2016, at 4:13 PM, Vladimir Ivanov wrote: > >> http://cr.openjdk.java.net/~iignatyev/8166925/webrev.00 > Reviewed. thank you for review. > >> there are several tests which depend on initialized JVM, but do not declare it (by using TEST_VM macros). here is the list of those tests along w/ observations where/when they need JVM: >> >> 1. all arrayOopDesc tests use arrayOopDesc::header_size_in_bytes which depends on a jvm state, it reads HeapWordSize and UseCompressedClassPointers and furthermore has an assert which checks that their values haven't changed. >> 2. all utilities/json tests use tty variable (as JSON::_st field), tty initialization is a port of jvm initialization. >> 3. SymbolTable.temp_new_symbol calls JavaThread::current() >> 4. LogFileOutput tests use ResourceMark which calls JavaThread::current() >> 5. LogDecorations.level, uptime, tags and timestamps tests directly or indirectly call os::javaTimeNanos() which assumes that some structures, e.g. Bsd::_timebase_info, are inited >> 6. LogConfigurationTest tests require ResourceMark or os::javaTimeNanos > > Maybe add these comments to the tests? I don?t think it will give lots of good, since the mentioned problems are just the first which tests bumped into, there can be other problems in the same tests. > > Best regards, > Vladimir Ivanov From vladimir.x.ivanov at oracle.com Tue Oct 4 18:48:20 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 4 Oct 2016 21:48:20 +0300 Subject: RFR(S) : 8166925 : several native TESTs should be changed to TEST_VM In-Reply-To: <6D6A282D-170C-48B1-9A3A-ED780D6B3298@oracle.com> References: <6D6A282D-170C-48B1-9A3A-ED780D6B3298@oracle.com> Message-ID: <9e307e03-8de5-0b85-3f7d-82a5cd6108c1@oracle.com> >>> 1. all arrayOopDesc tests use arrayOopDesc::header_size_in_bytes which depends on a jvm state, it reads HeapWordSize and UseCompressedClassPointers and furthermore has an assert which checks that their values haven't changed. >>> 2. all utilities/json tests use tty variable (as JSON::_st field), tty initialization is a port of jvm initialization. >>> 3. SymbolTable.temp_new_symbol calls JavaThread::current() >>> 4. LogFileOutput tests use ResourceMark which calls JavaThread::current() >>> 5. LogDecorations.level, uptime, tags and timestamps tests directly or indirectly call os::javaTimeNanos() which assumes that some structures, e.g. Bsd::_timebase_info, are inited >>> 6. LogConfigurationTest tests require ResourceMark or os::javaTimeNanos >> >> Maybe add these comments to the tests? > I don?t think it will give lots of good, since the mentioned problems are just the first which tests bumped into, there can be other problems in the same tests. Makes sense, looks good then. Best regards, Vladimir Ivanov From kim.barrett at oracle.com Tue Oct 4 23:12:26 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 4 Oct 2016 19:12:26 -0400 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> References: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> Message-ID: <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> > On Oct 4, 2016, at 5:04 AM, David Holmes wrote: > > Hi Kim, > > On 2/10/2016 7:29 AM, Kim Barrett wrote: >> Please review this change to the CMS collector to replace uses of >> klass_or_null with klass_or_null_acquire where needed. >> >> [Reviewing on hotspot-dev; only GC code is involved, but some non-GC >> folks have expressed interest.] >> >> All non-assert uses of klass_or_null are being replaced with the >> acquire variant. In most cases, the size is being accessed soon >> after, and the acquire is needed to ensure order of klass and possible >> inline size access. In a few cases, the need for acquire is less >> immediately obvious: >> >> - CompatibleFreeListSpace::block_is_obj - Size access is not lexically >> apparent, but callers access the size or other data ordered with the >> klass. >> >> - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: >> These perform other operations that need to be properly ordered wrto >> the klass access. > > Can you point me to the release-store that is being paired with these load-acquires please? It?s the release_set_klass recently added here: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/fd16b627ebc5 > Superficially this all seems fine and can't have any correctness issues, it is just a matter of where acquire semantics are actually needed. And it is hard to tell that without understanding exactly how the different read-paths may execute relative to the write-path. In MarkFromRootsClosure::do_bit, the first one is protecting the call to scanOopsInOop. The second is protecting the call to mark_range, which might not be strictly necessary, but seems like this should be pretty rare. And being consistent about the use of klass_or_null_acquire in non-assert contexts seems of value. Though I admit to finding this block of code confusing; it?s unconditional in a product build, but only executed if !_verifying in a debug build. Strange? but it?s been like that ?forever?. In ParMarkFromRootsClosure::do_bit, protecting call to scan_oops_in_oop. The scanners need to be protected against reading an uninitialize length or uninitialized fields / array elements. From david.holmes at oracle.com Wed Oct 5 02:16:53 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Oct 2016 12:16:53 +1000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: References: <4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap> <41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com> Message-ID: On 5/10/2016 2:52 AM, Lindenmaier, Goetz wrote: > Hi David, > > I fixed the change accordingly: > http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/ > > But do you really want me to incorporate a bugfix for ARM in the port? Sure - it is only an error message and we've probably never encountered it. All the other code has the case for Aarch64 first so this is no different - and no need to comment it as it isn't commented upon elsewhere. Thanks, David > Best regards, > Goetz. > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Dienstag, 4. Oktober 2016 09:56 >> To: Lindenmaier, Goetz ; Volker Simonis >> >> Cc: hotspot-dev at openjdk.java.net; core-libs-dev > dev at openjdk.java.net> >> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. >> >> Hi Goetz, >> >> On 28/09/2016 8:26 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> here a new webrev for this change: >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/hotspot.wr02/ >>> >>> I reverted the major reorderings in macros.hpp and os_linux.cpp. >>> David asked me to do so, and I guess it makes reviewing more simple. >> >> Thanks. That said ... >> >>> Also this fixes the issue spotted by David which actually was wrong. >>> The other renaming of ARM to ARM32 was correct, though, as >>> AARCH64 is defined in both ARM 64-bit ports, and is checked before. >>> So the existing case checking ARM is only reached if !LP64. >>> I documented this ... >> >> ... We actually have a bug with the Elf32_* defines in os_linux.cpp >> >> 1769 #elif (defined ARM) // ARM must come before AARCH64 because the >> closed 64-bit port requires so. >> 1770 static Elf32_Half running_arch_code=EM_ARM; >> >> Aarch64 should come first otherwise we will set the wrong value. I've >> been told it would only affect an error message but still ... can I ask >> you to fix this please. Thanks. >> >> --- >> src/share/vm/code/codeCache.cpp >> >> I'd prefer to just see something like: >> >> S390_ONLY(if (_heaps == NULL) return;) // May be true when NMT detail is >> used >> >> --- >> >> Otherwise seems okay. >> >> Thanks, >> David >> >>> Also I removed the change in mutex.hpp. Maybe we contribute >>> the full change this was part of, but independent of the s390 port. >>> >>> I withdraw the part of the change to jdk introducing jvm.cfg. Volker wants >> to >>> do a separate change for this. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>>> -----Original Message----- >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com] >>>> Sent: Dienstag, 27. September 2016 19:58 >>>> To: David Holmes >>>> Cc: Lindenmaier, Goetz ; hotspot- >>>> dev at openjdk.java.net; core-libs-dev >>>> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. >>>> >>>> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes >> >>>> wrote: >>>>> Hi Goetz, >>>>> >>>>> I see a change not related directly to S390 ie change from ARM to ARM32 >> in >>>>> src/os/linux/vm/os_linux.cpp >>>>> >>>> >>>> The change looks a little confusing because Goetz reordered the ifdef >>>> cascades alphabetically (which I think is good). >>>> >>>> Besides that, the only real change not related to s390 is indeed the >>>> change from ARM to ARM32 which happend two times in the file. >>>> >>>> @Goetz: have you done this intentionally? >>>> >>>>> It will be a while before I can go through this in any detail. >>>>> >>>>> David >>>>> >>>>> >>>>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> This change is part of the s390 port. It contains some basic adaptions >>>>>> needed for a full hotspot port for linux s390x. >>>>>> >>>>>> It defines the required macros, platform names and includes. >>>>>> >>>>>> The s390 port calles CodeCache::contains() in current_frame(), which is >>>>>> used in NMT. As NMT already collects stack traces before the >> CodeCache >>>> is >>>>>> initialized, contains() needs a check for this. >>>>>> >>>>>> Wherever a row of platforms are listed, I sorted them alphabetically. >>>>>> >>>>>> The jdk requires the file jvm.cfg. >>>>>> >>>>>> Please review. I please need a sponsor. >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>>> basic_s390/hotspot.wr01/ >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/jdk.wr01/ >>>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>> From shafi.s.ahmad at oracle.com Wed Oct 5 06:42:46 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 4 Oct 2016 23:42:46 -0700 (PDT) Subject: [8u] RFR for JDK-8166872: GPL header in /hotspot/src/share/vm/gc_implementation/g1/g1RemSetSummary.cpp Message-ID: <92044371-2cdb-430d-a090-3332f85efe9b@default> Hi All, Please review the trivial change of adding a comma after year in Copyright section. Summary: Does not get it's GPL header swapped to commercial in the licensee source bundle because of missing a comma after the second year. Open webrev: http://cr.openjdk.java.net/~shshahma/8166872/webrev.00/ Bug link: https://bugs.openjdk.java.net/browse/JDK-8166872 Regards, Shafi From david.holmes at oracle.com Wed Oct 5 08:15:08 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Oct 2016 18:15:08 +1000 Subject: [8u] RFR for JDK-8166872: GPL header in /hotspot/src/share/vm/gc_implementation/g1/g1RemSetSummary.cpp In-Reply-To: <92044371-2cdb-430d-a090-3332f85efe9b@default> References: <92044371-2cdb-430d-a090-3332f85efe9b@default> Message-ID: Looks fine. Thanks, David On 5/10/2016 4:42 PM, Shafi Ahmad wrote: > Hi All, > > Please review the trivial change of adding a comma after year in Copyright section. > > Summary: > Does not get it's GPL header swapped to commercial in the licensee source > bundle because of missing a comma after the second year. > > Open webrev: http://cr.openjdk.java.net/~shshahma/8166872/webrev.00/ > Bug link: https://bugs.openjdk.java.net/browse/JDK-8166872 > > Regards, > Shafi > From shafi.s.ahmad at oracle.com Wed Oct 5 08:36:05 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 5 Oct 2016 01:36:05 -0700 (PDT) Subject: [8u] RFR for JDK-8166872: GPL header in /hotspot/src/share/vm/gc_implementation/g1/g1RemSetSummary.cpp In-Reply-To: References: <92044371-2cdb-430d-a090-3332f85efe9b@default> Message-ID: Thank you David for review. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, October 05, 2016 1:45 PM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8166872: GPL header in > /hotspot/src/share/vm/gc_implementation/g1/g1RemSetSummary.cpp > > Looks fine. > > Thanks, > David > > On 5/10/2016 4:42 PM, Shafi Ahmad wrote: > > Hi All, > > > > Please review the trivial change of adding a comma after year in Copyright > section. > > > > Summary: > > Does not get it's GPL header swapped to commercial in the licensee > > source bundle because of missing a comma after the second year. > > > > Open webrev: > http://cr.openjdk.java.net/~shshahma/8166872/webrev.00/ > > Bug link: https://bugs.openjdk.java.net/browse/JDK-8166872 > > > > Regards, > > Shafi > > From thomas.schatzl at oracle.com Wed Oct 5 08:40:25 2016 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Oct 2016 10:40:25 +0200 Subject: [8u] RFR for JDK-8166872: GPL header in /hotspot/src/share/vm/gc_implementation/g1/g1RemSetSummary.cpp In-Reply-To: <92044371-2cdb-430d-a090-3332f85efe9b@default> References: <92044371-2cdb-430d-a090-3332f85efe9b@default> Message-ID: <1475656825.2626.27.camel@oracle.com> Hi, On Tue, 2016-10-04 at 23:42 -0700, Shafi Ahmad wrote: > Hi All, > > Please review the trivial change of adding a comma after year in > Copyright section. > > Summary:? > Does not get it's GPL header swapped to commercial in the licensee > source > bundle because of missing a comma after the second year.? > > Open webrev: http://cr.openjdk.java.net/~shshahma/8166872/webrev.00/ > Bug link: https://bugs.openjdk.java.net/browse/JDK-8166872 ? looks good. Thomas From shafi.s.ahmad at oracle.com Wed Oct 5 08:44:50 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 5 Oct 2016 01:44:50 -0700 (PDT) Subject: [8u] RFR for JDK-8166872: GPL header in /hotspot/src/share/vm/gc_implementation/g1/g1RemSetSummary.cpp In-Reply-To: <1475656825.2626.27.camel@oracle.com> References: <92044371-2cdb-430d-a090-3332f85efe9b@default> <1475656825.2626.27.camel@oracle.com> Message-ID: Thanks Thomas for the review. Regards, Shafi > -----Original Message----- > From: Thomas Schatzl > Sent: Wednesday, October 05, 2016 2:10 PM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8166872: GPL header in > /hotspot/src/share/vm/gc_implementation/g1/g1RemSetSummary.cpp > > Hi, > > On Tue, 2016-10-04 at 23:42 -0700, Shafi Ahmad wrote: > > Hi All, > > > > Please review the trivial change of adding a comma after year in > > Copyright section. > > > > Summary: > > Does not get it's GPL header swapped to commercial in the licensee > > source bundle because of missing a comma after the second year. > > > > Open webrev: > http://cr.openjdk.java.net/~shshahma/8166872/webrev.00/ > > Bug link: https://bugs.openjdk.java.net/browse/JDK-8166872 > > ? looks good. > > Thomas From doug.simon at oracle.com Wed Oct 5 08:57:30 2016 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 5 Oct 2016 10:57:30 +0200 Subject: ATTENTION: hs-comp is CLOSED for pushes!!! In-Reply-To: <24a59760-c99d-4c58-21a2-4d2c5c2b6cff@oracle.com> References: <24a59760-c99d-4c58-21a2-4d2c5c2b6cff@oracle.com> Message-ID: <3F8D6395-032E-480E-B39F-97ACE1497E8D@oracle.com> When we start to push into hs instead of hs-comp, I assume webrevs should now be sent to hotspot-dev at openjdk.java.net in addition to hotspot-compiler-dev at openjdk.java.net? -Doug > On 03 Oct 2016, at 22:31, Vladimir Kozlov wrote: > > I have to repeat. > > Please, wait until October 7th with pushes. Or push into jdk9/hs. > > According to proposed dates for "Merging jdk9/hs-comp with jdk9/hs" we are closing hs-comp/ for pushes except new nightly bugs fixes. > > We want to get clean Nightly testing results in preparation for merge jdk9/hs-comp with jdk9/hs on Friday, October 7th. > > Thanks, > Vladimir From david.holmes at oracle.com Wed Oct 5 11:39:34 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Oct 2016 21:39:34 +1000 Subject: ATTENTION: hs-comp is CLOSED for pushes!!! In-Reply-To: <3F8D6395-032E-480E-B39F-97ACE1497E8D@oracle.com> References: <24a59760-c99d-4c58-21a2-4d2c5c2b6cff@oracle.com> <3F8D6395-032E-480E-B39F-97ACE1497E8D@oracle.com> Message-ID: On 5/10/2016 6:57 PM, Doug Simon wrote: > When we start to push into hs instead of hs-comp, I assume webrevs should now be sent to hotspot-dev at openjdk.java.net in addition to hotspot-compiler-dev at openjdk.java.net? No. runtime and gc changes still get reviewed on the team aliases and only include hotspot-dev when appropriate. David > -Doug > >> On 03 Oct 2016, at 22:31, Vladimir Kozlov wrote: >> >> I have to repeat. >> >> Please, wait until October 7th with pushes. Or push into jdk9/hs. >> >> According to proposed dates for "Merging jdk9/hs-comp with jdk9/hs" we are closing hs-comp/ for pushes except new nightly bugs fixes. >> >> We want to get clean Nightly testing results in preparation for merge jdk9/hs-comp with jdk9/hs on Friday, October 7th. >> >> Thanks, >> Vladimir > From doug.simon at oracle.com Wed Oct 5 12:09:38 2016 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 5 Oct 2016 14:09:38 +0200 Subject: ATTENTION: hs-comp is CLOSED for pushes!!! In-Reply-To: References: <24a59760-c99d-4c58-21a2-4d2c5c2b6cff@oracle.com> <3F8D6395-032E-480E-B39F-97ACE1497E8D@oracle.com> Message-ID: <5285C562-97EA-437D-B2DB-F3AF4D838B6D@oracle.com> > On 05 Oct 2016, at 13:39, David Holmes wrote: > > On 5/10/2016 6:57 PM, Doug Simon wrote: >> When we start to push into hs instead of hs-comp, I assume webrevs should now be sent to hotspot-dev at openjdk.java.net in addition to hotspot-compiler-dev at openjdk.java.net? > > No. runtime and gc changes still get reviewed on the team aliases and only include hotspot-dev when appropriate. Ok, thanks for the clarification. -Doug >> -Doug >> >>> On 03 Oct 2016, at 22:31, Vladimir Kozlov wrote: >>> >>> I have to repeat. >>> >>> Please, wait until October 7th with pushes. Or push into jdk9/hs. >>> >>> According to proposed dates for "Merging jdk9/hs-comp with jdk9/hs" we are closing hs-comp/ for pushes except new nightly bugs fixes. >>> >>> We want to get clean Nightly testing results in preparation for merge jdk9/hs-comp with jdk9/hs on Friday, October 7th. >>> >>> Thanks, >>> Vladimir >> From volker.simonis at gmail.com Wed Oct 5 14:43:19 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 5 Oct 2016 16:43:19 +0200 Subject: RFR(XS): 8166800: [s390] Top-level build changes required for Linux/s390x Message-ID: Hi, can I please have a review and sponsor for the following tiny top-level build change required for building the OpenJDK on Linux/s390: http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166800/ https://bugs.openjdk.java.net/browse/JDK-8166800 All this change does is to add some s390-specifc C and C++ flags for the Hotspot and the class-library build which do not affect any of the existing platforms. Thank you and best regards, Volker From stanislav.smirnov at oracle.com Wed Oct 5 14:56:24 2016 From: stanislav.smirnov at oracle.com (Stanislav Smirnov) Date: Wed, 5 Oct 2016 17:56:24 +0300 Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test Message-ID: <5346A7A7-672C-4DF8-83CF-FB108355C171@oracle.com> Hi, Please review this fix for JDK-8165687 . Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files for uniformity to meet Oracle requirements. JBS bug: https://bugs.openjdk.java.net/browse/JDK-8165687 Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ Best regards, Stanislav Smirnov From erik.joelsson at oracle.com Wed Oct 5 15:07:10 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 5 Oct 2016 17:07:10 +0200 Subject: RFR(XS): 8166800: [s390] Top-level build changes required for Linux/s390x In-Reply-To: References: Message-ID: <6fbd2c03-7d6a-25ef-1072-9d3f7151f973@oracle.com> Looks ok to me. I can sponsor this tomorrow. /Erik On 2016-10-05 16:43, Volker Simonis wrote: > Hi, > > can I please have a review and sponsor for the following tiny > top-level build change required for building the OpenJDK on > Linux/s390: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166800/ > https://bugs.openjdk.java.net/browse/JDK-8166800 > > All this change does is to add some s390-specifc C and C++ flags for > the Hotspot and the class-library build which do not affect any of the > existing platforms. > > Thank you and best regards, > Volker From volker.simonis at gmail.com Wed Oct 5 15:22:15 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 5 Oct 2016 17:22:15 +0200 Subject: RFR(XS): 8166800: [s390] Top-level build changes required for Linux/s390x In-Reply-To: <6fbd2c03-7d6a-25ef-1072-9d3f7151f973@oracle.com> References: <6fbd2c03-7d6a-25ef-1072-9d3f7151f973@oracle.com> Message-ID: That would be really great! Could you please do it in the jdk9/hs forest as that's the place where we will need it first (i.e. when integrating the whole s390 hotspot platform files). Thanks, Volker On Wed, Oct 5, 2016 at 5:07 PM, Erik Joelsson wrote: > Looks ok to me. I can sponsor this tomorrow. > > /Erik > > > > On 2016-10-05 16:43, Volker Simonis wrote: >> >> Hi, >> >> can I please have a review and sponsor for the following tiny >> top-level build change required for building the OpenJDK on >> Linux/s390: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166800/ >> https://bugs.openjdk.java.net/browse/JDK-8166800 >> >> All this change does is to add some s390-specifc C and C++ flags for >> the Hotspot and the class-library build which do not affect any of the >> existing platforms. >> >> Thank you and best regards, >> Volker > > From vladimir.kozlov at oracle.com Wed Oct 5 15:28:54 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 5 Oct 2016 08:28:54 -0700 Subject: ATTENTION: hs-comp is CLOSED for pushes!!! In-Reply-To: <5285C562-97EA-437D-B2DB-F3AF4D838B6D@oracle.com> References: <24a59760-c99d-4c58-21a2-4d2c5c2b6cff@oracle.com> <3F8D6395-032E-480E-B39F-97ACE1497E8D@oracle.com> <5285C562-97EA-437D-B2DB-F3AF4D838B6D@oracle.com> Message-ID: <37b7abc7-d869-19e5-52c9-3eae195ab2c7@oracle.com> On 10/5/16 5:09 AM, Doug Simon wrote: > >> On 05 Oct 2016, at 13:39, David Holmes wrote: >> >> On 5/10/2016 6:57 PM, Doug Simon wrote: >>> When we start to push into hs instead of hs-comp, I assume webrevs should now be sent to hotspot-dev at openjdk.java.net in addition to hotspot-compiler-dev at openjdk.java.net? >> >> No. runtime and gc changes still get reviewed on the team aliases and only include hotspot-dev when appropriate. > > Ok, thanks for the clarification. Yes, only repository is changed. Mailing lists, RBT testing and review process are still the same. Vladimir > > -Doug > >>> -Doug >>> >>>> On 03 Oct 2016, at 22:31, Vladimir Kozlov wrote: >>>> >>>> I have to repeat. >>>> >>>> Please, wait until October 7th with pushes. Or push into jdk9/hs. >>>> >>>> According to proposed dates for "Merging jdk9/hs-comp with jdk9/hs" we are closing hs-comp/ for pushes except new nightly bugs fixes. >>>> >>>> We want to get clean Nightly testing results in preparation for merge jdk9/hs-comp with jdk9/hs on Friday, October 7th. >>>> >>>> Thanks, >>>> Vladimir >>> > From Derek.White at cavium.com Wed Oct 5 16:10:21 2016 From: Derek.White at cavium.com (White, Derek) Date: Wed, 5 Oct 2016 16:10:21 +0000 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> References: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> Message-ID: -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett Sent: Tuesday, October 04, 2016 7:12 PM To: David Holmes Cc: hotspot-dev Source Developers Subject: Re: RFR: 8166862: CMS needs klass_or_null_acquire > On Oct 4, 2016, at 5:04 AM, David Holmes > wrote: > > Hi Kim, > > On 2/10/2016 7:29 AM, Kim Barrett wrote: >> Please review this change to the CMS collector to replace uses of >> klass_or_null with klass_or_null_acquire where needed. >> >> [Reviewing on hotspot-dev; only GC code is involved, but some non-GC >> folks have expressed interest.] >> >> All non-assert uses of klass_or_null are being replaced with the >> acquire variant. In most cases, the size is being accessed soon >> after, and the acquire is needed to ensure order of klass and >> possible inline size access. In a few cases, the need for acquire is >> less immediately obvious: >> >> - CompatibleFreeListSpace::block_is_obj - Size access is not >> lexically apparent, but callers access the size or other data >> ordered with the klass. >> >> - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: >> These perform other operations that need to be properly ordered wrto >> the klass access. > > Can you point me to the release-store that is being paired with these load-acquires please? It's the release_set_klass recently added here: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/fd16b627ebc5 FYI I'm still looking, but I think this is true: I think a release-store also needs to be in the interpreter and jitted code, for processors that need it (e.g. aarch64 and ppc). For example in MacroAssembler::store_klass. Those processors have a storestore after object initialization to ensure that Java threads don't see uninitialized fields, but don't have a store-release on setting the class to prevent CMS from tripping over a mis-ordered header initialization. The nominal store order may also be wrong (e.g. "length" before "klass"). See C1_MacroAssembler::initialize_header() If this sounds right, I can file a bug... - Derek From iris.clark at oracle.com Wed Oct 5 16:32:57 2016 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 5 Oct 2016 09:32:57 -0700 (PDT) Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test In-Reply-To: <5346A7A7-672C-4DF8-83CF-FB108355C171@oracle.com> References: <5346A7A7-672C-4DF8-83CF-FB108355C171@oracle.com> Message-ID: Hi. > Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ These appear to be the expected changes. I find the updates to test/gc/TestHumongousReferenceObject.java interesting. You needed to add the word "Public" to the phrase "General Public License". It looks like somewhere during development, there was an overly enthusiastic search and replace of "Public" with "". Regardless, it looks fine now. Thanks, Iris (not a Reviewer) -----Original Message----- From: Stanislav Smirnov Sent: Wednesday, October 05, 2016 7:56 AM To: hotspot-dev at openjdk.java.net Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test Hi, Please review this fix for JDK-8165687 . Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files for uniformity to meet Oracle requirements. JBS bug: https://bugs.openjdk.java.net/browse/JDK-8165687 Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ Best regards, Stanislav Smirnov From stanislav.smirnov at oracle.com Wed Oct 5 16:41:11 2016 From: stanislav.smirnov at oracle.com (Stanislav Smirnov) Date: Wed, 5 Oct 2016 19:41:11 +0300 Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test In-Reply-To: References: <5346A7A7-672C-4DF8-83CF-FB108355C171@oracle.com> Message-ID: Hi Iris, thanks for a review. Btw, I will also need a sponsor to push it. Best regards, Stanislav Smirnov > On 05 Oct 2016, at 19:32, Iris Clark wrote: > > Hi. > >> Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ > > These appear to be the expected changes. > > I find the updates to test/gc/TestHumongousReferenceObject.java interesting. You needed to add the word "Public" to the phrase > "General Public License". It looks like somewhere during development, there was an overly enthusiastic search and replace of "Public" with "". Regardless, it looks fine now. > > Thanks, > Iris > (not a Reviewer) > > -----Original Message----- > From: Stanislav Smirnov > Sent: Wednesday, October 05, 2016 7:56 AM > To: hotspot-dev at openjdk.java.net > Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test > > Hi, > > Please review this fix for JDK-8165687 . > Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files for uniformity to meet Oracle requirements. > > JBS bug: https://bugs.openjdk.java.net/browse/JDK-8165687 > Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ > > Best regards, > Stanislav Smirnov From stanislav.smirnov at oracle.com Wed Oct 5 16:44:04 2016 From: stanislav.smirnov at oracle.com (Stanislav Smirnov) Date: Wed, 5 Oct 2016 19:44:04 +0300 Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under test/lib Message-ID: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com> Hi, Please review this fix for JDK-8163984 . This one is similar to another one, I have sent earlier. Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files under test/lib for uniformity to meet Oracle requirements. There are two files from test/lib/jdk do I need to include jdk9_dev mailing list? JBS bug: https://bugs.openjdk.java.net/browse/JDK-8163984 Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ Best regards, Stanislav Smirnov From iris.clark at oracle.com Wed Oct 5 16:56:39 2016 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 5 Oct 2016 09:56:39 -0700 (PDT) Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under test/lib In-Reply-To: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com> References: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com> Message-ID: <57e4e124-df94-4b44-a127-c2c75cff2555@default> Hi. > Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ These changes look fine to me. > There are two files from test/lib/jdk do I need to include jdk9_dev mailing list? These files appear to be testing APIs in java.util.function which was introduced in JDK 8 for Lambda. I recommend you use core-libs-dev at ojn instead of the more general jdk9-dev at ojn. Thanks, Iris (not a Reviewer) -----Original Message----- From: Stanislav Smirnov Sent: Wednesday, October 05, 2016 9:44 AM To: hotspot-dev at openjdk.java.net Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under test/lib Hi, Please review this fix for JDK-8163984 . This one is similar to another one, I have sent earlier. Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files under test/lib for uniformity to meet Oracle requirements. There are two files from test/lib/jdk do I need to include jdk9_dev mailing list? JBS bug: https://bugs.openjdk.java.net/browse/JDK-8163984 Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ Best regards, Stanislav Smirnov From aph at redhat.com Wed Oct 5 17:56:18 2016 From: aph at redhat.com (Andrew Haley) Date: Wed, 5 Oct 2016 18:56:18 +0100 Subject: RFR: 8167200: AArch64: Broken stack pointer adjustment in interpreter Message-ID: This is a thinko in the template interpreter. AArch64 has two stack pointers: the system SP and the expression SP. The expression SP must always point to an address greater than or equal to the system SP. When we allocate a new monitor in the interpreter we move the contents of the entire operand stack and then insert a monitor between the operand stack and the frame. So, we need to allocate two more words (the size of a monitor) on the stack. The expression SP is adjusted to allow for this, and we then do a comparison to see if the system SP needs also to be moved. This is wrong: whenever we allocate a monitor we should also unconditionally adjust the system SP. The bug here is that we might decide that we do not need any more space in the system stack but later push several items onto the expression stack. In this case the expression SP might be lower than the system SP, and method arguments are corrupted. This patch changes the logic so that whenever we insert a monitor we unconditionally adjust the system SP. http://cr.openjdk.java.net/~aph/8167200/ Andrew. From dean.long at oracle.com Wed Oct 5 18:18:45 2016 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 5 Oct 2016 11:18:45 -0700 Subject: RFR: 8167200: AArch64: Broken stack pointer adjustment in interpreter In-Reply-To: References: Message-ID: Looks good, but you forgot to remove Label no_adjust. dl On 10/5/16 10:56 AM, Andrew Haley wrote: > This is a thinko in the template interpreter. > > AArch64 has two stack pointers: the system SP and the expression > SP. The expression SP must always point to an address greater than or > equal to the system SP. > > When we allocate a new monitor in the interpreter we move the contents > of the entire operand stack and then insert a monitor between the > operand stack and the frame. So, we need to allocate two more words > (the size of a monitor) on the stack. The expression SP is adjusted to > allow for this, and we then do a comparison to see if the system SP > needs also to be moved. This is wrong: whenever we allocate a monitor > we should also unconditionally adjust the system SP. > > The bug here is that we might decide that we do not need any more > space in the system stack but later push several items onto the > expression stack. In this case the expression SP might be lower than > the system SP, and method arguments are corrupted. > > This patch changes the logic so that whenever we insert a monitor we > unconditionally adjust the system SP. > > http://cr.openjdk.java.net/~aph/8167200/ > > Andrew. From kim.barrett at oracle.com Wed Oct 5 18:42:04 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Oct 2016 14:42:04 -0400 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: References: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> Message-ID: > On Oct 5, 2016, at 12:10 PM, White, Derek wrote: > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett > Sent: Tuesday, October 04, 2016 7:12 PM > To: David Holmes > Cc: hotspot-dev Source Developers > Subject: Re: RFR: 8166862: CMS needs klass_or_null_acquire > > > On Oct 4, 2016, at 5:04 AM, David Holmes wrote: > > > > Hi Kim, > > > > On 2/10/2016 7:29 AM, Kim Barrett wrote: > >> Please review this change to the CMS collector to replace uses of > >> klass_or_null with klass_or_null_acquire where needed. > >> > >> [Reviewing on hotspot-dev; only GC code is involved, but some non-GC > >> folks have expressed interest.] > >> > >> All non-assert uses of klass_or_null are being replaced with the > >> acquire variant. In most cases, the size is being accessed soon > >> after, and the acquire is needed to ensure order of klass and > >> possible inline size access. In a few cases, the need for acquire is > >> less immediately obvious: > >> > >> - CompatibleFreeListSpace::block_is_obj - Size access is not > >> lexically apparent, but callers access the size or other data > >> ordered with the klass. > >> > >> - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: > >> These perform other operations that need to be properly ordered wrto > >> the klass access. > > > > Can you point me to the release-store that is being paired with these load-acquires please? > > It?s the release_set_klass recently added here: > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/fd16b627ebc5 > > FYI > I'm still looking, but I think this is true: > > I think a release-store also needs to be in the interpreter and jitted code, for processors that need it (e.g. aarch64 and ppc). For example in MacroAssembler::store_klass. > > Those processors have a storestore after object initialization to ensure that Java threads don't see uninitialized fields, but don't have a store-release on setting the class to prevent CMS from tripping over a mis-ordered header initialization. The nominal store order may also be wrong (e.g. "length" before "klass"). See C1_MacroAssembler::initialize_header() > > If this sounds right, I can file a bug? Those are all young-gen allocations. The issues that lead to require_set_klass and klass_or_null_acquire pairings only arise with old-gen allocations. All mutator initiated old-gen allocations go through the post_allocation_setup_common path. (That's also the slow path for young-gen allocations, which don't need the require fence, but conditionalizing the fence didn't seem worthwhile.) There is a separate problem that fences are needed when a new object escapes to some other thread. It must be a well-formed object, including having its klass set, before escaping. So a fence is between setting the klass and escape. I think those exist, though Dean has been chasing a bug for a while that might be a missing one. The difference is that these concurrent collectors can form a reference to a blob of memory that is in the process of being turned into a new object, e.g. before its klass has been set. That's where klass_or_null comes into play. G1 almost doesn't do that; see "RFR: 8166607: G1 needs klass_or_null_acquire" and JDK-8166995. From aph at redhat.com Wed Oct 5 18:53:34 2016 From: aph at redhat.com (Andrew Haley) Date: Wed, 5 Oct 2016 19:53:34 +0100 Subject: RFR: 8167200: AArch64: Broken stack pointer adjustment in interpreter In-Reply-To: References: Message-ID: <7a0f9a1c-70cc-00e2-d9bc-c599caf9ed20@redhat.com> On 05/10/16 19:18, dean.long at oracle.com wrote: > Looks good, but you forgot to remove Label no_adjust. D'oh!. :-) OK, I'll take it that it's OK with that label removed. Thanks, Andrew. From dean.long at oracle.com Wed Oct 5 19:11:56 2016 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 5 Oct 2016 12:11:56 -0700 Subject: RFR: 8167200: AArch64: Broken stack pointer adjustment in interpreter In-Reply-To: <7a0f9a1c-70cc-00e2-d9bc-c599caf9ed20@redhat.com> References: <7a0f9a1c-70cc-00e2-d9bc-c599caf9ed20@redhat.com> Message-ID: On 10/5/16 11:53 AM, Andrew Haley wrote: > On 05/10/16 19:18, dean.long at oracle.com wrote: >> Looks good, but you forgot to remove Label no_adjust. > D'oh!. :-) > > OK, I'll take it that it's OK with that label removed. Yes, no new webrev required for me. dl > > Thanks, > > Andrew. > From brent.christian at oracle.com Wed Oct 5 19:38:10 2016 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 5 Oct 2016 12:38:10 -0700 Subject: RFR 8151486 : Class.forName causes memory leak Message-ID: Hi, Please review my fix for 8151486, "Class.forName causes memory leak". Bug: https://bugs.openjdk.java.net/browse/JDK-8151486 Webrev: http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/ The test case reproduces the bug as such: With a SecurityManager enabled, a class ("ClassForName") is loaded from a user classloader (isa URLClassLoader, "LeakedClassLoader"), and from that class makes a call to Class.forName() on the system classloader. (The specific class name isn't so important, I just used java.util.List). The result is that the user's classloader is never collected. The leak occurs because of the following: Class.forName0() is passed the "caller" class, ClassForName. JVM_FindClassFromCaller will "find a class with this name (java.util.List) in this loader (the system classloader), using the caller's (ClassForName's) protection domain. A ProtectionDomain is created for ClassForName, with ProtectionDomain.classloader referring to LeakedClassLoader. This PD is passed to ClassLoader.checkPackageAccess(), and is added to 'domains' of the system classloader (ClassLoader.java line 643). Thus, the system classloader holds a reference to the user's classloader via 'domains'. Nothing is ever removed from 'domains'. In fact, besides being added to, I found no other uses of 'domains' - not in library code, not in hotspot. (From my research, it's questionable if 'domains' was ever used). Removal of 'domains' fixes the leak, and cleans out a little bit of unused code. Automated building and testing (JPRT, RBT) looks fine. Thanks! -Brent From mandy.chung at oracle.com Wed Oct 5 20:19:32 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 5 Oct 2016 13:19:32 -0700 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: References: Message-ID: > On Oct 5, 2016, at 12:38 PM, Brent Christian wrote: > > Hi, > > Please review my fix for 8151486, "Class.forName causes memory leak". > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8151486 > Webrev: > http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/ > Since domains is not used and not intended to keep PD alive and VM maintains its own cache of these initiating PD for performance, removing domains field looks good to me. Nit in the new test: a few long lines. Good to have line breaks. 64 ClassLoader classLoader = new URLClassLoader(new URL[]{jarFilePath.toFile().toURI().toURL()}) { You can call Path.toUri().toURL() 78 Path testClassesDir = FileSystems.getDefault().getPath( You can replace this with Paths.get(?) Otherwise looks good. Mandy From naoto.sato at oracle.com Wed Oct 5 21:08:20 2016 From: naoto.sato at oracle.com (Naoto Sato) Date: Wed, 5 Oct 2016 14:08:20 -0700 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: References: Message-ID: Typo in ClassForNameLeak.java? At line 103, "change" -> "chance"? Naoto On 10/5/16 12:38 PM, Brent Christian wrote: > Hi, > > Please review my fix for 8151486, "Class.forName causes memory leak". > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8151486 > Webrev: > http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/ > > The test case reproduces the bug as such: > > With a SecurityManager enabled, a class ("ClassForName") is loaded from > a user classloader (isa URLClassLoader, "LeakedClassLoader"), and from > that class makes a call to Class.forName() on the system classloader. > (The specific class name isn't so important, I just used > java.util.List). The result is that the user's classloader is never > collected. > > The leak occurs because of the following: > > Class.forName0() is passed the "caller" class, ClassForName. > > JVM_FindClassFromCaller will "find a class with this name > (java.util.List) in this loader (the system classloader), using the > caller's (ClassForName's) protection domain. A ProtectionDomain is > created for ClassForName, with ProtectionDomain.classloader referring to > LeakedClassLoader. > > This PD is passed to ClassLoader.checkPackageAccess(), and is added to > 'domains' of the system classloader (ClassLoader.java line 643). Thus, > the system classloader holds a reference to the user's classloader via > 'domains'. > > Nothing is ever removed from 'domains'. In fact, besides being added > to, I found no other uses of 'domains' - not in library code, not in > hotspot. (From my research, it's questionable if 'domains' was ever used). > > Removal of 'domains' fixes the leak, and cleans out a little bit of > unused code. > > Automated building and testing (JPRT, RBT) looks fine. > > Thanks! > -Brent > From david.holmes at oracle.com Wed Oct 5 21:26:18 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Oct 2016 07:26:18 +1000 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: References: Message-ID: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> On 6/10/2016 6:19 AM, Mandy Chung wrote: > >> On Oct 5, 2016, at 12:38 PM, Brent Christian wrote: >> >> Hi, >> >> Please review my fix for 8151486, "Class.forName causes memory leak". >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8151486 >> Webrev: >> http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/ >> > > Since domains is not used and not intended to keep PD alive and VM maintains its own cache of these initiating PD for performance, removing domains field looks good to me. What then is intended to keep PD alive? Or do we not need to keep them alive? I have always assumed that domains was used to keep the PD alive for as long as the classloader is alive. Thanks, David ----- > Nit in the new test: a few long lines. Good to have line breaks. > > 64 ClassLoader classLoader = new URLClassLoader(new URL[]{jarFilePath.toFile().toURI().toURL()}) { > > You can call Path.toUri().toURL() > > 78 Path testClassesDir = FileSystems.getDefault().getPath( > > You can replace this with Paths.get(?) > > Otherwise looks good. > > Mandy > From mikael.vidstedt at oracle.com Wed Oct 5 21:37:12 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 5 Oct 2016 14:37:12 -0700 Subject: Merging jdk9/hs-comp with jdk9/hs In-Reply-To: References: Message-ID: Having heard no objections, we?re looking to move forward with the consolidation. We of course want to make sure that what?s currently in hs-comp is ready for (the last) integration, and due to some technical difficulties we?re still waiting for the relevant test results. We?re still hoping to have everything ready to go on Friday, but if not we may have to postpone the merge. Cheers, Mikael > On Sep 29, 2016, at 11:23 AM, Mikael Vidstedt wrote: > > > All, > > Over the last two years we have worked towards simplifying the mercurial forest structure for Hotspot development by consolidating and removing forests. Last year we moved[1] the GC development work from jdk9/hs-gc[2] to jdk9/hs-rt[3], and earlier this year we further consolidated forests by moving[4] the jdk9/hs-rt work directly to the jdk9/hs[5] - aka. "hs main" - forest. The reduction in forests has led to less work spent on integration, faster quality feedback, more efficient hardware utilization, etc. All in all, the experience so far has been a good one. > > However, the Hotspot (JIT) compiler changes are still integrated through the jdk9/hs-comp[6] forest, and are tested separately before they are (bulk) integrated up to jdk9/hs. The separate forest of course comes with the expected integration/gatekeeper overhead and lead times. Since JDK 9 development is ramping down, we expect the number of changes to ramp down as well. This means that the benefit of having a separate jdk9/hs-comp forest is going to reduce over time. Now seems like a good time to look at doing a final hotspot forest consolidation. > > In line with this, we propose closing down jdk9/hs-comp and doing all remaining JDK 9 Hotspot development to jdk9/hs. We propose that this final forest consolidation to take place on Friday, October 7th. Much like the earlier forest consolidations the jdk9/hs-comp forest will be kept around, but will be locked down so that no accidental integrations are made to it. Any work in flight based on jdk9/hs-comp would have to be rebased on jdk9/hs. > > The earlier forest consolidations have gone very smoothly, and we expect this one to go smoothly as well, but if for some reason things don't work out we will of course have an option to go back to what we have today. That said, before using the escape hatch and reverting back, we will of course want to try to address any issues - thank you in advance for your patience while we work through any such issues. > > Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Wednesday, 5 October 2016, we will go ahead with the forest consolidation. > > Cheers, > Mikael > > [1]http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/thread.html > [2]http://hg.openjdk.java.net/jdk9/hs-gc > [3]http://hg.openjdk.java.net/jdk9/hs-rt > [4]http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-March/022218.html > [5]http://hg.openjdk.java.net/jdk9/hs > [6]http://hg.openjdk.java.net/jdk9/hs-comp > From david.holmes at oracle.com Wed Oct 5 21:45:09 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Oct 2016 07:45:09 +1000 Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test In-Reply-To: References: <5346A7A7-672C-4DF8-83CF-FB108355C171@oracle.com> Message-ID: Hi Stanislav, On 6/10/2016 2:41 AM, Stanislav Smirnov wrote: > Hi Iris, > > thanks for a review. > Btw, I will also need a sponsor to push it. Reviewed! And I will sponsor for you. I will push this to jdk9/hs to avoid future potential conflicts with test changes if that is okay? Or do you want it to go to jdk9/dev? Thanks, David > Best regards, > Stanislav Smirnov > > > > > >> On 05 Oct 2016, at 19:32, Iris Clark wrote: >> >> Hi. >> >>> Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ >> >> These appear to be the expected changes. >> >> I find the updates to test/gc/TestHumongousReferenceObject.java interesting. You needed to add the word "Public" to the phrase >> "General Public License". It looks like somewhere during development, there was an overly enthusiastic search and replace of "Public" with "". Regardless, it looks fine now. >> >> Thanks, >> Iris >> (not a Reviewer) >> >> -----Original Message----- >> From: Stanislav Smirnov >> Sent: Wednesday, October 05, 2016 7:56 AM >> To: hotspot-dev at openjdk.java.net >> Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test >> >> Hi, >> >> Please review this fix for JDK-8165687 . >> Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files for uniformity to meet Oracle requirements. >> >> JBS bug: https://bugs.openjdk.java.net/browse/JDK-8165687 >> Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ >> >> Best regards, >> Stanislav Smirnov > From brent.christian at oracle.com Wed Oct 5 21:48:37 2016 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 5 Oct 2016 14:48:37 -0700 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: References: Message-ID: Yes! Good catch, thanks. -Brent On 10/5/16 2:08 PM, Naoto Sato wrote: > Typo in ClassForNameLeak.java? At line 103, "change" -> "chance"? > > Naoto > > On 10/5/16 12:38 PM, Brent Christian wrote: >> Hi, >> >> Please review my fix for 8151486, "Class.forName causes memory leak". >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8151486 >> Webrev: >> http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/ >> >> The test case reproduces the bug as such: >> >> With a SecurityManager enabled, a class ("ClassForName") is loaded from >> a user classloader (isa URLClassLoader, "LeakedClassLoader"), and from >> that class makes a call to Class.forName() on the system classloader. >> (The specific class name isn't so important, I just used >> java.util.List). The result is that the user's classloader is never >> collected. >> >> The leak occurs because of the following: >> >> Class.forName0() is passed the "caller" class, ClassForName. >> >> JVM_FindClassFromCaller will "find a class with this name >> (java.util.List) in this loader (the system classloader), using the >> caller's (ClassForName's) protection domain. A ProtectionDomain is >> created for ClassForName, with ProtectionDomain.classloader referring to >> LeakedClassLoader. >> >> This PD is passed to ClassLoader.checkPackageAccess(), and is added to >> 'domains' of the system classloader (ClassLoader.java line 643). Thus, >> the system classloader holds a reference to the user's classloader via >> 'domains'. >> >> Nothing is ever removed from 'domains'. In fact, besides being added >> to, I found no other uses of 'domains' - not in library code, not in >> hotspot. (From my research, it's questionable if 'domains' was ever >> used). >> >> Removal of 'domains' fixes the leak, and cleans out a little bit of >> unused code. >> >> Automated building and testing (JPRT, RBT) looks fine. >> >> Thanks! >> -Brent >> From mandy.chung at oracle.com Wed Oct 5 22:55:20 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 5 Oct 2016 15:55:20 -0700 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> References: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> Message-ID: <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com> > On Oct 5, 2016, at 2:26 PM, David Holmes wrote: > > On 6/10/2016 6:19 AM, Mandy Chung wrote: >> >> >> Since domains is not used and not intended to keep PD alive and VM maintains its own cache of these initiating PD for performance, removing domains field looks good to me. > > What then is intended to keep PD alive? Or do we not need to keep them alive? > > I have always assumed that domains was used to keep the PD alive for as long as the classloader is alive. Note that PD is the protection domain of an initiating class that resolves a target type T. PD is kept in T?s class loader L. It?s not the protection domains of the classes defined by L. VM keeps its own cache of protection domains that have been validated with a resolved type. VM does not depend on ClassLoader::domains field to keep it alive. IIUC, it?s kept as a weak reference in VM which is what we want here. Brent did the archeology and found that the initial fix introduced the domains field attempted to use it. But the initial fix was back out and reimplemented and domains is not used. In any case, this field is not used and depended by VM. Mandy From david.holmes at oracle.com Wed Oct 5 23:02:56 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Oct 2016 09:02:56 +1000 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com> References: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com> Message-ID: <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com> On 6/10/2016 8:55 AM, Mandy Chung wrote: > >> On Oct 5, 2016, at 2:26 PM, David Holmes wrote: >> >> On 6/10/2016 6:19 AM, Mandy Chung wrote: >>> >>> >>> Since domains is not used and not intended to keep PD alive and VM maintains its own cache of these initiating PD for performance, removing domains field looks good to me. >> >> What then is intended to keep PD alive? Or do we not need to keep them alive? >> >> I have always assumed that domains was used to keep the PD alive for as long as the classloader is alive. > > > Note that PD is the protection domain of an initiating class that resolves a target type T. PD is kept in T?s class loader L. It?s not the protection domains of the classes defined by L. VM keeps its own cache of protection domains that have been validated with a resolved type. VM does not depend on ClassLoader::domains field to keep it alive. IIUC, it?s kept as a weak reference in VM which is what we want here. Okay but this will still affect the lifecycle of the PDs because without the strong reference in L, those weak references in the VM will quickly be cleared. I'm unclear what the implications of this might be, but I doubt regular short-lived tests would expose any issues. David ----- > Brent did the archeology and found that the initial fix introduced the domains field attempted to use it. But the initial fix was back out and reimplemented and domains is not used. In any case, this field is not used and depended by VM. > > Mandy > From mandy.chung at oracle.com Wed Oct 5 23:19:20 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 5 Oct 2016 16:19:20 -0700 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com> References: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com> <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com> Message-ID: <56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com> > On Oct 5, 2016, at 4:02 PM, David Holmes wrote: > > On 6/10/2016 8:55 AM, Mandy Chung wrote: >> >> >> Note that PD is the protection domain of an initiating class that resolves a target type T. PD is kept in T?s class loader L. It?s not the protection domains of the classes defined by L. VM keeps its own cache of protection domains that have been validated with a resolved type. VM does not depend on ClassLoader::domains field to keep it alive. IIUC, it?s kept as a weak reference in VM which is what we want here. > > Okay but this will still affect the lifecycle of the PDs because without the strong reference in L, those weak references in the VM will quickly be cleared. I'm unclear what the implications of this might be, but I doubt regular short-lived tests would expose any issues. There is no issue there. PD will be GC?ed together when the classes along with its class loader are unloaded. These class loaders may be one or more and they are L1, L2 ?. != L. Mandy From Derek.White at cavium.com Wed Oct 5 23:36:34 2016 From: Derek.White at cavium.com (White, Derek) Date: Wed, 5 Oct 2016 23:36:34 +0000 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: References: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> Message-ID: Thanks for the reminder Kim! (Boy, you walk away from this code for a few days...) So we have the case where the slow-path "runtime" allocators always do a store_release on the klass (if CMS might run), but never do a StoreStore barrier after initializing the object fields/elements. The interpreter/compilers are responsible for that (typically done by generated code). That leaves other runtime callers of the CollectedHeap allocators to also make sure that they have the correct barriers. It's not yet clear that they do. It might help to separate allocators directly called by the interpreter/compiler. Here's a 2-deep fan-out: InstanceMirrorKlass::allocate_instance InstanceKlass::allocate_instance ArrayKlass::allocate_arrayArray InstanceKlass::allocate_objArray - oopFactory::new_objArray ObjArrayKlass::allocate - Deoptimization::realloc_objects - oopFactory ::new_objectArray TypeArrayKlass::allocate_common - oopFactory::new_metaDataArray - oopFactory::new_typeArray_nozero (OK) JVM_Clone (OK?). - Derek -----Original Message----- From: Kim Barrett [mailto:kim.barrett at oracle.com] Sent: Wednesday, October 05, 2016 2:42 PM To: White, Derek Cc: David Holmes ; hotspot-dev Source Developers Subject: Re: RFR: 8166862: CMS needs klass_or_null_acquire > On Oct 5, 2016, at 12:10 PM, White, Derek wrote: > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Kim Barrett > Sent: Tuesday, October 04, 2016 7:12 PM > To: David Holmes > Cc: hotspot-dev Source Developers > Subject: Re: RFR: 8166862: CMS needs klass_or_null_acquire > > > On Oct 4, 2016, at 5:04 AM, David Holmes wrote: > > > > Hi Kim, > > > > On 2/10/2016 7:29 AM, Kim Barrett wrote: > >> Please review this change to the CMS collector to replace uses of > >> klass_or_null with klass_or_null_acquire where needed. > >> > >> [Reviewing on hotspot-dev; only GC code is involved, but some > >> non-GC folks have expressed interest.] > >> > >> All non-assert uses of klass_or_null are being replaced with the > >> acquire variant. In most cases, the size is being accessed soon > >> after, and the acquire is needed to ensure order of klass and > >> possible inline size access. In a few cases, the need for acquire > >> is less immediately obvious: > >> > >> - CompatibleFreeListSpace::block_is_obj - Size access is not > >> lexically apparent, but callers access the size or other data > >> ordered with the klass. > >> > >> - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: > >> These perform other operations that need to be properly ordered > >> wrto the klass access. > > > > Can you point me to the release-store that is being paired with these load-acquires please? > > It?s the release_set_klass recently added here: > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/fd16b627ebc5 > > FYI > I'm still looking, but I think this is true: > > I think a release-store also needs to be in the interpreter and jitted code, for processors that need it (e.g. aarch64 and ppc). For example in MacroAssembler::store_klass. > > Those processors have a storestore after object initialization to > ensure that Java threads don't see uninitialized fields, but don't > have a store-release on setting the class to prevent CMS from tripping > over a mis-ordered header initialization. The nominal store order may > also be wrong (e.g. "length" before "klass"). See > C1_MacroAssembler::initialize_header() > > If this sounds right, I can file a bug? Those are all young-gen allocations. The issues that lead to require_set_klass and klass_or_null_acquire pairings only arise with old-gen allocations. All mutator initiated old-gen allocations go through the post_allocation_setup_common path. (That's also the slow path for young-gen allocations, which don't need the require fence, but conditionalizing the fence didn't seem worthwhile.) There is a separate problem that fences are needed when a new object escapes to some other thread. It must be a well-formed object, including having its klass set, before escaping. So a fence is between setting the klass and escape. I think those exist, though Dean has been chasing a bug for a while that might be a missing one. The difference is that these concurrent collectors can form a reference to a blob of memory that is in the process of being turned into a new object, e.g. before its klass has been set. That's where klass_or_null comes into play. G1 almost doesn't do that; see "RFR: 8166607: G1 needs klass_or_null_acquire" and JDK-8166995. From david.holmes at oracle.com Wed Oct 5 23:43:21 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Oct 2016 09:43:21 +1000 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: <56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com> References: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com> <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com> <56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com> Message-ID: <508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com> On 6/10/2016 9:19 AM, Mandy Chung wrote: > >> On Oct 5, 2016, at 4:02 PM, David Holmes wrote: >> >> On 6/10/2016 8:55 AM, Mandy Chung wrote: >>> >>> >>> Note that PD is the protection domain of an initiating class that resolves a target type T. PD is kept in T?s class loader L. It?s not the protection domains of the classes defined by L. VM keeps its own cache of protection domains that have been validated with a resolved type. VM does not depend on ClassLoader::domains field to keep it alive. IIUC, it?s kept as a weak reference in VM which is what we want here. >> >> Okay but this will still affect the lifecycle of the PDs because without the strong reference in L, those weak references in the VM will quickly be cleared. I'm unclear what the implications of this might be, but I doubt regular short-lived tests would expose any issues. > > There is no issue there. > > PD will be GC?ed together when the classes along with its class loader are unloaded. These class loaders may be one or more and they are L1, L2 ?. != L. Okay, thanks for walking me through this. David > Mandy > From kim.barrett at oracle.com Thu Oct 6 00:25:57 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Oct 2016 20:25:57 -0400 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: References: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> Message-ID: <27B21E6B-0B6C-417D-96F5-10E301779176@oracle.com> > On Oct 5, 2016, at 7:36 PM, White, Derek wrote: > > Thanks for the reminder Kim! (Boy, you walk away from this code for a few days...) > > So we have the case where the slow-path "runtime" allocators always do a store_release on the klass (if CMS might run), but never do a StoreStore barrier after initializing the object fields/elements. The interpreter/compilers are responsible for that (typically done by generated code). Yes, except G1 also needs the release_store of the klass. > That leaves other runtime callers of the CollectedHeap allocators to also make sure that they have the correct barriers. Yes. > It's not yet clear that they do. It might help to separate allocators directly called by the interpreter/compiler. Here's a 2-deep fan-out: > InstanceMirrorKlass::allocate_instance > InstanceKlass::allocate_instance > ArrayKlass::allocate_arrayArray > InstanceKlass::allocate_objArray > - oopFactory::new_objArray > ObjArrayKlass::allocate > - Deoptimization::realloc_objects > - oopFactory ::new_objectArray > TypeArrayKlass::allocate_common > - oopFactory::new_metaDataArray > - oopFactory::new_typeArray_nozero (OK) > JVM_Clone (OK?). Maybe there are some missing, and maybe not. But that?s not relevant to *this* bug, which is about the setting of klass and access to it by concurrent collectors.. I?m trying to get the fix for *this* bug reviewed. From kim.barrett at oracle.com Thu Oct 6 00:27:46 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Oct 2016 20:27:46 -0400 Subject: RFR: 8167200: AArch64: Broken stack pointer adjustment in interpreter In-Reply-To: References: Message-ID: > On Oct 5, 2016, at 1:56 PM, Andrew Haley wrote: > > This is a thinko in the template interpreter. > > AArch64 has two stack pointers: the system SP and the expression > SP. The expression SP must always point to an address greater than or > equal to the system SP. > > When we allocate a new monitor in the interpreter we move the contents > of the entire operand stack and then insert a monitor between the > operand stack and the frame. So, we need to allocate two more words > (the size of a monitor) on the stack. The expression SP is adjusted to > allow for this, and we then do a comparison to see if the system SP > needs also to be moved. This is wrong: whenever we allocate a monitor > we should also unconditionally adjust the system SP. > > The bug here is that we might decide that we do not need any more > space in the system stack but later push several items onto the > expression stack. In this case the expression SP might be lower than > the system SP, and method arguments are corrupted. > > This patch changes the logic so that whenever we insert a monitor we > unconditionally adjust the system SP. > > http://cr.openjdk.java.net/~aph/8167200/ > > Andrew. Looks good with Dean?s suggested removal of the now unused no_adjust label. From david.holmes at oracle.com Thu Oct 6 01:02:17 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Oct 2016 11:02:17 +1000 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> References: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> Message-ID: On 5/10/2016 9:12 AM, Kim Barrett wrote: >> On Oct 4, 2016, at 5:04 AM, David Holmes wrote: >> >> Hi Kim, >> >> On 2/10/2016 7:29 AM, Kim Barrett wrote: >>> Please review this change to the CMS collector to replace uses of >>> klass_or_null with klass_or_null_acquire where needed. >>> >>> [Reviewing on hotspot-dev; only GC code is involved, but some non-GC >>> folks have expressed interest.] >>> >>> All non-assert uses of klass_or_null are being replaced with the >>> acquire variant. In most cases, the size is being accessed soon >>> after, and the acquire is needed to ensure order of klass and possible >>> inline size access. In a few cases, the need for acquire is less >>> immediately obvious: >>> >>> - CompatibleFreeListSpace::block_is_obj - Size access is not lexically >>> apparent, but callers access the size or other data ordered with the >>> klass. >>> >>> - MarkFromRootsClosure::do_bit and ParMarkFromRootsClosure::do_bit: >>> These perform other operations that need to be properly ordered wrto >>> the klass access. >> >> Can you point me to the release-store that is being paired with these load-acquires please? > > It?s the release_set_klass recently added here: > http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/fd16b627ebc5 Okay so this generally just sync's with object allocation. I don't have anywhere near enough knowledge of the GC code to know how these read paths may interact with the object allocation code, to determine if acquire semantics are needed. So as I said the change seem fine and are perfectly "correct" in a functional sense. Thanks, David >> Superficially this all seems fine and can't have any correctness issues, it is just a matter of where acquire semantics are actually needed. And it is hard to tell that without understanding exactly how the different read-paths may execute relative to the write-path. > > In MarkFromRootsClosure::do_bit, the first one is protecting the call to scanOopsInOop. The second is protecting the call to mark_range, which might not be strictly necessary, but seems like this should be pretty rare. And being consistent about the use of klass_or_null_acquire in non-assert contexts seems of value. Though I admit to finding this block of code confusing; it?s unconditional in a product build, but only executed if !_verifying in a debug build. Strange? but it?s been like that ?forever?. > > In ParMarkFromRootsClosure::do_bit, protecting call to scan_oops_in_oop. > > The scanners need to be protected against reading an uninitialize length or uninitialized fields / array elements. > From Derek.White at cavium.com Thu Oct 6 01:25:45 2016 From: Derek.White at cavium.com (White, Derek) Date: Thu, 6 Oct 2016 01:25:45 +0000 Subject: RFR: 8166862: CMS needs klass_or_null_acquire In-Reply-To: <27B21E6B-0B6C-417D-96F5-10E301779176@oracle.com> References: <81264090-044e-434f-6d8a-86e084380c41@oracle.com> <43822535-6A16-4788-858A-0F93BCF74516@oracle.com> , <27B21E6B-0B6C-417D-96F5-10E301779176@oracle.com> Message-ID: <4C89D18A-0D0E-4586-9579-7DDFCD70E1ED@cavium.com> Thanks for the info Kim. Yes, my comments are not relevant to this bug. - Derek On Oct 5, 2016, at 8:26 PM, Kim Barrett wrote: >> On Oct 5, 2016, at 7:36 PM, White, Derek wrote: >> >> Thanks for the reminder Kim! (Boy, you walk away from this code for a few days...) >> >> So we have the case where the slow-path "runtime" allocators always do a store_release on the klass (if CMS might run), but never do a StoreStore barrier after initializing the object fields/elements. The interpreter/compilers are responsible for that (typically done by generated code). > > Yes, except G1 also needs the release_store of the klass. > >> That leaves other runtime callers of the CollectedHeap allocators to also make sure that they have the correct barriers. > > Yes. > >> It's not yet clear that they do. It might help to separate allocators directly called by the interpreter/compiler. Here's a 2-deep fan-out: >> InstanceMirrorKlass::allocate_instance >> InstanceKlass::allocate_instance >> ArrayKlass::allocate_arrayArray >> InstanceKlass::allocate_objArray >> - oopFactory::new_objArray >> ObjArrayKlass::allocate >> - Deoptimization::realloc_objects >> - oopFactory ::new_objectArray >> TypeArrayKlass::allocate_common >> - oopFactory::new_metaDataArray >> - oopFactory::new_typeArray_nozero (OK) >> JVM_Clone (OK?). > > Maybe there are some missing, and maybe not. But that?s not relevant to *this* bug, which is about the setting of klass and access to it by concurrent collectors.. I?m trying to get the fix for *this* bug reviewed. > From david.holmes at oracle.com Thu Oct 6 01:32:57 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Oct 2016 11:32:57 +1000 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20161004094252.GL20056@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> Message-ID: <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> On 4/10/2016 7:42 PM, Erik Helin wrote: > On 2016-09-30, Leonid Mesnik wrote: >> Hi >> >> I think that it would be better to split this test into 4 tests. Currently >> this test is going to be skipped if any specific GCare set. Or it could even >> fail if any G1 specific options is set without setting G1 explicitly. >> >> So there is no way to run this with additional G1/CMS/ParGC specific options >> now. > > Ok, I've split the test into four "tests" (they just run GCBasher with > different GCs). Please see new patches: > > - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ That seems fine to me too. One minor issue - all the copyright notices in the new files are slightly wrong (and we have a big patch in the pipeline that is fixing all the existing tests that have errors). In the copyright notice you have: 1 /* 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved. 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ... 21 * questions. 22 */ but the correct format requires that all the asterisks line up: 1 /* 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved. 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ... 21 * questions. 22 */ Thanks, David > Thanks, > Erik > >> Leonid >> >> >> On 28.09.2016 18:01, Erik Helin wrote: >>> Hi all, >>> >>> this patch adds a new GC stress test called GCBasher. GCBasher builds up >>> large (well, for some definiton of large) object graphs by figuring out >>> the relations between classes in the JDK. The test usually stresses the >>> GC quite a lot, especially when run with a smaller heap. >>> >>> The changes in the top-level repository are for JPRT. JPRT will now run >>> the jtreg test GCBasher instead of the old version. >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8166790 >>> >>> Webrev: >>> - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ >>> - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ >>> >>> Testing: >>> - Running the new jtreg test locally on Linux x86-64: >>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java >>> >>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) >>> - JPRT >>> >>> Thanks, >>> Erik >> From stanislav.smirnov at oracle.com Thu Oct 6 12:17:27 2016 From: stanislav.smirnov at oracle.com (Stanislav Smirnov) Date: Thu, 6 Oct 2016 15:17:27 +0300 Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test In-Reply-To: References: <5346A7A7-672C-4DF8-83CF-FB108355C171@oracle.com> Message-ID: Hi David, thanks, I am ok to push it into hs. I will prepare and send you a prepared changeset. Best regards, Stanislav Smirnov > On 06 Oct 2016, at 00:45, David Holmes wrote: > > Hi Stanislav, > > On 6/10/2016 2:41 AM, Stanislav Smirnov wrote: >> Hi Iris, >> >> thanks for a review. >> Btw, I will also need a sponsor to push it. > > Reviewed! And I will sponsor for you. I will push this to jdk9/hs to avoid future potential conflicts with test changes if that is okay? Or do you want it to go to jdk9/dev? > > Thanks, > David > >> Best regards, >> Stanislav Smirnov >> >> >> >> >> >>> On 05 Oct 2016, at 19:32, Iris Clark wrote: >>> >>> Hi. >>> >>>> Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ >>> >>> These appear to be the expected changes. >>> >>> I find the updates to test/gc/TestHumongousReferenceObject.java interesting. You needed to add the word "Public" to the phrase >>> "General Public License". It looks like somewhere during development, there was an overly enthusiastic search and replace of "Public" with "". Regardless, it looks fine now. >>> >>> Thanks, >>> Iris >>> (not a Reviewer) >>> >>> -----Original Message----- >>> From: Stanislav Smirnov >>> Sent: Wednesday, October 05, 2016 7:56 AM >>> To: hotspot-dev at openjdk.java.net >>> Subject: RFR: 8165687: Fix license and copyright headers in jd9 under hotspot/test >>> >>> Hi, >>> >>> Please review this fix for JDK-8165687 . >>> Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files for uniformity to meet Oracle requirements. >>> >>> JBS bug: https://bugs.openjdk.java.net/browse/JDK-8165687 >>> Webrev: http://cr.openjdk.java.net/~stsmirno/8165687/webrev.00/ >>> >>> Best regards, >>> Stanislav Smirnov >> From stanislav.smirnov at oracle.com Thu Oct 6 12:19:47 2016 From: stanislav.smirnov at oracle.com (Stanislav Smirnov) Date: Thu, 6 Oct 2016 15:19:47 +0300 Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under test/lib In-Reply-To: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com> References: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com> Message-ID: <4F4B0746-C6D5-400A-95BA-EFDD211C5DD5@oracle.com> adding core-libs-dev mailing list, since my change affect some files for testing APIs in java.util.function Best regards, Stanislav Smirnov > On 05 Oct 2016, at 19:44, Stanislav Smirnov wrote: > > Hi, > > Please review this fix for JDK-8163984 . > This one is similar to another one, I have sent earlier. > Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files under test/lib for uniformity to meet Oracle requirements. > > There are two files from test/lib/jdk do I need to include jdk9_dev mailing list? > > JBS bug: https://bugs.openjdk.java.net/browse/JDK-8163984 > Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ > Best regards, > Stanislav Smirnov > > > > > From stanislav.smirnov at oracle.com Thu Oct 6 12:20:28 2016 From: stanislav.smirnov at oracle.com (Stanislav Smirnov) Date: Thu, 6 Oct 2016 15:20:28 +0300 Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under test/lib In-Reply-To: <57e4e124-df94-4b44-a127-c2c75cff2555@default> References: <6EE02652-2A31-469D-9595-FCBA1DAD0F99@oracle.com> <57e4e124-df94-4b44-a127-c2c75cff2555@default> Message-ID: <28B85C57-BA48-4382-A23A-A8EB721F489F@oracle.com> Thank you Iris Best regards, Stanislav Smirnov > On 05 Oct 2016, at 19:56, Iris Clark wrote: > > Hi. > >> Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ > > These changes look fine to me. > >> There are two files from test/lib/jdk do I need to include jdk9_dev mailing list? > > These files appear to be testing APIs in java.util.function which was introduced in JDK 8 for Lambda. I recommend you use core-libs-dev at ojn instead of the more general jdk9-dev at ojn. > > Thanks, > Iris > (not a Reviewer) > > -----Original Message----- > From: Stanislav Smirnov > Sent: Wednesday, October 05, 2016 9:44 AM > To: hotspot-dev at openjdk.java.net > Subject: RFR: 8163984: Fix license and copyright headers in jdk9 under test/lib > > Hi, > > Please review this fix for JDK-8163984 . > This one is similar to another one, I have sent earlier. > Legal notices and Oracle copyrights were updated (white and blank space, commas) in tests files under test/lib for uniformity to meet Oracle requirements. > > There are two files from test/lib/jdk do I need to include jdk9_dev mailing list? > > JBS bug: https://bugs.openjdk.java.net/browse/JDK-8163984 > Webrev: http://cr.openjdk.java.net/~stsmirno/8163984/webrev.00/ > Best regards, > Stanislav Smirnov > > > > > From erik.helin at oracle.com Thu Oct 6 12:57:39 2016 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 6 Oct 2016 14:57:39 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> Message-ID: <20161006125739.GA6035@ehelin.jrpg.bea.com> On 2016-10-06, David Holmes wrote: > On 4/10/2016 7:42 PM, Erik Helin wrote: > >On 2016-09-30, Leonid Mesnik wrote: > >>Hi > >> > >>I think that it would be better to split this test into 4 tests. Currently > >>this test is going to be skipped if any specific GCare set. Or it could even > >>fail if any G1 specific options is set without setting G1 explicitly. > >> > >>So there is no way to run this with additional G1/CMS/ParGC specific options > >>now. > > > >Ok, I've split the test into four "tests" (they just run GCBasher with > >different GCs). Please see new patches: > > > >- incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ > >- full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ > > That seems fine to me too. > > One minor issue - all the copyright notices in the new files are slightly > wrong (and we have a big patch in the pipeline that is fixing all the > existing tests that have errors). In the copyright notice you have: > > 1 /* > 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights > reserved. > 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ... > 21 * questions. > 22 */ > > but the correct format requires that all the asterisks line up: > > 1 /* > 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights > reserved. > 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ... > 21 * questions. > 22 */ Ok, no problem. You can find updated patches below (I also added an @requires for -server): - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01-02/ - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02/ Thanks, Erik > Thanks, > David > > >Thanks, > >Erik > > > >>Leonid > >> > >> > >>On 28.09.2016 18:01, Erik Helin wrote: > >>>Hi all, > >>> > >>>this patch adds a new GC stress test called GCBasher. GCBasher builds up > >>>large (well, for some definiton of large) object graphs by figuring out > >>>the relations between classes in the JDK. The test usually stresses the > >>>GC quite a lot, especially when run with a smaller heap. > >>> > >>>The changes in the top-level repository are for JPRT. JPRT will now run > >>>the jtreg test GCBasher instead of the old version. > >>> > >>>Enhancement: > >>>https://bugs.openjdk.java.net/browse/JDK-8166790 > >>> > >>>Webrev: > >>>- hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > >>>- top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > >>> > >>>Testing: > >>>- Running the new jtreg test locally on Linux x86-64: > >>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > >>> > >>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > >>>- JPRT > >>> > >>>Thanks, > >>>Erik > >> From gromero at linux.vnet.ibm.com Thu Oct 6 13:02:07 2016 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 6 Oct 2016 10:02:07 -0300 Subject: ADL/ADLC documentation Message-ID: <57F64B4F.3060908@linux.vnet.ibm.com> Hi, Is there a more comprehensive documentation on the Architecture Description Language Compiler then the one found in the hotspot tree? http://hg.openjdk.java.net/jdk9/hs/hotspot/file/1d70c7ca92cc/src/share/vm/adlc/Doc/Syntax.doc Regards, Gustavo From jesper.wilhelmsson at oracle.com Thu Oct 6 13:24:01 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 6 Oct 2016 15:24:01 +0200 Subject: jdk9/hs is CLOSED Message-ID: Hi, Due to issues with our nightly testing we have decided to close jdk9/hs for new changes. Please do not push anything to hs until further notice. Thanks, /Jesper From dmitry.fazunenko at oracle.com Thu Oct 6 13:36:09 2016 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 6 Oct 2016 16:36:09 +0300 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20161006125739.GA6035@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> <20161006125739.GA6035@ehelin.jrpg.bea.com> Message-ID: <78be1d1a-8672-202b-5639-6f06f46e8324@oracle.com> Hi Erik, 1) Would you update: @requires vm.gc.X | vm.gc=="null" --> @requires vm.gc.X vm.gc.X means that X GC could be set, i.e. X is supported by VM and no other GC is set via command line. 2) @requires for -server is this test server specific? If not, will it make sense to remove "-server" from @run instead? Thanks, Dima On 06.10.2016 15:57, Erik Helin wrote: > On 2016-10-06, David Holmes wrote: >> On 4/10/2016 7:42 PM, Erik Helin wrote: >>> On 2016-09-30, Leonid Mesnik wrote: >>>> Hi >>>> >>>> I think that it would be better to split this test into 4 tests. Currently >>>> this test is going to be skipped if any specific GCare set. Or it could even >>>> fail if any G1 specific options is set without setting G1 explicitly. >>>> >>>> So there is no way to run this with additional G1/CMS/ParGC specific options >>>> now. >>> Ok, I've split the test into four "tests" (they just run GCBasher with >>> different GCs). Please see new patches: >>> >>> - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ >>> - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ >> That seems fine to me too. >> >> One minor issue - all the copyright notices in the new files are slightly >> wrong (and we have a big patch in the pipeline that is fixing all the >> existing tests that have errors). In the copyright notice you have: >> >> 1 /* >> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights >> reserved. >> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> ... >> 21 * questions. >> 22 */ >> >> but the correct format requires that all the asterisks line up: >> >> 1 /* >> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights >> reserved. >> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> ... >> 21 * questions. >> 22 */ > Ok, no problem. You can find updated patches below (I also added an > @requires for -server): > - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01-02/ > - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02/ > > Thanks, > Erik > >> Thanks, >> David >> >>> Thanks, >>> Erik >>> >>>> Leonid >>>> >>>> >>>> On 28.09.2016 18:01, Erik Helin wrote: >>>>> Hi all, >>>>> >>>>> this patch adds a new GC stress test called GCBasher. GCBasher builds up >>>>> large (well, for some definiton of large) object graphs by figuring out >>>>> the relations between classes in the JDK. The test usually stresses the >>>>> GC quite a lot, especially when run with a smaller heap. >>>>> >>>>> The changes in the top-level repository are for JPRT. JPRT will now run >>>>> the jtreg test GCBasher instead of the old version. >>>>> >>>>> Enhancement: >>>>> https://bugs.openjdk.java.net/browse/JDK-8166790 >>>>> >>>>> Webrev: >>>>> - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ >>>>> - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ >>>>> >>>>> Testing: >>>>> - Running the new jtreg test locally on Linux x86-64: >>>>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java >>>>> >>>>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) >>>>> - JPRT >>>>> >>>>> Thanks, >>>>> Erik From david.holmes at oracle.com Thu Oct 6 13:41:43 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Oct 2016 23:41:43 +1000 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20161006125739.GA6035@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> <20161006125739.GA6035@ehelin.jrpg.bea.com> Message-ID: <070b407f-b888-4fe2-ba3a-3c40ef0958e3@oracle.com> On 6/10/2016 10:57 PM, Erik Helin wrote: > On 2016-10-06, David Holmes wrote: >> On 4/10/2016 7:42 PM, Erik Helin wrote: >>> On 2016-09-30, Leonid Mesnik wrote: >>>> Hi >>>> >>>> I think that it would be better to split this test into 4 tests. Currently >>>> this test is going to be skipped if any specific GCare set. Or it could even >>>> fail if any G1 specific options is set without setting G1 explicitly. >>>> >>>> So there is no way to run this with additional G1/CMS/ParGC specific options >>>> now. >>> >>> Ok, I've split the test into four "tests" (they just run GCBasher with >>> different GCs). Please see new patches: >>> >>> - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ >>> - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ >> >> That seems fine to me too. >> >> One minor issue - all the copyright notices in the new files are slightly >> wrong (and we have a big patch in the pipeline that is fixing all the >> existing tests that have errors). In the copyright notice you have: >> >> 1 /* >> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights >> reserved. >> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> ... >> 21 * questions. >> 22 */ >> >> but the correct format requires that all the asterisks line up: >> >> 1 /* >> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights >> reserved. >> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> ... >> 21 * questions. >> 22 */ > > Ok, no problem. You can find updated patches below (I also added an > @requires for -server): > - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01-02/ > - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02/ Sorry I wasn't clear - it is all of the new files, not just the "split 4". Thanks, David > Thanks, > Erik > >> Thanks, >> David >> >>> Thanks, >>> Erik >>> >>>> Leonid >>>> >>>> >>>> On 28.09.2016 18:01, Erik Helin wrote: >>>>> Hi all, >>>>> >>>>> this patch adds a new GC stress test called GCBasher. GCBasher builds up >>>>> large (well, for some definiton of large) object graphs by figuring out >>>>> the relations between classes in the JDK. The test usually stresses the >>>>> GC quite a lot, especially when run with a smaller heap. >>>>> >>>>> The changes in the top-level repository are for JPRT. JPRT will now run >>>>> the jtreg test GCBasher instead of the old version. >>>>> >>>>> Enhancement: >>>>> https://bugs.openjdk.java.net/browse/JDK-8166790 >>>>> >>>>> Webrev: >>>>> - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ >>>>> - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ >>>>> >>>>> Testing: >>>>> - Running the new jtreg test locally on Linux x86-64: >>>>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java >>>>> >>>>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) >>>>> - JPRT >>>>> >>>>> Thanks, >>>>> Erik >>>> From david.holmes at oracle.com Thu Oct 6 13:44:03 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Oct 2016 23:44:03 +1000 Subject: ADL/ADLC documentation In-Reply-To: <57F64B4F.3060908@linux.vnet.ibm.com> References: <57F64B4F.3060908@linux.vnet.ibm.com> Message-ID: <339393de-c60c-9dce-a3ee-48a25999d62c@oracle.com> On 6/10/2016 11:02 PM, Gustavo Romero wrote: > Hi, > > Is there a more comprehensive documentation on the Architecture Description > Language Compiler then the one found in the hotspot tree? > > http://hg.openjdk.java.net/jdk9/hs/hotspot/file/1d70c7ca92cc/src/share/vm/adlc/Doc/Syntax.doc You may find additional info on the wiki: https://wiki.openjdk.java.net/display/HotSpot/Main David > > Regards, > Gustavo > From erik.helin at oracle.com Thu Oct 6 14:38:11 2016 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 6 Oct 2016 16:38:11 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <78be1d1a-8672-202b-5639-6f06f46e8324@oracle.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> <20161006125739.GA6035@ehelin.jrpg.bea.com> <78be1d1a-8672-202b-5639-6f06f46e8324@oracle.com> Message-ID: <20161006143811.GB6035@ehelin.jrpg.bea.com> David, Dmitry, below you will find new patches: - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02-03/ - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/03/ I've fixed the indentation of the comments of the GPL header in all files. I also removed '| vm.gc=="null"' from all @requires lines. I also filed an enhancement to investigate if we can remove -server option [0]. Thanks, Erik [0]: https://bugs.openjdk.java.net/browse/JDK-8167290 On 2016-10-06, Dmitry Fazunenko wrote: > Hi Erik, > > 1) Would you update: > > @requires vm.gc.X | vm.gc=="null" > --> > @requires vm.gc.X > > vm.gc.X means that X GC could be set, i.e. X is supported by VM and no other GC is set via command line. > > > 2) @requires for -server > is this test server specific? > If not, will it make sense to remove "-server" from @run instead? > > Thanks, > Dima > > > > > On 06.10.2016 15:57, Erik Helin wrote: > >On 2016-10-06, David Holmes wrote: > >>On 4/10/2016 7:42 PM, Erik Helin wrote: > >>>On 2016-09-30, Leonid Mesnik wrote: > >>>>Hi > >>>> > >>>>I think that it would be better to split this test into 4 tests. Currently > >>>>this test is going to be skipped if any specific GCare set. Or it could even > >>>>fail if any G1 specific options is set without setting G1 explicitly. > >>>> > >>>>So there is no way to run this with additional G1/CMS/ParGC specific options > >>>>now. > >>>Ok, I've split the test into four "tests" (they just run GCBasher with > >>>different GCs). Please see new patches: > >>> > >>>- incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ > >>>- full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ > >>That seems fine to me too. > >> > >>One minor issue - all the copyright notices in the new files are slightly > >>wrong (and we have a big patch in the pipeline that is fixing all the > >>existing tests that have errors). In the copyright notice you have: > >> > >> 1 /* > >> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights > >>reserved. > >> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > >> ... > >> 21 * questions. > >> 22 */ > >> > >>but the correct format requires that all the asterisks line up: > >> > >> 1 /* > >> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights > >>reserved. > >> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > >> ... > >> 21 * questions. > >> 22 */ > >Ok, no problem. You can find updated patches below (I also added an > >@requires for -server): > >- incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01-02/ > >- full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02/ > > > >Thanks, > >Erik > > > >>Thanks, > >>David > >> > >>>Thanks, > >>>Erik > >>> > >>>>Leonid > >>>> > >>>> > >>>>On 28.09.2016 18:01, Erik Helin wrote: > >>>>>Hi all, > >>>>> > >>>>>this patch adds a new GC stress test called GCBasher. GCBasher builds up > >>>>>large (well, for some definiton of large) object graphs by figuring out > >>>>>the relations between classes in the JDK. The test usually stresses the > >>>>>GC quite a lot, especially when run with a smaller heap. > >>>>> > >>>>>The changes in the top-level repository are for JPRT. JPRT will now run > >>>>>the jtreg test GCBasher instead of the old version. > >>>>> > >>>>>Enhancement: > >>>>>https://bugs.openjdk.java.net/browse/JDK-8166790 > >>>>> > >>>>>Webrev: > >>>>>- hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > >>>>>- top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > >>>>> > >>>>>Testing: > >>>>>- Running the new jtreg test locally on Linux x86-64: > >>>>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > >>>>> > >>>>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > >>>>>- JPRT > >>>>> > >>>>>Thanks, > >>>>>Erik > From dmitry.fazunenko at oracle.com Thu Oct 6 15:00:32 2016 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 6 Oct 2016 18:00:32 +0300 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20161006143811.GB6035@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> <20161006125739.GA6035@ehelin.jrpg.bea.com> <78be1d1a-8672-202b-5639-6f06f46e8324@oracle.com> <20161006143811.GB6035@ehelin.jrpg.bea.com> Message-ID: Erik, Looks to me. Thanks for addressing my notes. -- Dima On 06.10.2016 17:38, Erik Helin wrote: > David, Dmitry, > > below you will find new patches: > - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02-03/ > - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/03/ > > I've fixed the indentation of the comments of the GPL header in all > files. I also removed '| vm.gc=="null"' from all @requires lines. I also > filed an enhancement to investigate if we can remove -server option [0]. > > Thanks, > Erik > > [0]: https://bugs.openjdk.java.net/browse/JDK-8167290 > > On 2016-10-06, Dmitry Fazunenko wrote: >> Hi Erik, >> >> 1) Would you update: >> >> @requires vm.gc.X | vm.gc=="null" >> --> >> @requires vm.gc.X >> >> vm.gc.X means that X GC could be set, i.e. X is supported by VM and no other GC is set via command line. >> >> >> 2) @requires for -server >> is this test server specific? >> If not, will it make sense to remove "-server" from @run instead? >> >> Thanks, >> Dima >> >> >> >> >> On 06.10.2016 15:57, Erik Helin wrote: >>> On 2016-10-06, David Holmes wrote: >>>> On 4/10/2016 7:42 PM, Erik Helin wrote: >>>>> On 2016-09-30, Leonid Mesnik wrote: >>>>>> Hi >>>>>> >>>>>> I think that it would be better to split this test into 4 tests. Currently >>>>>> this test is going to be skipped if any specific GCare set. Or it could even >>>>>> fail if any G1 specific options is set without setting G1 explicitly. >>>>>> >>>>>> So there is no way to run this with additional G1/CMS/ParGC specific options >>>>>> now. >>>>> Ok, I've split the test into four "tests" (they just run GCBasher with >>>>> different GCs). Please see new patches: >>>>> >>>>> - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ >>>>> - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ >>>> That seems fine to me too. >>>> >>>> One minor issue - all the copyright notices in the new files are slightly >>>> wrong (and we have a big patch in the pipeline that is fixing all the >>>> existing tests that have errors). In the copyright notice you have: >>>> >>>> 1 /* >>>> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights >>>> reserved. >>>> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>> ... >>>> 21 * questions. >>>> 22 */ >>>> >>>> but the correct format requires that all the asterisks line up: >>>> >>>> 1 /* >>>> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights >>>> reserved. >>>> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>> ... >>>> 21 * questions. >>>> 22 */ >>> Ok, no problem. You can find updated patches below (I also added an >>> @requires for -server): >>> - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01-02/ >>> - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02/ >>> >>> Thanks, >>> Erik >>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> Erik >>>>> >>>>>> Leonid >>>>>> >>>>>> >>>>>> On 28.09.2016 18:01, Erik Helin wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> this patch adds a new GC stress test called GCBasher. GCBasher builds up >>>>>>> large (well, for some definiton of large) object graphs by figuring out >>>>>>> the relations between classes in the JDK. The test usually stresses the >>>>>>> GC quite a lot, especially when run with a smaller heap. >>>>>>> >>>>>>> The changes in the top-level repository are for JPRT. JPRT will now run >>>>>>> the jtreg test GCBasher instead of the old version. >>>>>>> >>>>>>> Enhancement: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166790 >>>>>>> >>>>>>> Webrev: >>>>>>> - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ >>>>>>> - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ >>>>>>> >>>>>>> Testing: >>>>>>> - Running the new jtreg test locally on Linux x86-64: >>>>>>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java >>>>>>> >>>>>>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) >>>>>>> - JPRT >>>>>>> >>>>>>> Thanks, >>>>>>> Erik From gromero at linux.vnet.ibm.com Thu Oct 6 15:23:46 2016 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 6 Oct 2016 12:23:46 -0300 Subject: ADL/ADLC documentation In-Reply-To: <339393de-c60c-9dce-a3ee-48a25999d62c@oracle.com> References: <57F64B4F.3060908@linux.vnet.ibm.com> <339393de-c60c-9dce-a3ee-48a25999d62c@oracle.com> Message-ID: <57F66C82.1050004@linux.vnet.ibm.com> On 06-10-2016 10:44, David Holmes wrote: > On 6/10/2016 11:02 PM, Gustavo Romero wrote: >> Hi, >> >> Is there a more comprehensive documentation on the Architecture Description >> Language Compiler then the one found in the hotspot tree? >> >> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/1d70c7ca92cc/src/share/vm/adlc/Doc/Syntax.doc > > You may find additional info on the wiki: > > https://wiki.openjdk.java.net/display/HotSpot/Main Thanks David. Regards, Gustavo From coleen.phillimore at oracle.com Thu Oct 6 17:54:21 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 6 Oct 2016 13:54:21 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> Message-ID: Here is an update to the Metadata leak change. There was a bug introduced when cleaning this up, which Mikael also found. open webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev Changes include adding logging for report_metadata_oome, which necessitated removing ResourceMarks in ClassLoaderData::dump because the stream passed in already required a ResourceMark, so it got a nested ResourceMark message for the stream. I changed logging for tracing block allocations to log_trace(gc, metaspace, freelist, blocks). In BlockFreelist::get_block and BlockFreelist::return_block() assert that it's not called for a size smaller than the minimum allocation (which was the bug). Renamed get_raw_word_size() to get_allocation_word_size(). This rounds up to the minimum allocation size which is the same as small_block_min_size. Also, I added a test that takes a long time to execute to verify this, and excluded it from JPRT. I could skip adding this test if people don't want it. Also, the test verifies that continuously redefining the same class gets memory for the new class that was released because the block sizes are the same. When the test exits, it gets a metaspace OOM because loading new classes and allocating metadata can't use the blocks returned (wrong size). There is still fragmentation in this implementation, but it's better that things deallocated < 12 words are actually freed. I'll file an RFE to work on a perfect algorithm, or to investigate finding a better one, although I consider this a stress test that uses all of metaspace to MaxMetaspaceSize, leaving allocation only to the block fragments left. This isn't a typical use case. Some comments and corrections to my responses to Mikael below: On 10/4/16 12:15 PM, Coleen Phillimore wrote: > > Hi Mikael, > > Thanks for looking at this change. > > On 10/4/16 8:32 AM, Mikael Gerdin wrote: >> Hi Coleen, >> >> On 2016-09-30 21:02, Coleen Phillimore wrote: >>> Summary: Return Metablocks smaller than dictionary's dark matter. >>> >>> This change contributed by Jon Masamitsu and myself. To reclaim "dark >>> matter" this change adds an array of small blocks by size, created >>> lazily, to return Metablocks smaller than the BinaryTreeDictionary >>> entry's minimum size. This change also fixed a bug in small object >>> double free and adds debugging code to check for this case. >>> >>> With this change, the submitted test case runs indefinitely. Also >>> passed rbt tier 1-5 testing. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >> >> I'd prefer it if SmallBlocks didn't expose its implementation by >> returning its FreeLists by reference, could you change it to have >> * return_chunk() >> * get_chunk() >> * num_chunks(word_size) >> >> and get rid of list_at? > > Okay, I refactored this so small_chunks()->get_block() and > return_block() are used rather than getting list_at. I didn't see > where you got num_chunks, but list_at is hidden. >> >> -- >> >> For how long do you plan to keep BlockFreelist::_all_blocks? >> I see that it's debug only but I fear that it could case problems >> with running out of native memory in our internal testing of debug >> builds. > > I thought about taking it out, but it helped me find the double free > bug. I think if we add new code to metadata and have to call > deallocate_contents on it, we risk re-introducting these double free > bugs. I could take it out. I don't think this gets that big but I'd > hate to introduce some sort of OOM bug in our testing. >> >> BlockFreelist::min_size() is a bit of a misnomer since it returns the >> minimum size of blocks to be put on the BlockTreeDictionary, not the >> minimum size of blocks which are reusable. > > How about min_dictionary_size() ? > >> >> Is there any particular reason behind _small_blocks being lazy >> allocated and _dictionary not? > > We lazily create the BlockFreelists with this change. > > // Lazily create a block_freelist > if (block_freelists() == NULL) { > _block_freelists = new BlockFreelist(); > } > > So the small_blocks are lazily created in the BlockFreelists but the > dictionary is not. I guess if we're going to create the > BlockFreelists here, we'll most likely need both and maybe > small_blocks need not be lazily created. Was that your suggestion? > > My concern with this change was all the space used by the > small_blocks() but if we're doing any deallocation within the > metaspace, at least one of the things will be <12 words. > > I'll make small_blocks() not be lazily allocated since BlockFreelist > are. These are pretty expensive, but should be limited to a few > metaspaces. Class SpaceManager doesn't need small_blocks() so I left small_blocks as lazily allocated. >> >> I would prefer if BlockFreelist::return_block would perform the >> checks in reverse order instead of having a return inside the first >> if block, something like >> >> if (word_size > small_block_max_size) { >> dict()->return_chunk(c) >> } else if (word_size > small_block_min_size) { >> small_blocks()->return_chunk(c) >> } else { >> // dark matter >> } > > Why? We don't want to cast Metablock into dark matter so check if > word_size < small_block_min_size first. > > Metablock* free_chunk = ::new (p) Metablock(word_size); > if (word_size < SmallBlocks::small_block_max_size()) { > small_blocks()->return_chunk(word_size); > } else { > dictionary()->return_chunk(free_chunk); > } > There is no dark matter in these functions anymore. >> >> >> For BlockFreelist::get_block I realize that the code is a bit more >> complex but this also raises a few questions. >> * When we allocate from the dictionary we search for a block which is >> at least as large as we ask for and split it, returning whatever was >> left back to the free list. We don't search for "large enough" blocks >> from the small blocks manager, is that intentional or just to keep >> the code simple (I like simple)? > > I'm glad you asked about this so I could give background. It turns > out that we deallocate metaspace items better this way. I had a > version that did exactly what you said. It was a simple sorted linked > list of returned blocks < min_dictionary_size (12) where get_block > returned the first block where the item would fit. It had some best > fit algorithm so if the block returned was a lot bigger, it wouldn't > pick it. > > My implementation could get through 69300 retransformations before the > list didn't work anymore (too many small block fragments of the wrong > size) and metaspace was exhausted (metaspace was limited to 12M in > this test). Jon's implementation ran this test indefinitely. So it's > somewhat simple but it worked really well. > >> >> In the last part of get_block where we return the unused part of a >> block retrieved from the dictionary uses compares with >> BlockFreelist::min_size() which, as I mentioned above, is the min >> size for blocks in the dictionary, not the min size for blocks to be >> reusable. >> I think this code can just call return_block unconditionally for any >> nonzero value of "unused" and let return_block deal with dark matter. >> > yes, I'll make that change. This change has to be conditional because I assert that BlockFreelist::return_block() is never called for < small_block_min_size. Thanks, Coleen >> >> >> Are the changes to Method and ConstMethod the "bug fix to small >> object double free"? > > Yes. >> Is the problem that when a method was freed its annotations were >> deallocated as well? Could the annotations fields in the "old" >> ConstMethod be cleared instead so that the old annotation arrays >> could be kept or is that just needless complexity? > > I'd rather copy the annotations, because I don't know how the old > method, which could still be running, might use the annotations. I > don't want to mess with that but I see your point. > > Coleen > >> >> Thanks >> /Mikael >> >> >> >>> >>> Thanks, >>> Coleen and Jon > From patrick at reini.net Thu Oct 6 19:08:11 2016 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 6 Oct 2016 21:08:11 +0200 Subject: Command line help for java states wrong arguments Message-ID: <7C310B77-EA7B-498C-B295-F6F8F57B53D9@reini.net> Hi there, I recently started testing the JDK 9 early access builds. And I found that there are some inconsistencies among the command line arguments. Such as ?-mp? for the module path, but actually it should be ?-p? as also the long version is ??module-path? instead of "-modulepath" as documented? Should this be addressed in the issue JDK-8065825? -Patrick From Alan.Bateman at oracle.com Thu Oct 6 19:19:10 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 6 Oct 2016 20:19:10 +0100 Subject: Command line help for java states wrong arguments In-Reply-To: <7C310B77-EA7B-498C-B295-F6F8F57B53D9@reini.net> References: <7C310B77-EA7B-498C-B295-F6F8F57B53D9@reini.net> Message-ID: <2d2bbe26-b573-f3cd-063a-179a50b14911@oracle.com> On 06/10/2016 20:08, Patrick Reinhart wrote: > Hi there, > > I recently started testing the JDK 9 early access builds. And I found that there are some inconsistencies among the command line arguments. Such as ?-mp? for the module path, but actually it should be ?-p? as also the long version is ??module-path? instead of "-modulepath" as documented? > > Should this be addressed in the issue JDK-8065825? > > -Patrick Which language/locale is this? Just wondering if this is just that the translations haven't caught up. -Alan. From patrick at reini.net Thu Oct 6 19:21:10 2016 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 6 Oct 2016 21:21:10 +0200 Subject: Command line help for java states wrong arguments In-Reply-To: <2d2bbe26-b573-f3cd-063a-179a50b14911@oracle.com> References: <7C310B77-EA7B-498C-B295-F6F8F57B53D9@reini.net> <2d2bbe26-b573-f3cd-063a-179a50b14911@oracle.com> Message-ID: Hi Alan, java version "9-ea" Java(TM) SE Runtime Environment (build 9-ea+138) Java HotSpot(TM) 64-Bit Server VM (build 9-ea+138, mixed mode) I work with the german version in Mac OSX. -Patrick > Am 06.10.2016 um 21:19 schrieb Alan Bateman : > > On 06/10/2016 20:08, Patrick Reinhart wrote: > >> Hi there, >> >> I recently started testing the JDK 9 early access builds. And I found that there are some inconsistencies among the command line arguments. Such as ?-mp? for the module path, but actually it should be ?-p? as also the long version is ??module-path? instead of "-modulepath" as documented? >> >> Should this be addressed in the issue JDK-8065825? >> >> -Patrick > Which language/locale is this? Just wondering if this is just that the translations haven't caught up. > > -Alan. From Alan.Bateman at oracle.com Thu Oct 6 19:28:59 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 6 Oct 2016 20:28:59 +0100 Subject: Command line help for java states wrong arguments In-Reply-To: References: <7C310B77-EA7B-498C-B295-F6F8F57B53D9@reini.net> <2d2bbe26-b573-f3cd-063a-179a50b14911@oracle.com> Message-ID: On 06/10/2016 20:21, Patrick Reinhart wrote: > Hi Alan, > > java version "9-ea" > Java(TM) SE Runtime Environment (build 9-ea+138) > Java HotSpot(TM) 64-Bit Server VM (build 9-ea+138, mixed mode) > > > I work with the german version in Mac OSX. > Thanks for confirming. I think you'll find the usage messages are correct when using English. The German and other translations will catch up, I just don't know when the next drop it. -Alan From patrick at reini.net Thu Oct 6 19:31:38 2016 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 6 Oct 2016 21:31:38 +0200 Subject: Command line help for java states wrong arguments In-Reply-To: References: <7C310B77-EA7B-498C-B295-F6F8F57B53D9@reini.net> <2d2bbe26-b573-f3cd-063a-179a50b14911@oracle.com> Message-ID: <7E4A3FCE-FFC7-4E6E-83FF-A9F6E583E219@reini.net> Thanks for the quick answer. Just out of curiosity: Where are those messages in the source tree anyway? -Patrick > Am 06.10.2016 um 21:28 schrieb Alan Bateman : > > On 06/10/2016 20:21, Patrick Reinhart wrote: >> Hi Alan, >> >> java version "9-ea" >> Java(TM) SE Runtime Environment (build 9-ea+138) >> Java HotSpot(TM) 64-Bit Server VM (build 9-ea+138, mixed mode) >> >> >> I work with the german version in Mac OSX. >> > Thanks for confirming. I think you'll find the usage messages are correct when using English. The German and other translations will catch up, I just don't know when the next drop it. > > -Alan From Alan.Bateman at oracle.com Thu Oct 6 19:36:59 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 6 Oct 2016 20:36:59 +0100 Subject: Command line help for java states wrong arguments In-Reply-To: <7E4A3FCE-FFC7-4E6E-83FF-A9F6E583E219@reini.net> References: <7C310B77-EA7B-498C-B295-F6F8F57B53D9@reini.net> <2d2bbe26-b573-f3cd-063a-179a50b14911@oracle.com> <7E4A3FCE-FFC7-4E6E-83FF-A9F6E583E219@reini.net> Message-ID: <92fd1fda-3ac4-b077-689f-19c77e7aa286@oracle.com> On 06/10/2016 20:31, Patrick Reinhart wrote: > Thanks for the quick answer. > > Just out of curiosity: > > Where are those messages in the source tree anyway? jdk/src/java.base/share/classes/sun/launcher/resources We don't edit the translations when updating the resources, instead there are "translation drops" every so often that update all the resources together (the supported languages, all areas). -Alan From david.holmes at oracle.com Thu Oct 6 22:58:20 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Oct 2016 08:58:20 +1000 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20161006143811.GB6035@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> <20161006125739.GA6035@ehelin.jrpg.bea.com> <78be1d1a-8672-202b-5639-6f06f46e8324@oracle.com> <20161006143811.GB6035@ehelin.jrpg.bea.com> Message-ID: <6bc9856c-1c84-9112-38c6-937992df566e@oracle.com> Updates look fine to me. Thanks, David On 7/10/2016 12:38 AM, Erik Helin wrote: > David, Dmitry, > > below you will find new patches: > - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02-03/ > - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/03/ > > I've fixed the indentation of the comments of the GPL header in all > files. I also removed '| vm.gc=="null"' from all @requires lines. I also > filed an enhancement to investigate if we can remove -server option [0]. > > Thanks, > Erik > > [0]: https://bugs.openjdk.java.net/browse/JDK-8167290 > > On 2016-10-06, Dmitry Fazunenko wrote: >> Hi Erik, >> >> 1) Would you update: >> >> @requires vm.gc.X | vm.gc=="null" >> --> >> @requires vm.gc.X >> >> vm.gc.X means that X GC could be set, i.e. X is supported by VM and no other GC is set via command line. >> >> >> 2) @requires for -server >> is this test server specific? >> If not, will it make sense to remove "-server" from @run instead? >> >> Thanks, >> Dima >> >> >> >> >> On 06.10.2016 15:57, Erik Helin wrote: >>> On 2016-10-06, David Holmes wrote: >>>> On 4/10/2016 7:42 PM, Erik Helin wrote: >>>>> On 2016-09-30, Leonid Mesnik wrote: >>>>>> Hi >>>>>> >>>>>> I think that it would be better to split this test into 4 tests. Currently >>>>>> this test is going to be skipped if any specific GCare set. Or it could even >>>>>> fail if any G1 specific options is set without setting G1 explicitly. >>>>>> >>>>>> So there is no way to run this with additional G1/CMS/ParGC specific options >>>>>> now. >>>>> Ok, I've split the test into four "tests" (they just run GCBasher with >>>>> different GCs). Please see new patches: >>>>> >>>>> - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00-01/ >>>>> - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01/ >>>> That seems fine to me too. >>>> >>>> One minor issue - all the copyright notices in the new files are slightly >>>> wrong (and we have a big patch in the pipeline that is fixing all the >>>> existing tests that have errors). In the copyright notice you have: >>>> >>>> 1 /* >>>> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights >>>> reserved. >>>> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>> ... >>>> 21 * questions. >>>> 22 */ >>>> >>>> but the correct format requires that all the asterisks line up: >>>> >>>> 1 /* >>>> 2 * Copyright (c) 2016, Oracle and/or its affiliates. All rights >>>> reserved. >>>> 3 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>> ... >>>> 21 * questions. >>>> 22 */ >>> Ok, no problem. You can find updated patches below (I also added an >>> @requires for -server): >>> - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/01-02/ >>> - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02/ >>> >>> Thanks, >>> Erik >>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> Erik >>>>> >>>>>> Leonid >>>>>> >>>>>> >>>>>> On 28.09.2016 18:01, Erik Helin wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> this patch adds a new GC stress test called GCBasher. GCBasher builds up >>>>>>> large (well, for some definiton of large) object graphs by figuring out >>>>>>> the relations between classes in the JDK. The test usually stresses the >>>>>>> GC quite a lot, especially when run with a smaller heap. >>>>>>> >>>>>>> The changes in the top-level repository are for JPRT. JPRT will now run >>>>>>> the jtreg test GCBasher instead of the old version. >>>>>>> >>>>>>> Enhancement: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166790 >>>>>>> >>>>>>> Webrev: >>>>>>> - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ >>>>>>> - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ >>>>>>> >>>>>>> Testing: >>>>>>> - Running the new jtreg test locally on Linux x86-64: >>>>>>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java >>>>>>> >>>>>>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) >>>>>>> - JPRT >>>>>>> >>>>>>> Thanks, >>>>>>> Erik >> From thomas.schatzl at oracle.com Fri Oct 7 07:20:37 2016 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 07 Oct 2016 09:20:37 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20161006143811.GB6035@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> <20161004094252.GL20056@ehelin.jrpg.bea.com> <7d227adf-e591-86ba-9baa-34583e28b0b5@oracle.com> <20161006125739.GA6035@ehelin.jrpg.bea.com> <78be1d1a-8672-202b-5639-6f06f46e8324@oracle.com> <20161006143811.GB6035@ehelin.jrpg.bea.com> Message-ID: <1475824837.2622.1.camel@oracle.com> Hi, On Thu, 2016-10-06 at 16:38 +0200, Erik Helin wrote: > David, Dmitry, > > below you will find new patches: > - incremental: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/02- > 03/ > - full: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/03/ > > I've fixed the indentation of the comments of the GPL header in all > files. I also removed '| vm.gc=="null"' from all @requires lines. I > also > filed an enhancement to investigate if we can remove -server option > [0]. > ? looks good. Thomas From serguei.spitsyn at oracle.com Fri Oct 7 12:02:06 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Oct 2016 05:02:06 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> Message-ID: Hi Coleen, It looks good to me. Some minor comments. http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html 253 const static uint _small_block_max_size = sizeof(TreeChunk >)/HeapWordSize; 260 assert(word_size >= _small_block_min_size, "There are no metaspace objects less than %u words", _small_block_min_size); Extra space before FreeList and after %u. 903 // Try small_blocks first 904 if (word_size < SmallBlocks::small_block_max_size()) { 905 // Don't create small_blocks() until needed. It makes sense to combine both comments. http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests/RedefineLeak.java.html 38 import java.lang.NoSuchFieldException; 39 import java.lang.NoSuchMethodException; These imports can be removed. Thanks, Serguei On 10/6/16 10:54, Coleen Phillimore wrote: > Here is an update to the Metadata leak change. There was a bug > introduced when cleaning this up, which Mikael also found. open webrev > at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev Changes > include adding logging for report_metadata_oome, which necessitated > removing ResourceMarks in ClassLoaderData::dump because the stream > passed in already required a ResourceMark, so it got a nested > ResourceMark message for the stream. I changed logging for tracing > block allocations to log_trace(gc, metaspace, freelist, blocks). In > BlockFreelist::get_block and BlockFreelist::return_block() assert that > it's not called for a size smaller than the minimum allocation (which > was the bug). Renamed get_raw_word_size() to > get_allocation_word_size(). This rounds up to the minimum allocation > size which is the same as small_block_min_size. Also, I added a test > that takes a long time to execute to verify this, and excluded it from > JPRT. I could skip adding this test if people don't want it. Also, > the test verifies that continuously redefining the same class gets > memory for the new class that was released because the block sizes are > the same. When the test exits, it gets a metaspace OOM because > loading new classes and allocating metadata can't use the blocks > returned (wrong size). There is still fragmentation in this > implementation, but it's better that things deallocated < 12 words are > actually freed. I'll file an RFE to work on a perfect algorithm, or > to investigate finding a better one, although I consider this a stress > test that uses all of metaspace to MaxMetaspaceSize, leaving > allocation only to the block fragments left. This isn't a typical use > case. Some comments and corrections to my responses to Mikael below: > On 10/4/16 12:15 PM, Coleen Phillimore wrote: >> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >> Mikael Gerdin wrote: >>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>> "dark matter" this change adds an array of small blocks by size, >>>> created lazily, to return Metablocks smaller than the >>>> BinaryTreeDictionary entry's minimum size. This change also fixed >>>> a bug in small object double free and adds debugging code to check >>>> for this case. With this change, the submitted test case runs >>>> indefinitely. Also passed rbt tier 1-5 testing. open webrev at >>>> http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>>> https://bugs.openjdk.java.net/browse/JDK-8164921 >>> I'd prefer it if SmallBlocks didn't expose its implementation by >>> returning its FreeLists by reference, could you change it to have * >>> return_chunk() * get_chunk() * num_chunks(word_size) and get rid of >>> list_at? >> Okay, I refactored this so small_chunks()->get_block() and >> return_block() are used rather than getting list_at. I didn't see >> where you got num_chunks, but list_at is hidden. >>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>> see that it's debug only but I fear that it could case problems with >>> running out of native memory in our internal testing of debug builds. >> I thought about taking it out, but it helped me find the double free >> bug. I think if we add new code to metadata and have to call >> deallocate_contents on it, we risk re-introducting these double free >> bugs. I could take it out. I don't think this gets that big but >> I'd hate to introduce some sort of OOM bug in our testing. >>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>> the minimum size of blocks to be put on the BlockTreeDictionary, not >>> the minimum size of blocks which are reusable. >> How about min_dictionary_size() ? >>> Is there any particular reason behind _small_blocks being lazy >>> allocated and _dictionary not? >> We lazily create the BlockFreelists with this change. // Lazily >> create a block_freelist if (block_freelists() == NULL) { >> _block_freelists = new BlockFreelist(); } So the small_blocks are >> lazily created in the BlockFreelists but the dictionary is not. I >> guess if we're going to create the BlockFreelists here, we'll most >> likely need both and maybe small_blocks need not be lazily created. >> Was that your suggestion? My concern with this change was all the >> space used by the small_blocks() but if we're doing any deallocation >> within the metaspace, at least one of the things will be <12 words. >> I'll make small_blocks() not be lazily allocated since BlockFreelist >> are. These are pretty expensive, but should be limited to a few >> metaspaces. > Class SpaceManager doesn't need small_blocks() so I left small_blocks > as lazily allocated. >>> I would prefer if BlockFreelist::return_block would perform the >>> checks in reverse order instead of having a return inside the first >>> if block, something like if (word_size > small_block_max_size) { >>> dict()->return_chunk(c) } else if (word_size > small_block_min_size) >>> { small_blocks()->return_chunk(c) } else { // dark matter } >> Why? We don't want to cast Metablock into dark matter so check if >> word_size < small_block_min_size first. Metablock* free_chunk = >> ::new (p) Metablock(word_size); if (word_size < >> SmallBlocks::small_block_max_size()) { >> small_blocks()->return_chunk(word_size); } else { >> dictionary()->return_chunk(free_chunk); } > There is no dark matter in these functions anymore. >>> For BlockFreelist::get_block I realize that the code is a bit more >>> complex but this also raises a few questions. * When we allocate >>> from the dictionary we search for a block which is at least as large >>> as we ask for and split it, returning whatever was left back to the >>> free list. We don't search for "large enough" blocks from the small >>> blocks manager, is that intentional or just to keep the code simple >>> (I like simple)? >> I'm glad you asked about this so I could give background. It turns >> out that we deallocate metaspace items better this way. I had a >> version that did exactly what you said. It was a simple sorted >> linked list of returned blocks < min_dictionary_size (12) where >> get_block returned the first block where the item would fit. It had >> some best fit algorithm so if the block returned was a lot bigger, it >> wouldn't pick it. My implementation could get through 69300 >> retransformations before the list didn't work anymore (too many small >> block fragments of the wrong size) and metaspace was exhausted >> (metaspace was limited to 12M in this test). Jon's implementation >> ran this test indefinitely. So it's somewhat simple but it worked >> really well. >>> In the last part of get_block where we return the unused part of a >>> block retrieved from the dictionary uses compares with >>> BlockFreelist::min_size() which, as I mentioned above, is the min >>> size for blocks in the dictionary, not the min size for blocks to be >>> reusable. I think this code can just call return_block >>> unconditionally for any nonzero value of "unused" and let >>> return_block deal with dark matter. >> yes, I'll make that change. > This change has to be conditional because I assert that > BlockFreelist::return_block() is never called for < > small_block_min_size. Thanks, Coleen >>> Are the changes to Method and ConstMethod the "bug fix to small >>> object double free"? >> Yes. >>> Is the problem that when a method was freed its annotations were >>> deallocated as well? Could the annotations fields in the "old" >>> ConstMethod be cleared instead so that the old annotation arrays >>> could be kept or is that just needless complexity? >> I'd rather copy the annotations, because I don't know how the old >> method, which could still be running, might use the annotations. I >> don't want to mess with that but I see your point. Coleen >>> Thanks /Mikael >>>> Thanks, Coleen and Jon From serguei.spitsyn at oracle.com Fri Oct 7 12:15:26 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Oct 2016 05:15:26 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> Message-ID: <60cae5fe-4db4-9d3d-d114-9b6d78ff35aa@oracle.com> On 10/7/16 05:02, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > It looks good to me. > Some minor comments. > > http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html > > 253 const static uint _small_block_max_size = > sizeof(TreeChunk >)/HeapWordSize; 260 > assert(word_size >= _small_block_min_size, "There are no metaspace > objects less than %u words", _small_block_min_size); > Extra space before FreeList and after %u. > 903 // Try small_blocks first > 904 if (word_size < SmallBlocks::small_block_max_size()) { > 905 // Don't create small_blocks() until needed. > It makes sense to combine both comments. > http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests One small question. 27 * @summary Test that type annotations are retained after a retransform . . . 49 static class Tester {} Is it a correct summary? I do not see type annotations in the class Tester that is being retransformed. Probably, I'm missing something. Thanks, Serguei > /RedefineLeak.java.html > 38 import java.lang.NoSuchFieldException; > 39 import java.lang.NoSuchMethodException; > These imports can be removed. Thanks, Serguei On 10/6/16 10:54, > Coleen Phillimore wrote: >> Here is an update to the Metadata leak change. There was a bug >> introduced when cleaning this up, which Mikael also found. open >> webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >> Changes include adding logging for report_metadata_oome, which >> necessitated removing ResourceMarks in ClassLoaderData::dump because >> the stream passed in already required a ResourceMark, so it got a >> nested ResourceMark message for the stream. I changed logging for >> tracing block allocations to log_trace(gc, metaspace, freelist, >> blocks). In BlockFreelist::get_block and >> BlockFreelist::return_block() assert that it's not called for a size >> smaller than the minimum allocation (which was the bug). Renamed >> get_raw_word_size() to get_allocation_word_size(). This rounds up to >> the minimum allocation size which is the same as >> small_block_min_size. Also, I added a test that takes a long time to >> execute to verify this, and excluded it from JPRT. I could skip >> adding this test if people don't want it. Also, the test verifies >> that continuously redefining the same class gets memory for the new >> class that was released because the block sizes are the same. When >> the test exits, it gets a metaspace OOM because loading new classes >> and allocating metadata can't use the blocks returned (wrong size). >> There is still fragmentation in this implementation, but it's better >> that things deallocated < 12 words are actually freed. I'll file an >> RFE to work on a perfect algorithm, or to investigate finding a >> better one, although I consider this a stress test that uses all of >> metaspace to MaxMetaspaceSize, leaving allocation only to the block >> fragments left. This isn't a typical use case. Some comments and >> corrections to my responses to Mikael below: On 10/4/16 12:15 PM, >> Coleen Phillimore wrote: >>> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >>> Mikael Gerdin wrote: >>>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>>> "dark matter" this change adds an array of small blocks by size, >>>>> created lazily, to return Metablocks smaller than the >>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>> fixed a bug in small object double free and adds debugging code to >>>>> check for this case. With this change, the submitted test case >>>>> runs indefinitely. Also passed rbt tier 1-5 testing. open webrev >>>>> at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>>>> https://bugs.openjdk.java.net/browse/JDK-8164921 >>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>> returning its FreeLists by reference, could you change it to have * >>>> return_chunk() * get_chunk() * num_chunks(word_size) and get rid of >>>> list_at? >>> Okay, I refactored this so small_chunks()->get_block() and >>> return_block() are used rather than getting list_at. I didn't see >>> where you got num_chunks, but list_at is hidden. >>>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>>> see that it's debug only but I fear that it could case problems >>>> with running out of native memory in our internal testing of debug >>>> builds. >>> I thought about taking it out, but it helped me find the double free >>> bug. I think if we add new code to metadata and have to call >>> deallocate_contents on it, we risk re-introducting these double free >>> bugs. I could take it out. I don't think this gets that big but >>> I'd hate to introduce some sort of OOM bug in our testing. >>>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>>> the minimum size of blocks to be put on the BlockTreeDictionary, >>>> not the minimum size of blocks which are reusable. >>> How about min_dictionary_size() ? >>>> Is there any particular reason behind _small_blocks being lazy >>>> allocated and _dictionary not? >>> We lazily create the BlockFreelists with this change. // Lazily >>> create a block_freelist if (block_freelists() == NULL) { >>> _block_freelists = new BlockFreelist(); } So the small_blocks are >>> lazily created in the BlockFreelists but the dictionary is not. I >>> guess if we're going to create the BlockFreelists here, we'll most >>> likely need both and maybe small_blocks need not be lazily created. >>> Was that your suggestion? My concern with this change was all the >>> space used by the small_blocks() but if we're doing any deallocation >>> within the metaspace, at least one of the things will be <12 words. >>> I'll make small_blocks() not be lazily allocated since BlockFreelist >>> are. These are pretty expensive, but should be limited to a few >>> metaspaces. >> Class SpaceManager doesn't need small_blocks() so I left small_blocks >> as lazily allocated. >>>> I would prefer if BlockFreelist::return_block would perform the >>>> checks in reverse order instead of having a return inside the first >>>> if block, something like if (word_size > small_block_max_size) { >>>> dict()->return_chunk(c) } else if (word_size > >>>> small_block_min_size) { small_blocks()->return_chunk(c) } else { >>>> // dark matter } >>> Why? We don't want to cast Metablock into dark matter so check if >>> word_size < small_block_min_size first. Metablock* free_chunk = >>> ::new (p) Metablock(word_size); if (word_size < >>> SmallBlocks::small_block_max_size()) { >>> small_blocks()->return_chunk(word_size); } else { >>> dictionary()->return_chunk(free_chunk); } >> There is no dark matter in these functions anymore. >>>> For BlockFreelist::get_block I realize that the code is a bit more >>>> complex but this also raises a few questions. * When we allocate >>>> from the dictionary we search for a block which is at least as >>>> large as we ask for and split it, returning whatever was left back >>>> to the free list. We don't search for "large enough" blocks from >>>> the small blocks manager, is that intentional or just to keep the >>>> code simple (I like simple)? >>> I'm glad you asked about this so I could give background. It turns >>> out that we deallocate metaspace items better this way. I had a >>> version that did exactly what you said. It was a simple sorted >>> linked list of returned blocks < min_dictionary_size (12) where >>> get_block returned the first block where the item would fit. It had >>> some best fit algorithm so if the block returned was a lot bigger, >>> it wouldn't pick it. My implementation could get through 69300 >>> retransformations before the list didn't work anymore (too many >>> small block fragments of the wrong size) and metaspace was exhausted >>> (metaspace was limited to 12M in this test). Jon's implementation >>> ran this test indefinitely. So it's somewhat simple but it worked >>> really well. >>>> In the last part of get_block where we return the unused part of a >>>> block retrieved from the dictionary uses compares with >>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>> size for blocks in the dictionary, not the min size for blocks to >>>> be reusable. I think this code can just call return_block >>>> unconditionally for any nonzero value of "unused" and let >>>> return_block deal with dark matter. >>> yes, I'll make that change. >> This change has to be conditional because I assert that >> BlockFreelist::return_block() is never called for < >> small_block_min_size. Thanks, Coleen >>>> Are the changes to Method and ConstMethod the "bug fix to small >>>> object double free"? >>> Yes. >>>> Is the problem that when a method was freed its annotations were >>>> deallocated as well? Could the annotations fields in the "old" >>>> ConstMethod be cleared instead so that the old annotation arrays >>>> could be kept or is that just needless complexity? >>> I'd rather copy the annotations, because I don't know how the old >>> method, which could still be running, might use the annotations. I >>> don't want to mess with that but I see your point. Coleen >>>> Thanks /Mikael >>>>> Thanks, Coleen and Jon From coleen.phillimore at oracle.com Fri Oct 7 12:32:00 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 7 Oct 2016 08:32:00 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> Message-ID: <41975ac2-41fe-42ac-457e-3d0f329dd1da@oracle.com> On 10/7/16 8:02 AM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > It looks good to me. > Some minor comments. > > http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html > > 253 const static uint _small_block_max_size = > sizeof(TreeChunk >)/HeapWordSize; 260 > assert(word_size >= _small_block_min_size, "There are no metaspace > objects less than %u words", _small_block_min_size); > Extra space before FreeList and after %u. Fixed. > 903 // Try small_blocks first > 904 if (word_size < SmallBlocks::small_block_max_size()) { > 905 // Don't create small_blocks() until needed. > It makes sense to combine both comments. The first comment applies to like 904 and the second applies to 906. I could add to 905, that calling small_blocks() allocates the small block lists. > http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests/RedefineLeak.java.html > > 38 import java.lang.NoSuchFieldException; > 39 import java.lang.NoSuchMethodException; You're right - copied from another test. Thanks! Coleen > These imports can be removed. Thanks, Serguei On 10/6/16 10:54, > Coleen Phillimore wrote: >> Here is an update to the Metadata leak change. There was a bug >> introduced when cleaning this up, which Mikael also found. open >> webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >> Changes include adding logging for report_metadata_oome, which >> necessitated removing ResourceMarks in ClassLoaderData::dump because >> the stream passed in already required a ResourceMark, so it got a >> nested ResourceMark message for the stream. I changed logging for >> tracing block allocations to log_trace(gc, metaspace, freelist, >> blocks). In BlockFreelist::get_block and >> BlockFreelist::return_block() assert that it's not called for a size >> smaller than the minimum allocation (which was the bug). Renamed >> get_raw_word_size() to get_allocation_word_size(). This rounds up to >> the minimum allocation size which is the same as >> small_block_min_size. Also, I added a test that takes a long time to >> execute to verify this, and excluded it from JPRT. I could skip >> adding this test if people don't want it. Also, the test verifies >> that continuously redefining the same class gets memory for the new >> class that was released because the block sizes are the same. When >> the test exits, it gets a metaspace OOM because loading new classes >> and allocating metadata can't use the blocks returned (wrong size). >> There is still fragmentation in this implementation, but it's better >> that things deallocated < 12 words are actually freed. I'll file an >> RFE to work on a perfect algorithm, or to investigate finding a >> better one, although I consider this a stress test that uses all of >> metaspace to MaxMetaspaceSize, leaving allocation only to the block >> fragments left. This isn't a typical use case. Some comments and >> corrections to my responses to Mikael below: On 10/4/16 12:15 PM, >> Coleen Phillimore wrote: >>> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >>> Mikael Gerdin wrote: >>>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>>> "dark matter" this change adds an array of small blocks by size, >>>>> created lazily, to return Metablocks smaller than the >>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>> fixed a bug in small object double free and adds debugging code to >>>>> check for this case. With this change, the submitted test case >>>>> runs indefinitely. Also passed rbt tier 1-5 testing. open webrev >>>>> at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>>>> https://bugs.openjdk.java.net/browse/JDK-8164921 >>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>> returning its FreeLists by reference, could you change it to have * >>>> return_chunk() * get_chunk() * num_chunks(word_size) and get rid of >>>> list_at? >>> Okay, I refactored this so small_chunks()->get_block() and >>> return_block() are used rather than getting list_at. I didn't see >>> where you got num_chunks, but list_at is hidden. >>>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>>> see that it's debug only but I fear that it could case problems >>>> with running out of native memory in our internal testing of debug >>>> builds. >>> I thought about taking it out, but it helped me find the double free >>> bug. I think if we add new code to metadata and have to call >>> deallocate_contents on it, we risk re-introducting these double free >>> bugs. I could take it out. I don't think this gets that big but >>> I'd hate to introduce some sort of OOM bug in our testing. >>>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>>> the minimum size of blocks to be put on the BlockTreeDictionary, >>>> not the minimum size of blocks which are reusable. >>> How about min_dictionary_size() ? >>>> Is there any particular reason behind _small_blocks being lazy >>>> allocated and _dictionary not? >>> We lazily create the BlockFreelists with this change. // Lazily >>> create a block_freelist if (block_freelists() == NULL) { >>> _block_freelists = new BlockFreelist(); } So the small_blocks are >>> lazily created in the BlockFreelists but the dictionary is not. I >>> guess if we're going to create the BlockFreelists here, we'll most >>> likely need both and maybe small_blocks need not be lazily created. >>> Was that your suggestion? My concern with this change was all the >>> space used by the small_blocks() but if we're doing any deallocation >>> within the metaspace, at least one of the things will be <12 words. >>> I'll make small_blocks() not be lazily allocated since BlockFreelist >>> are. These are pretty expensive, but should be limited to a few >>> metaspaces. >> Class SpaceManager doesn't need small_blocks() so I left small_blocks >> as lazily allocated. >>>> I would prefer if BlockFreelist::return_block would perform the >>>> checks in reverse order instead of having a return inside the first >>>> if block, something like if (word_size > small_block_max_size) { >>>> dict()->return_chunk(c) } else if (word_size > >>>> small_block_min_size) { small_blocks()->return_chunk(c) } else { >>>> // dark matter } >>> Why? We don't want to cast Metablock into dark matter so check if >>> word_size < small_block_min_size first. Metablock* free_chunk = >>> ::new (p) Metablock(word_size); if (word_size < >>> SmallBlocks::small_block_max_size()) { >>> small_blocks()->return_chunk(word_size); } else { >>> dictionary()->return_chunk(free_chunk); } >> There is no dark matter in these functions anymore. >>>> For BlockFreelist::get_block I realize that the code is a bit more >>>> complex but this also raises a few questions. * When we allocate >>>> from the dictionary we search for a block which is at least as >>>> large as we ask for and split it, returning whatever was left back >>>> to the free list. We don't search for "large enough" blocks from >>>> the small blocks manager, is that intentional or just to keep the >>>> code simple (I like simple)? >>> I'm glad you asked about this so I could give background. It turns >>> out that we deallocate metaspace items better this way. I had a >>> version that did exactly what you said. It was a simple sorted >>> linked list of returned blocks < min_dictionary_size (12) where >>> get_block returned the first block where the item would fit. It had >>> some best fit algorithm so if the block returned was a lot bigger, >>> it wouldn't pick it. My implementation could get through 69300 >>> retransformations before the list didn't work anymore (too many >>> small block fragments of the wrong size) and metaspace was exhausted >>> (metaspace was limited to 12M in this test). Jon's implementation >>> ran this test indefinitely. So it's somewhat simple but it worked >>> really well. >>>> In the last part of get_block where we return the unused part of a >>>> block retrieved from the dictionary uses compares with >>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>> size for blocks in the dictionary, not the min size for blocks to >>>> be reusable. I think this code can just call return_block >>>> unconditionally for any nonzero value of "unused" and let >>>> return_block deal with dark matter. >>> yes, I'll make that change. >> This change has to be conditional because I assert that >> BlockFreelist::return_block() is never called for < >> small_block_min_size. Thanks, Coleen >>>> Are the changes to Method and ConstMethod the "bug fix to small >>>> object double free"? >>> Yes. >>>> Is the problem that when a method was freed its annotations were >>>> deallocated as well? Could the annotations fields in the "old" >>>> ConstMethod be cleared instead so that the old annotation arrays >>>> could be kept or is that just needless complexity? >>> I'd rather copy the annotations, because I don't know how the old >>> method, which could still be running, might use the annotations. I >>> don't want to mess with that but I see your point. Coleen >>>> Thanks /Mikael >>>>> Thanks, Coleen and Jon From coleen.phillimore at oracle.com Fri Oct 7 12:33:17 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 7 Oct 2016 08:33:17 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <60cae5fe-4db4-9d3d-d114-9b6d78ff35aa@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <60cae5fe-4db4-9d3d-d114-9b6d78ff35aa@oracle.com> Message-ID: On 10/7/16 8:15 AM, serguei.spitsyn at oracle.com wrote: > On 10/7/16 05:02, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> It looks good to me. >> Some minor comments. >> >> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html >> >> >> 253 const static uint _small_block_max_size = >> sizeof(TreeChunk >)/HeapWordSize; 260 >> assert(word_size >= _small_block_min_size, "There are no metaspace >> objects less than %u words", _small_block_min_size); >> Extra space before FreeList and after %u. >> 903 // Try small_blocks first >> 904 if (word_size < SmallBlocks::small_block_max_size()) { >> 905 // Don't create small_blocks() until needed. >> It makes sense to combine both comments. >> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests > > One small question. > > 27 * @summary Test that type annotations are retained after a > retransform > . . . > 49 static class Tester {} > > > Is it a correct summary? I do not see type annotations in the > class Tester that is being retransformed. Probably, I'm missing > something. Thanks, Serguei I copied that from the other test too. I thought I'd fixed the summary. I'll fix it now to: @summary Test that redefinition reuses metaspace blocks that are freed Thanks, Coleen >> /RedefineLeak.java.html >> 38 import java.lang.NoSuchFieldException; >> 39 import java.lang.NoSuchMethodException; >> These imports can be removed. Thanks, Serguei On 10/6/16 10:54, >> Coleen Phillimore wrote: >>> Here is an update to the Metadata leak change. There was a bug >>> introduced when cleaning this up, which Mikael also found. open >>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >>> Changes include adding logging for report_metadata_oome, which >>> necessitated removing ResourceMarks in ClassLoaderData::dump because >>> the stream passed in already required a ResourceMark, so it got a >>> nested ResourceMark message for the stream. I changed logging for >>> tracing block allocations to log_trace(gc, metaspace, freelist, >>> blocks). In BlockFreelist::get_block and >>> BlockFreelist::return_block() assert that it's not called for a size >>> smaller than the minimum allocation (which was the bug). Renamed >>> get_raw_word_size() to get_allocation_word_size(). This rounds up >>> to the minimum allocation size which is the same as >>> small_block_min_size. Also, I added a test that takes a long time to >>> execute to verify this, and excluded it from JPRT. I could skip >>> adding this test if people don't want it. Also, the test verifies >>> that continuously redefining the same class gets memory for the new >>> class that was released because the block sizes are the same. When >>> the test exits, it gets a metaspace OOM because loading new classes >>> and allocating metadata can't use the blocks returned (wrong size). >>> There is still fragmentation in this implementation, but it's better >>> that things deallocated < 12 words are actually freed. I'll file an >>> RFE to work on a perfect algorithm, or to investigate finding a >>> better one, although I consider this a stress test that uses all of >>> metaspace to MaxMetaspaceSize, leaving allocation only to the block >>> fragments left. This isn't a typical use case. Some comments and >>> corrections to my responses to Mikael below: On 10/4/16 12:15 PM, >>> Coleen Phillimore wrote: >>>> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >>>> Mikael Gerdin wrote: >>>>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>>>> "dark matter" this change adds an array of small blocks by size, >>>>>> created lazily, to return Metablocks smaller than the >>>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>>> fixed a bug in small object double free and adds debugging code >>>>>> to check for this case. With this change, the submitted test case >>>>>> runs indefinitely. Also passed rbt tier 1-5 testing. open webrev >>>>>> at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>>>>> https://bugs.openjdk.java.net/browse/JDK-8164921 >>>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>>> returning its FreeLists by reference, could you change it to have >>>>> * return_chunk() * get_chunk() * num_chunks(word_size) and get rid >>>>> of list_at? >>>> Okay, I refactored this so small_chunks()->get_block() and >>>> return_block() are used rather than getting list_at. I didn't see >>>> where you got num_chunks, but list_at is hidden. >>>>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>>>> see that it's debug only but I fear that it could case problems >>>>> with running out of native memory in our internal testing of debug >>>>> builds. >>>> I thought about taking it out, but it helped me find the double >>>> free bug. I think if we add new code to metadata and have to call >>>> deallocate_contents on it, we risk re-introducting these double >>>> free bugs. I could take it out. I don't think this gets that big >>>> but I'd hate to introduce some sort of OOM bug in our testing. >>>>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>>>> the minimum size of blocks to be put on the BlockTreeDictionary, >>>>> not the minimum size of blocks which are reusable. >>>> How about min_dictionary_size() ? >>>>> Is there any particular reason behind _small_blocks being lazy >>>>> allocated and _dictionary not? >>>> We lazily create the BlockFreelists with this change. // Lazily >>>> create a block_freelist if (block_freelists() == NULL) { >>>> _block_freelists = new BlockFreelist(); } So the small_blocks are >>>> lazily created in the BlockFreelists but the dictionary is not. I >>>> guess if we're going to create the BlockFreelists here, we'll most >>>> likely need both and maybe small_blocks need not be lazily >>>> created. Was that your suggestion? My concern with this change was >>>> all the space used by the small_blocks() but if we're doing any >>>> deallocation within the metaspace, at least one of the things will >>>> be <12 words. I'll make small_blocks() not be lazily allocated >>>> since BlockFreelist are. These are pretty expensive, but should be >>>> limited to a few metaspaces. >>> Class SpaceManager doesn't need small_blocks() so I left >>> small_blocks as lazily allocated. >>>>> I would prefer if BlockFreelist::return_block would perform the >>>>> checks in reverse order instead of having a return inside the >>>>> first if block, something like if (word_size > >>>>> small_block_max_size) { dict()->return_chunk(c) } else if >>>>> (word_size > small_block_min_size) { >>>>> small_blocks()->return_chunk(c) } else { // dark matter } >>>> Why? We don't want to cast Metablock into dark matter so check if >>>> word_size < small_block_min_size first. Metablock* free_chunk = >>>> ::new (p) Metablock(word_size); if (word_size < >>>> SmallBlocks::small_block_max_size()) { >>>> small_blocks()->return_chunk(word_size); } else { >>>> dictionary()->return_chunk(free_chunk); } >>> There is no dark matter in these functions anymore. >>>>> For BlockFreelist::get_block I realize that the code is a bit more >>>>> complex but this also raises a few questions. * When we allocate >>>>> from the dictionary we search for a block which is at least as >>>>> large as we ask for and split it, returning whatever was left back >>>>> to the free list. We don't search for "large enough" blocks from >>>>> the small blocks manager, is that intentional or just to keep the >>>>> code simple (I like simple)? >>>> I'm glad you asked about this so I could give background. It turns >>>> out that we deallocate metaspace items better this way. I had a >>>> version that did exactly what you said. It was a simple sorted >>>> linked list of returned blocks < min_dictionary_size (12) where >>>> get_block returned the first block where the item would fit. It >>>> had some best fit algorithm so if the block returned was a lot >>>> bigger, it wouldn't pick it. My implementation could get through >>>> 69300 retransformations before the list didn't work anymore (too >>>> many small block fragments of the wrong size) and metaspace was >>>> exhausted (metaspace was limited to 12M in this test). Jon's >>>> implementation ran this test indefinitely. So it's somewhat simple >>>> but it worked really well. >>>>> In the last part of get_block where we return the unused part of a >>>>> block retrieved from the dictionary uses compares with >>>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>>> size for blocks in the dictionary, not the min size for blocks to >>>>> be reusable. I think this code can just call return_block >>>>> unconditionally for any nonzero value of "unused" and let >>>>> return_block deal with dark matter. >>>> yes, I'll make that change. >>> This change has to be conditional because I assert that >>> BlockFreelist::return_block() is never called for < >>> small_block_min_size. Thanks, Coleen >>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>> object double free"? >>>> Yes. >>>>> Is the problem that when a method was freed its annotations were >>>>> deallocated as well? Could the annotations fields in the "old" >>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>> could be kept or is that just needless complexity? >>>> I'd rather copy the annotations, because I don't know how the old >>>> method, which could still be running, might use the annotations. I >>>> don't want to mess with that but I see your point. Coleen >>>>> Thanks /Mikael >>>>>> Thanks, Coleen and Jon > From coleen.phillimore at oracle.com Fri Oct 7 12:34:26 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 7 Oct 2016 08:34:26 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <60cae5fe-4db4-9d3d-d114-9b6d78ff35aa@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <60cae5fe-4db4-9d3d-d114-9b6d78ff35aa@oracle.com> Message-ID: On 10/7/16 8:15 AM, serguei.spitsyn at oracle.com wrote: > On 10/7/16 05:02, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> It looks good to me. >> Some minor comments. >> >> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html >> >> >> 253 const static uint _small_block_max_size = >> sizeof(TreeChunk >)/HeapWordSize; 260 >> assert(word_size >= _small_block_min_size, "There are no metaspace >> objects less than %u words", _small_block_min_size); >> Extra space before FreeList and after %u. >> 903 // Try small_blocks first >> 904 if (word_size < SmallBlocks::small_block_max_size()) { >> 905 // Don't create small_blocks() until needed. >> It makes sense to combine both comments. >> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests > > One small question. > > 27 * @summary Test that type annotations are retained after a > retransform > . . . > 49 static class Tester {} > > > Is it a correct summary? I do not see type annotations in the > class Tester that is being retransformed. Probably, I'm missing > something. Thanks, Serguei And I took out a module (asm) that wasn't needed also. thanks, Coleen >> /RedefineLeak.java.html >> 38 import java.lang.NoSuchFieldException; >> 39 import java.lang.NoSuchMethodException; >> These imports can be removed. Thanks, Serguei On 10/6/16 10:54, >> Coleen Phillimore wrote: >>> Here is an update to the Metadata leak change. There was a bug >>> introduced when cleaning this up, which Mikael also found. open >>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >>> Changes include adding logging for report_metadata_oome, which >>> necessitated removing ResourceMarks in ClassLoaderData::dump because >>> the stream passed in already required a ResourceMark, so it got a >>> nested ResourceMark message for the stream. I changed logging for >>> tracing block allocations to log_trace(gc, metaspace, freelist, >>> blocks). In BlockFreelist::get_block and >>> BlockFreelist::return_block() assert that it's not called for a size >>> smaller than the minimum allocation (which was the bug). Renamed >>> get_raw_word_size() to get_allocation_word_size(). This rounds up >>> to the minimum allocation size which is the same as >>> small_block_min_size. Also, I added a test that takes a long time to >>> execute to verify this, and excluded it from JPRT. I could skip >>> adding this test if people don't want it. Also, the test verifies >>> that continuously redefining the same class gets memory for the new >>> class that was released because the block sizes are the same. When >>> the test exits, it gets a metaspace OOM because loading new classes >>> and allocating metadata can't use the blocks returned (wrong size). >>> There is still fragmentation in this implementation, but it's better >>> that things deallocated < 12 words are actually freed. I'll file an >>> RFE to work on a perfect algorithm, or to investigate finding a >>> better one, although I consider this a stress test that uses all of >>> metaspace to MaxMetaspaceSize, leaving allocation only to the block >>> fragments left. This isn't a typical use case. Some comments and >>> corrections to my responses to Mikael below: On 10/4/16 12:15 PM, >>> Coleen Phillimore wrote: >>>> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >>>> Mikael Gerdin wrote: >>>>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>>>> "dark matter" this change adds an array of small blocks by size, >>>>>> created lazily, to return Metablocks smaller than the >>>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>>> fixed a bug in small object double free and adds debugging code >>>>>> to check for this case. With this change, the submitted test case >>>>>> runs indefinitely. Also passed rbt tier 1-5 testing. open webrev >>>>>> at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>>>>> https://bugs.openjdk.java.net/browse/JDK-8164921 >>>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>>> returning its FreeLists by reference, could you change it to have >>>>> * return_chunk() * get_chunk() * num_chunks(word_size) and get rid >>>>> of list_at? >>>> Okay, I refactored this so small_chunks()->get_block() and >>>> return_block() are used rather than getting list_at. I didn't see >>>> where you got num_chunks, but list_at is hidden. >>>>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>>>> see that it's debug only but I fear that it could case problems >>>>> with running out of native memory in our internal testing of debug >>>>> builds. >>>> I thought about taking it out, but it helped me find the double >>>> free bug. I think if we add new code to metadata and have to call >>>> deallocate_contents on it, we risk re-introducting these double >>>> free bugs. I could take it out. I don't think this gets that big >>>> but I'd hate to introduce some sort of OOM bug in our testing. >>>>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>>>> the minimum size of blocks to be put on the BlockTreeDictionary, >>>>> not the minimum size of blocks which are reusable. >>>> How about min_dictionary_size() ? >>>>> Is there any particular reason behind _small_blocks being lazy >>>>> allocated and _dictionary not? >>>> We lazily create the BlockFreelists with this change. // Lazily >>>> create a block_freelist if (block_freelists() == NULL) { >>>> _block_freelists = new BlockFreelist(); } So the small_blocks are >>>> lazily created in the BlockFreelists but the dictionary is not. I >>>> guess if we're going to create the BlockFreelists here, we'll most >>>> likely need both and maybe small_blocks need not be lazily >>>> created. Was that your suggestion? My concern with this change was >>>> all the space used by the small_blocks() but if we're doing any >>>> deallocation within the metaspace, at least one of the things will >>>> be <12 words. I'll make small_blocks() not be lazily allocated >>>> since BlockFreelist are. These are pretty expensive, but should be >>>> limited to a few metaspaces. >>> Class SpaceManager doesn't need small_blocks() so I left >>> small_blocks as lazily allocated. >>>>> I would prefer if BlockFreelist::return_block would perform the >>>>> checks in reverse order instead of having a return inside the >>>>> first if block, something like if (word_size > >>>>> small_block_max_size) { dict()->return_chunk(c) } else if >>>>> (word_size > small_block_min_size) { >>>>> small_blocks()->return_chunk(c) } else { // dark matter } >>>> Why? We don't want to cast Metablock into dark matter so check if >>>> word_size < small_block_min_size first. Metablock* free_chunk = >>>> ::new (p) Metablock(word_size); if (word_size < >>>> SmallBlocks::small_block_max_size()) { >>>> small_blocks()->return_chunk(word_size); } else { >>>> dictionary()->return_chunk(free_chunk); } >>> There is no dark matter in these functions anymore. >>>>> For BlockFreelist::get_block I realize that the code is a bit more >>>>> complex but this also raises a few questions. * When we allocate >>>>> from the dictionary we search for a block which is at least as >>>>> large as we ask for and split it, returning whatever was left back >>>>> to the free list. We don't search for "large enough" blocks from >>>>> the small blocks manager, is that intentional or just to keep the >>>>> code simple (I like simple)? >>>> I'm glad you asked about this so I could give background. It turns >>>> out that we deallocate metaspace items better this way. I had a >>>> version that did exactly what you said. It was a simple sorted >>>> linked list of returned blocks < min_dictionary_size (12) where >>>> get_block returned the first block where the item would fit. It >>>> had some best fit algorithm so if the block returned was a lot >>>> bigger, it wouldn't pick it. My implementation could get through >>>> 69300 retransformations before the list didn't work anymore (too >>>> many small block fragments of the wrong size) and metaspace was >>>> exhausted (metaspace was limited to 12M in this test). Jon's >>>> implementation ran this test indefinitely. So it's somewhat simple >>>> but it worked really well. >>>>> In the last part of get_block where we return the unused part of a >>>>> block retrieved from the dictionary uses compares with >>>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>>> size for blocks in the dictionary, not the min size for blocks to >>>>> be reusable. I think this code can just call return_block >>>>> unconditionally for any nonzero value of "unused" and let >>>>> return_block deal with dark matter. >>>> yes, I'll make that change. >>> This change has to be conditional because I assert that >>> BlockFreelist::return_block() is never called for < >>> small_block_min_size. Thanks, Coleen >>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>> object double free"? >>>> Yes. >>>>> Is the problem that when a method was freed its annotations were >>>>> deallocated as well? Could the annotations fields in the "old" >>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>> could be kept or is that just needless complexity? >>>> I'd rather copy the annotations, because I don't know how the old >>>> method, which could still be running, might use the annotations. I >>>> don't want to mess with that but I see your point. Coleen >>>>> Thanks /Mikael >>>>>> Thanks, Coleen and Jon > From marcus.larsson at oracle.com Fri Oct 7 14:26:41 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 7 Oct 2016 16:26:41 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism Message-ID: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Hi, Making another attempt to fix this issue. Summary: The following patch resolves a problem where the VM would crash during shutdown due to static log related memory being de-initialized before the last use of the logging framework. The solution involves parts of the Nifty Counter idiom [0] to control static initialization and de-initialization of stdout's and stderr's LogOutputs. Both objects are now allocated using placement new, and avoids destructor calls during de-initialization. The LogStreamInitializer makes sure both objects are initialized before first use. Because the LogOutput::Stdout/err pointers could no longer be kept in LogOutput, I've replaced all usages of them with the new references instead. The patch includes a regression test for this issue, contributed by Michail Chernov. Webrev: http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 Issue: https://bugs.openjdk.java.net/browse/JDK-8146009 Testing: JPRT testset hotspot, included test on supported platforms. Thanks, Marcus [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter From mikael.gerdin at oracle.com Fri Oct 7 14:39:22 2016 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 7 Oct 2016 16:39:22 +0200 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> Message-ID: <11f3d2d6-acac-fc06-3d03-96728b7905e9@oracle.com> Hi Coleen, On 2016-10-06 19:54, Coleen Phillimore wrote: > > Here is an update to the Metadata leak change. There was a bug > introduced when cleaning this up, which Mikael also found. > > open webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev > > Changes include adding logging for report_metadata_oome, which > necessitated removing ResourceMarks in ClassLoaderData::dump because the > stream passed in already required a ResourceMark, so it got a nested > ResourceMark message for the stream. > > I changed logging for tracing block allocations to log_trace(gc, > metaspace, freelist, blocks). > > In BlockFreelist::get_block and BlockFreelist::return_block() assert > that it's not called for a size smaller than the minimum allocation > (which was the bug). Renamed get_raw_word_size() to > get_allocation_word_size(). This rounds up to the minimum allocation > size which is the same as small_block_min_size. > > Also, I added a test that takes a long time to execute to verify this, > and excluded it from JPRT. I could skip adding this test if people > don't want it. Also, the test verifies that continuously redefining the > same class gets memory for the new class that was released because the > block sizes are the same. When the test exits, it gets a metaspace OOM > because loading new classes and allocating metadata can't use the blocks > returned (wrong size). There is still fragmentation in this > implementation, but it's better that things deallocated < 12 words are > actually freed. I'll file an RFE to work on a perfect algorithm, or to > investigate finding a better one, although I consider this a stress test > that uses all of metaspace to MaxMetaspaceSize, leaving allocation only > to the block fragments left. This isn't a typical use case. > > Some comments and corrections to my responses to Mikael below: > > On 10/4/16 12:15 PM, Coleen Phillimore wrote: >> >> Hi Mikael, >> >> Thanks for looking at this change. >> >> On 10/4/16 8:32 AM, Mikael Gerdin wrote: >>> Hi Coleen, >>> >>> On 2016-09-30 21:02, Coleen Phillimore wrote: >>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>> >>>> This change contributed by Jon Masamitsu and myself. To reclaim "dark >>>> matter" this change adds an array of small blocks by size, created >>>> lazily, to return Metablocks smaller than the BinaryTreeDictionary >>>> entry's minimum size. This change also fixed a bug in small object >>>> double free and adds debugging code to check for this case. >>>> >>>> With this change, the submitted test case runs indefinitely. Also >>>> passed rbt tier 1-5 testing. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >>> >>> I'd prefer it if SmallBlocks didn't expose its implementation by >>> returning its FreeLists by reference, could you change it to have >>> * return_chunk() >>> * get_chunk() >>> * num_chunks(word_size) >>> >>> and get rid of list_at? >> >> Okay, I refactored this so small_chunks()->get_block() and >> return_block() are used rather than getting list_at. I didn't see >> where you got num_chunks, but list_at is hidden. Thanks, that's much better! >>> >>> -- >>> >>> For how long do you plan to keep BlockFreelist::_all_blocks? >>> I see that it's debug only but I fear that it could case problems >>> with running out of native memory in our internal testing of debug >>> builds. >> >> I thought about taking it out, but it helped me find the double free >> bug. I think if we add new code to metadata and have to call >> deallocate_contents on it, we risk re-introducting these double free >> bugs. I could take it out. I don't think this gets that big but I'd >> hate to introduce some sort of OOM bug in our testing. Perhaps a trade off could be to have a bound on the size of the array and deallocate and disable it if it grows too large? >>> >>> BlockFreelist::min_size() is a bit of a misnomer since it returns the >>> minimum size of blocks to be put on the BlockTreeDictionary, not the >>> minimum size of blocks which are reusable. >> >> How about min_dictionary_size() ? That's fine. >> >>> >>> Is there any particular reason behind _small_blocks being lazy >>> allocated and _dictionary not? >> >> We lazily create the BlockFreelists with this change. >> >> // Lazily create a block_freelist >> if (block_freelists() == NULL) { >> _block_freelists = new BlockFreelist(); >> } >> >> So the small_blocks are lazily created in the BlockFreelists but the >> dictionary is not. I guess if we're going to create the >> BlockFreelists here, we'll most likely need both and maybe >> small_blocks need not be lazily created. Was that your suggestion? >> >> My concern with this change was all the space used by the >> small_blocks() but if we're doing any deallocation within the >> metaspace, at least one of the things will be <12 words. >> >> I'll make small_blocks() not be lazily allocated since BlockFreelist >> are. These are pretty expensive, but should be limited to a few >> metaspaces. > > Class SpaceManager doesn't need small_blocks() so I left small_blocks as > lazily allocated. Ok, I agree that the size of small_blocks() motivates its lazy allocation scheme. >>> >>> I would prefer if BlockFreelist::return_block would perform the >>> checks in reverse order instead of having a return inside the first >>> if block, something like >>> >>> if (word_size > small_block_max_size) { >>> dict()->return_chunk(c) >>> } else if (word_size > small_block_min_size) { >>> small_blocks()->return_chunk(c) >>> } else { >>> // dark matter >>> } >> >> Why? We don't want to cast Metablock into dark matter so check if >> word_size < small_block_min_size first. >> >> Metablock* free_chunk = ::new (p) Metablock(word_size); >> if (word_size < SmallBlocks::small_block_max_size()) { >> small_blocks()->return_chunk(word_size); >> } else { >> dictionary()->return_chunk(free_chunk); >> } >> > > There is no dark matter in these functions anymore. Oh! Right. >>> >>> >>> For BlockFreelist::get_block I realize that the code is a bit more >>> complex but this also raises a few questions. >>> * When we allocate from the dictionary we search for a block which is >>> at least as large as we ask for and split it, returning whatever was >>> left back to the free list. We don't search for "large enough" blocks >>> from the small blocks manager, is that intentional or just to keep >>> the code simple (I like simple)? >> >> I'm glad you asked about this so I could give background. It turns >> out that we deallocate metaspace items better this way. I had a >> version that did exactly what you said. It was a simple sorted linked >> list of returned blocks < min_dictionary_size (12) where get_block >> returned the first block where the item would fit. It had some best >> fit algorithm so if the block returned was a lot bigger, it wouldn't >> pick it. >> >> My implementation could get through 69300 retransformations before the >> list didn't work anymore (too many small block fragments of the wrong >> size) and metaspace was exhausted (metaspace was limited to 12M in >> this test). Jon's implementation ran this test indefinitely. So it's >> somewhat simple but it worked really well. >> >>> >>> In the last part of get_block where we return the unused part of a >>> block retrieved from the dictionary uses compares with >>> BlockFreelist::min_size() which, as I mentioned above, is the min >>> size for blocks in the dictionary, not the min size for blocks to be >>> reusable. >>> I think this code can just call return_block unconditionally for any >>> nonzero value of "unused" and let return_block deal with dark matter. >>> >> yes, I'll make that change. > > This change has to be conditional because I assert that > BlockFreelist::return_block() is never called for < small_block_min_size. Ok. There's one more thing, though. In SpaceManager::retire_current_chunk the last piece of the current allocation chunk is made reusable through a call to SpaceManager::deallocate. There are two problems with this: * retire_current_chunk only calls deallocate for wasted blocks larger than the min dictionary size, this means that wasted blocks which would fit in SmallBlocks will be wasted instead of reusable. * the remaining_words parameter which retire_current_chunk passes to deallocate is _exact_ and must not be aligned up through get_allocation_word_size. Also, are you sure about logging the output of ClassLoaderData::dump on the info level? Perhaps logging the cld dump on the debug level is a good trade-off? It looks like the trace level causes the VSM to print the entire free lists to the log so that's probably a bit much. > > Thanks, > Coleen > >>> >>> >>> Are the changes to Method and ConstMethod the "bug fix to small >>> object double free"? >> >> Yes. >>> Is the problem that when a method was freed its annotations were >>> deallocated as well? Could the annotations fields in the "old" >>> ConstMethod be cleared instead so that the old annotation arrays >>> could be kept or is that just needless complexity? >> >> I'd rather copy the annotations, because I don't know how the old >> method, which could still be running, might use the annotations. I >> don't want to mess with that but I see your point. That's fine I think, I wouldn't want to mess around with it either. I just wanted to understand the reasoning behind the change, thanks for confirming my suspicion. /Mikael >> >> Coleen >> >>> >>> Thanks >>> /Mikael >>> >>> >>> >>>> >>>> Thanks, >>>> Coleen and Jon >> > From coleen.phillimore at oracle.com Fri Oct 7 15:24:30 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 7 Oct 2016 11:24:30 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <11f3d2d6-acac-fc06-3d03-96728b7905e9@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <11f3d2d6-acac-fc06-3d03-96728b7905e9@oracle.com> Message-ID: <6bf01692-a37f-6ff2-8a92-27e379143ebe@oracle.com> Mikael, thanks again ... embedded replies and 2 questions: On 10/7/16 10:39 AM, Mikael Gerdin wrote: > Hi Coleen, > > On 2016-10-06 19:54, Coleen Phillimore wrote: >> >> Here is an update to the Metadata leak change. There was a bug >> introduced when cleaning this up, which Mikael also found. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >> >> Changes include adding logging for report_metadata_oome, which >> necessitated removing ResourceMarks in ClassLoaderData::dump because the >> stream passed in already required a ResourceMark, so it got a nested >> ResourceMark message for the stream. >> >> I changed logging for tracing block allocations to log_trace(gc, >> metaspace, freelist, blocks). >> >> In BlockFreelist::get_block and BlockFreelist::return_block() assert >> that it's not called for a size smaller than the minimum allocation >> (which was the bug). Renamed get_raw_word_size() to >> get_allocation_word_size(). This rounds up to the minimum allocation >> size which is the same as small_block_min_size. >> >> Also, I added a test that takes a long time to execute to verify this, >> and excluded it from JPRT. I could skip adding this test if people >> don't want it. Also, the test verifies that continuously redefining the >> same class gets memory for the new class that was released because the >> block sizes are the same. When the test exits, it gets a metaspace OOM >> because loading new classes and allocating metadata can't use the blocks >> returned (wrong size). There is still fragmentation in this >> implementation, but it's better that things deallocated < 12 words are >> actually freed. I'll file an RFE to work on a perfect algorithm, or to >> investigate finding a better one, although I consider this a stress test >> that uses all of metaspace to MaxMetaspaceSize, leaving allocation only >> to the block fragments left. This isn't a typical use case. >> >> Some comments and corrections to my responses to Mikael below: >> >> On 10/4/16 12:15 PM, Coleen Phillimore wrote: >>> >>> Hi Mikael, >>> >>> Thanks for looking at this change. >>> >>> On 10/4/16 8:32 AM, Mikael Gerdin wrote: >>>> Hi Coleen, >>>> >>>> On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>>> >>>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>>> "dark >>>>> matter" this change adds an array of small blocks by size, created >>>>> lazily, to return Metablocks smaller than the BinaryTreeDictionary >>>>> entry's minimum size. This change also fixed a bug in small object >>>>> double free and adds debugging code to check for this case. >>>>> >>>>> With this change, the submitted test case runs indefinitely. Also >>>>> passed rbt tier 1-5 testing. >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >>>> >>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>> returning its FreeLists by reference, could you change it to have >>>> * return_chunk() >>>> * get_chunk() >>>> * num_chunks(word_size) >>>> >>>> and get rid of list_at? >>> >>> Okay, I refactored this so small_chunks()->get_block() and >>> return_block() are used rather than getting list_at. I didn't see >>> where you got num_chunks, but list_at is hidden. > > Thanks, that's much better! > >>>> >>>> -- >>>> >>>> For how long do you plan to keep BlockFreelist::_all_blocks? >>>> I see that it's debug only but I fear that it could case problems >>>> with running out of native memory in our internal testing of debug >>>> builds. >>> >>> I thought about taking it out, but it helped me find the double free >>> bug. I think if we add new code to metadata and have to call >>> deallocate_contents on it, we risk re-introducting these double free >>> bugs. I could take it out. I don't think this gets that big but I'd >>> hate to introduce some sort of OOM bug in our testing. > > Perhaps a trade off could be to have a bound on the size of the array > and deallocate and disable it if it grows too large? I just removed it. I had run all the tests with the debugging code and they passed. If we have to, we can add it back again pretty easily. > >>>> >>>> BlockFreelist::min_size() is a bit of a misnomer since it returns the >>>> minimum size of blocks to be put on the BlockTreeDictionary, not the >>>> minimum size of blocks which are reusable. >>> >>> How about min_dictionary_size() ? > > That's fine. > >>> >>>> >>>> Is there any particular reason behind _small_blocks being lazy >>>> allocated and _dictionary not? >>> >>> We lazily create the BlockFreelists with this change. >>> >>> // Lazily create a block_freelist >>> if (block_freelists() == NULL) { >>> _block_freelists = new BlockFreelist(); >>> } >>> >>> So the small_blocks are lazily created in the BlockFreelists but the >>> dictionary is not. I guess if we're going to create the >>> BlockFreelists here, we'll most likely need both and maybe >>> small_blocks need not be lazily created. Was that your suggestion? >>> >>> My concern with this change was all the space used by the >>> small_blocks() but if we're doing any deallocation within the >>> metaspace, at least one of the things will be <12 words. >>> >>> I'll make small_blocks() not be lazily allocated since BlockFreelist >>> are. These are pretty expensive, but should be limited to a few >>> metaspaces. >> >> Class SpaceManager doesn't need small_blocks() so I left small_blocks as >> lazily allocated. > > Ok, I agree that the size of small_blocks() motivates its lazy > allocation scheme. > >>>> >>>> I would prefer if BlockFreelist::return_block would perform the >>>> checks in reverse order instead of having a return inside the first >>>> if block, something like >>>> >>>> if (word_size > small_block_max_size) { >>>> dict()->return_chunk(c) >>>> } else if (word_size > small_block_min_size) { >>>> small_blocks()->return_chunk(c) >>>> } else { >>>> // dark matter >>>> } >>> >>> Why? We don't want to cast Metablock into dark matter so check if >>> word_size < small_block_min_size first. >>> >>> Metablock* free_chunk = ::new (p) Metablock(word_size); >>> if (word_size < SmallBlocks::small_block_max_size()) { >>> small_blocks()->return_chunk(word_size); >>> } else { >>> dictionary()->return_chunk(free_chunk); >>> } >>> >> >> There is no dark matter in these functions anymore. > > Oh! Right. > > >>>> >>>> >>>> For BlockFreelist::get_block I realize that the code is a bit more >>>> complex but this also raises a few questions. >>>> * When we allocate from the dictionary we search for a block which is >>>> at least as large as we ask for and split it, returning whatever was >>>> left back to the free list. We don't search for "large enough" blocks >>>> from the small blocks manager, is that intentional or just to keep >>>> the code simple (I like simple)? >>> >>> I'm glad you asked about this so I could give background. It turns >>> out that we deallocate metaspace items better this way. I had a >>> version that did exactly what you said. It was a simple sorted linked >>> list of returned blocks < min_dictionary_size (12) where get_block >>> returned the first block where the item would fit. It had some best >>> fit algorithm so if the block returned was a lot bigger, it wouldn't >>> pick it. >>> >>> My implementation could get through 69300 retransformations before the >>> list didn't work anymore (too many small block fragments of the wrong >>> size) and metaspace was exhausted (metaspace was limited to 12M in >>> this test). Jon's implementation ran this test indefinitely. So it's >>> somewhat simple but it worked really well. >>> >>>> >>>> In the last part of get_block where we return the unused part of a >>>> block retrieved from the dictionary uses compares with >>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>> size for blocks in the dictionary, not the min size for blocks to be >>>> reusable. >>>> I think this code can just call return_block unconditionally for any >>>> nonzero value of "unused" and let return_block deal with dark matter. >>>> >>> yes, I'll make that change. >> >> This change has to be conditional because I assert that >> BlockFreelist::return_block() is never called for < >> small_block_min_size. > > Ok. > > > There's one more thing, though. > In SpaceManager::retire_current_chunk the last piece of the current > allocation chunk is made reusable through a call to > SpaceManager::deallocate. There are two problems with this: > > * retire_current_chunk only calls deallocate for wasted blocks larger > than the min dictionary size, this means that wasted blocks which > would fit in SmallBlocks will be wasted instead of reusable. Yeah. I didn't know if this was worth doing for 12 words or less. Do you? > * the remaining_words parameter which retire_current_chunk passes to > deallocate is _exact_ and must not be aligned up through > get_allocation_word_size. > Since it's already greater than the allocation_word_size, this isn't a problem. > > Also, > are you sure about logging the output of ClassLoaderData::dump on the > info level? Perhaps logging the cld dump on the debug level is a good > trade-off? It looks like the trace level causes the VSM to print the > entire free lists to the log so that's probably a bit much. It is a lot of stuff I didn't want to see. I could change it back to trace though because the thing I really wanted to see in MetaspaceAux::dump() is guarded by trace if (log_is_enabled(Trace, gc, metaspace, freelist)) { if (block_freelists() != NULL) block_freelists()->print_on(out); } It seems like debug would be a better level though for both. > > >> >> Thanks, >> Coleen >> >>>> >>>> >>>> Are the changes to Method and ConstMethod the "bug fix to small >>>> object double free"? >>> >>> Yes. >>>> Is the problem that when a method was freed its annotations were >>>> deallocated as well? Could the annotations fields in the "old" >>>> ConstMethod be cleared instead so that the old annotation arrays >>>> could be kept or is that just needless complexity? >>> >>> I'd rather copy the annotations, because I don't know how the old >>> method, which could still be running, might use the annotations. I >>> don't want to mess with that but I see your point. > > That's fine I think, I wouldn't want to mess around with it either. I > just wanted to understand the reasoning behind the change, thanks for > confirming my suspicion. > thanks, Coleen > /Mikael > >>> >>> Coleen >>> >>>> >>>> Thanks >>>> /Mikael >>>> >>>> >>>> >>>>> >>>>> Thanks, >>>>> Coleen and Jon >>> >> From michail.chernov at oracle.com Fri Oct 7 16:05:24 2016 From: michail.chernov at oracle.com (Michail Chernov) Date: Fri, 7 Oct 2016 19:05:24 +0300 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: <6ee40cf5-15f5-d89e-7653-bcec9cfe76ef@oracle.com> Hi Marcus, Thank you for fixing this. Could you please change @requires in the test? It should be * @requires vm.gc.G1 instead of * @requires vm.gc=="G1" | vm.gc=="null" New form of tag is applicable for platforms where G1 are not supported and allows to exclude this test on those platforms. Older version of tag can cause to the test failure. Thanks, Michail On 10/07/2016 05:26 PM, Marcus Larsson wrote: > Hi, > > Making another attempt to fix this issue. > > Summary: > The following patch resolves a problem where the VM would crash during > shutdown due to static log related memory being de-initialized before > the last use of the logging framework. The solution involves parts of > the Nifty Counter idiom [0] to control static initialization and > de-initialization of stdout's and stderr's LogOutputs. Both objects > are now allocated using placement new, and avoids destructor calls > during de-initialization. The LogStreamInitializer makes sure both > objects are initialized before first use. > > Because the LogOutput::Stdout/err pointers could no longer be kept in > LogOutput, I've replaced all usages of them with the new references > instead. > > The patch includes a regression test for this issue, contributed by > Michail Chernov. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8146009 > > Testing: > JPRT testset hotspot, included test on supported platforms. > > Thanks, > Marcus > > [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter From serguei.spitsyn at oracle.com Fri Oct 7 17:05:55 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Oct 2016 10:05:55 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <60cae5fe-4db4-9d3d-d114-9b6d78ff35aa@oracle.com> Message-ID: On 10/7/16 05:34, Coleen Phillimore wrote: > > > On 10/7/16 8:15 AM, serguei.spitsyn at oracle.com wrote: >> On 10/7/16 05:02, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> It looks good to me. >>> Some minor comments. >>> >>> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html >>> >>> >>> 253 const static uint _small_block_max_size = >>> sizeof(TreeChunk >)/HeapWordSize; 260 >>> assert(word_size >= _small_block_min_size, "There are no metaspace >>> objects less than %u words", _small_block_min_size); >>> Extra space before FreeList and after %u. >>> 903 // Try small_blocks first >>> 904 if (word_size < SmallBlocks::small_block_max_size()) { >>> 905 // Don't create small_blocks() until needed. >>> It makes sense to combine both comments. >>> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests >> >> One small question. >> >> 27 * @summary Test that type annotations are retained after a >> retransform >> . . . >> 49 static class Tester {} >> >> >> Is it a correct summary? I do not see type annotations in the >> class Tester that is being retransformed. Probably, I'm missing >> something. Thanks, Serguei > > And I took out a module (asm) that wasn't needed also. Ok, thanks. I was puzzled with this too, but forgot to ask a question. :) Thanks, Serguei > thanks, > Coleen > >>> /RedefineLeak.java.html >>> 38 import java.lang.NoSuchFieldException; >>> 39 import java.lang.NoSuchMethodException; >>> These imports can be removed. Thanks, Serguei On 10/6/16 10:54, >>> Coleen Phillimore wrote: >>>> Here is an update to the Metadata leak change. There was a bug >>>> introduced when cleaning this up, which Mikael also found. open >>>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >>>> Changes include adding logging for report_metadata_oome, which >>>> necessitated removing ResourceMarks in ClassLoaderData::dump >>>> because the stream passed in already required a ResourceMark, so it >>>> got a nested ResourceMark message for the stream. I changed logging >>>> for tracing block allocations to log_trace(gc, metaspace, freelist, >>>> blocks). In BlockFreelist::get_block and >>>> BlockFreelist::return_block() assert that it's not called for a >>>> size smaller than the minimum allocation (which was the bug). >>>> Renamed get_raw_word_size() to get_allocation_word_size(). This >>>> rounds up to the minimum allocation size which is the same as >>>> small_block_min_size. Also, I added a test that takes a long time >>>> to execute to verify this, and excluded it from JPRT. I could skip >>>> adding this test if people don't want it. Also, the test verifies >>>> that continuously redefining the same class gets memory for the new >>>> class that was released because the block sizes are the same. >>>> When the test exits, it gets a metaspace OOM because loading new >>>> classes and allocating metadata can't use the blocks returned >>>> (wrong size). There is still fragmentation in this implementation, >>>> but it's better that things deallocated < 12 words are actually >>>> freed. I'll file an RFE to work on a perfect algorithm, or to >>>> investigate finding a better one, although I consider this a stress >>>> test that uses all of metaspace to MaxMetaspaceSize, leaving >>>> allocation only to the block fragments left. This isn't a typical >>>> use case. Some comments and corrections to my responses to Mikael >>>> below: On 10/4/16 12:15 PM, Coleen Phillimore wrote: >>>>> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >>>>> Mikael Gerdin wrote: >>>>>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>>>> Summary: Return Metablocks smaller than dictionary's dark >>>>>>> matter. This change contributed by Jon Masamitsu and myself. To >>>>>>> reclaim "dark matter" this change adds an array of small blocks >>>>>>> by size, created lazily, to return Metablocks smaller than the >>>>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>>>> fixed a bug in small object double free and adds debugging code >>>>>>> to check for this case. With this change, the submitted test >>>>>>> case runs indefinitely. Also passed rbt tier 1-5 testing. open >>>>>>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >>>>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>>>> returning its FreeLists by reference, could you change it to have >>>>>> * return_chunk() * get_chunk() * num_chunks(word_size) and get >>>>>> rid of list_at? >>>>> Okay, I refactored this so small_chunks()->get_block() and >>>>> return_block() are used rather than getting list_at. I didn't see >>>>> where you got num_chunks, but list_at is hidden. >>>>>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>>>>> see that it's debug only but I fear that it could case problems >>>>>> with running out of native memory in our internal testing of >>>>>> debug builds. >>>>> I thought about taking it out, but it helped me find the double >>>>> free bug. I think if we add new code to metadata and have to >>>>> call deallocate_contents on it, we risk re-introducting these >>>>> double free bugs. I could take it out. I don't think this gets >>>>> that big but I'd hate to introduce some sort of OOM bug in our >>>>> testing. >>>>>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>>>>> the minimum size of blocks to be put on the BlockTreeDictionary, >>>>>> not the minimum size of blocks which are reusable. >>>>> How about min_dictionary_size() ? >>>>>> Is there any particular reason behind _small_blocks being lazy >>>>>> allocated and _dictionary not? >>>>> We lazily create the BlockFreelists with this change. // Lazily >>>>> create a block_freelist if (block_freelists() == NULL) { >>>>> _block_freelists = new BlockFreelist(); } So the small_blocks >>>>> are lazily created in the BlockFreelists but the dictionary is >>>>> not. I guess if we're going to create the BlockFreelists here, >>>>> we'll most likely need both and maybe small_blocks need not be >>>>> lazily created. Was that your suggestion? My concern with this >>>>> change was all the space used by the small_blocks() but if we're >>>>> doing any deallocation within the metaspace, at least one of the >>>>> things will be <12 words. I'll make small_blocks() not be lazily >>>>> allocated since BlockFreelist are. These are pretty expensive, >>>>> but should be limited to a few metaspaces. >>>> Class SpaceManager doesn't need small_blocks() so I left >>>> small_blocks as lazily allocated. >>>>>> I would prefer if BlockFreelist::return_block would perform the >>>>>> checks in reverse order instead of having a return inside the >>>>>> first if block, something like if (word_size > >>>>>> small_block_max_size) { dict()->return_chunk(c) } else if >>>>>> (word_size > small_block_min_size) { >>>>>> small_blocks()->return_chunk(c) } else { // dark matter } >>>>> Why? We don't want to cast Metablock into dark matter so check if >>>>> word_size < small_block_min_size first. Metablock* free_chunk = >>>>> ::new (p) Metablock(word_size); if (word_size < >>>>> SmallBlocks::small_block_max_size()) { >>>>> small_blocks()->return_chunk(word_size); } else { >>>>> dictionary()->return_chunk(free_chunk); } >>>> There is no dark matter in these functions anymore. >>>>>> For BlockFreelist::get_block I realize that the code is a bit >>>>>> more complex but this also raises a few questions. * When we >>>>>> allocate from the dictionary we search for a block which is at >>>>>> least as large as we ask for and split it, returning whatever was >>>>>> left back to the free list. We don't search for "large enough" >>>>>> blocks from the small blocks manager, is that intentional or just >>>>>> to keep the code simple (I like simple)? >>>>> I'm glad you asked about this so I could give background. It turns >>>>> out that we deallocate metaspace items better this way. I had a >>>>> version that did exactly what you said. It was a simple sorted >>>>> linked list of returned blocks < min_dictionary_size (12) where >>>>> get_block returned the first block where the item would fit. It >>>>> had some best fit algorithm so if the block returned was a lot >>>>> bigger, it wouldn't pick it. My implementation could get through >>>>> 69300 retransformations before the list didn't work anymore (too >>>>> many small block fragments of the wrong size) and metaspace was >>>>> exhausted (metaspace was limited to 12M in this test). Jon's >>>>> implementation ran this test indefinitely. So it's somewhat >>>>> simple but it worked really well. >>>>>> In the last part of get_block where we return the unused part of >>>>>> a block retrieved from the dictionary uses compares with >>>>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>>>> size for blocks in the dictionary, not the min size for blocks to >>>>>> be reusable. I think this code can just call return_block >>>>>> unconditionally for any nonzero value of "unused" and let >>>>>> return_block deal with dark matter. >>>>> yes, I'll make that change. >>>> This change has to be conditional because I assert that >>>> BlockFreelist::return_block() is never called for < >>>> small_block_min_size. Thanks, Coleen >>>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>>> object double free"? >>>>> Yes. >>>>>> Is the problem that when a method was freed its annotations were >>>>>> deallocated as well? Could the annotations fields in the "old" >>>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>>> could be kept or is that just needless complexity? >>>>> I'd rather copy the annotations, because I don't know how the old >>>>> method, which could still be running, might use the annotations. I >>>>> don't want to mess with that but I see your point. Coleen >>>>>> Thanks /Mikael >>>>>>> Thanks, Coleen and Jon >> > From serguei.spitsyn at oracle.com Fri Oct 7 17:06:33 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Oct 2016 10:06:33 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <60cae5fe-4db4-9d3d-d114-9b6d78ff35aa@oracle.com> Message-ID: <545b613b-3351-5d5b-181d-7c5cf31b9363@oracle.com> On 10/7/16 05:33, Coleen Phillimore wrote: > > > On 10/7/16 8:15 AM, serguei.spitsyn at oracle.com wrote: >> On 10/7/16 05:02, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> It looks good to me. >>> Some minor comments. >>> >>> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html >>> >>> >>> 253 const static uint _small_block_max_size = >>> sizeof(TreeChunk >)/HeapWordSize; 260 >>> assert(word_size >= _small_block_min_size, "There are no metaspace >>> objects less than %u words", _small_block_min_size); >>> Extra space before FreeList and after %u. >>> 903 // Try small_blocks first >>> 904 if (word_size < SmallBlocks::small_block_max_size()) { >>> 905 // Don't create small_blocks() until needed. >>> It makes sense to combine both comments. >>> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests >> >> One small question. >> >> 27 * @summary Test that type annotations are retained after a >> retransform >> . . . >> 49 static class Tester {} >> >> >> Is it a correct summary? I do not see type annotations in the >> class Tester that is being retransformed. Probably, I'm missing >> something. Thanks, Serguei > > I copied that from the other test too. I thought I'd fixed the > summary. I'll fix it now to: > > @summary Test that redefinition reuses metaspace blocks that are freed Looks good. Thanks! Serguei > > Thanks, > Coleen > >>> /RedefineLeak.java.html >>> 38 import java.lang.NoSuchFieldException; >>> 39 import java.lang.NoSuchMethodException; >>> These imports can be removed. Thanks, Serguei On 10/6/16 10:54, >>> Coleen Phillimore wrote: >>>> Here is an update to the Metadata leak change. There was a bug >>>> introduced when cleaning this up, which Mikael also found. open >>>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >>>> Changes include adding logging for report_metadata_oome, which >>>> necessitated removing ResourceMarks in ClassLoaderData::dump >>>> because the stream passed in already required a ResourceMark, so it >>>> got a nested ResourceMark message for the stream. I changed logging >>>> for tracing block allocations to log_trace(gc, metaspace, freelist, >>>> blocks). In BlockFreelist::get_block and >>>> BlockFreelist::return_block() assert that it's not called for a >>>> size smaller than the minimum allocation (which was the bug). >>>> Renamed get_raw_word_size() to get_allocation_word_size(). This >>>> rounds up to the minimum allocation size which is the same as >>>> small_block_min_size. Also, I added a test that takes a long time >>>> to execute to verify this, and excluded it from JPRT. I could skip >>>> adding this test if people don't want it. Also, the test verifies >>>> that continuously redefining the same class gets memory for the new >>>> class that was released because the block sizes are the same. >>>> When the test exits, it gets a metaspace OOM because loading new >>>> classes and allocating metadata can't use the blocks returned >>>> (wrong size). There is still fragmentation in this implementation, >>>> but it's better that things deallocated < 12 words are actually >>>> freed. I'll file an RFE to work on a perfect algorithm, or to >>>> investigate finding a better one, although I consider this a stress >>>> test that uses all of metaspace to MaxMetaspaceSize, leaving >>>> allocation only to the block fragments left. This isn't a typical >>>> use case. Some comments and corrections to my responses to Mikael >>>> below: On 10/4/16 12:15 PM, Coleen Phillimore wrote: >>>>> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >>>>> Mikael Gerdin wrote: >>>>>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>>>> Summary: Return Metablocks smaller than dictionary's dark >>>>>>> matter. This change contributed by Jon Masamitsu and myself. To >>>>>>> reclaim "dark matter" this change adds an array of small blocks >>>>>>> by size, created lazily, to return Metablocks smaller than the >>>>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>>>> fixed a bug in small object double free and adds debugging code >>>>>>> to check for this case. With this change, the submitted test >>>>>>> case runs indefinitely. Also passed rbt tier 1-5 testing. open >>>>>>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >>>>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>>>> returning its FreeLists by reference, could you change it to have >>>>>> * return_chunk() * get_chunk() * num_chunks(word_size) and get >>>>>> rid of list_at? >>>>> Okay, I refactored this so small_chunks()->get_block() and >>>>> return_block() are used rather than getting list_at. I didn't see >>>>> where you got num_chunks, but list_at is hidden. >>>>>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>>>>> see that it's debug only but I fear that it could case problems >>>>>> with running out of native memory in our internal testing of >>>>>> debug builds. >>>>> I thought about taking it out, but it helped me find the double >>>>> free bug. I think if we add new code to metadata and have to >>>>> call deallocate_contents on it, we risk re-introducting these >>>>> double free bugs. I could take it out. I don't think this gets >>>>> that big but I'd hate to introduce some sort of OOM bug in our >>>>> testing. >>>>>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>>>>> the minimum size of blocks to be put on the BlockTreeDictionary, >>>>>> not the minimum size of blocks which are reusable. >>>>> How about min_dictionary_size() ? >>>>>> Is there any particular reason behind _small_blocks being lazy >>>>>> allocated and _dictionary not? >>>>> We lazily create the BlockFreelists with this change. // Lazily >>>>> create a block_freelist if (block_freelists() == NULL) { >>>>> _block_freelists = new BlockFreelist(); } So the small_blocks >>>>> are lazily created in the BlockFreelists but the dictionary is >>>>> not. I guess if we're going to create the BlockFreelists here, >>>>> we'll most likely need both and maybe small_blocks need not be >>>>> lazily created. Was that your suggestion? My concern with this >>>>> change was all the space used by the small_blocks() but if we're >>>>> doing any deallocation within the metaspace, at least one of the >>>>> things will be <12 words. I'll make small_blocks() not be lazily >>>>> allocated since BlockFreelist are. These are pretty expensive, >>>>> but should be limited to a few metaspaces. >>>> Class SpaceManager doesn't need small_blocks() so I left >>>> small_blocks as lazily allocated. >>>>>> I would prefer if BlockFreelist::return_block would perform the >>>>>> checks in reverse order instead of having a return inside the >>>>>> first if block, something like if (word_size > >>>>>> small_block_max_size) { dict()->return_chunk(c) } else if >>>>>> (word_size > small_block_min_size) { >>>>>> small_blocks()->return_chunk(c) } else { // dark matter } >>>>> Why? We don't want to cast Metablock into dark matter so check if >>>>> word_size < small_block_min_size first. Metablock* free_chunk = >>>>> ::new (p) Metablock(word_size); if (word_size < >>>>> SmallBlocks::small_block_max_size()) { >>>>> small_blocks()->return_chunk(word_size); } else { >>>>> dictionary()->return_chunk(free_chunk); } >>>> There is no dark matter in these functions anymore. >>>>>> For BlockFreelist::get_block I realize that the code is a bit >>>>>> more complex but this also raises a few questions. * When we >>>>>> allocate from the dictionary we search for a block which is at >>>>>> least as large as we ask for and split it, returning whatever was >>>>>> left back to the free list. We don't search for "large enough" >>>>>> blocks from the small blocks manager, is that intentional or just >>>>>> to keep the code simple (I like simple)? >>>>> I'm glad you asked about this so I could give background. It turns >>>>> out that we deallocate metaspace items better this way. I had a >>>>> version that did exactly what you said. It was a simple sorted >>>>> linked list of returned blocks < min_dictionary_size (12) where >>>>> get_block returned the first block where the item would fit. It >>>>> had some best fit algorithm so if the block returned was a lot >>>>> bigger, it wouldn't pick it. My implementation could get through >>>>> 69300 retransformations before the list didn't work anymore (too >>>>> many small block fragments of the wrong size) and metaspace was >>>>> exhausted (metaspace was limited to 12M in this test). Jon's >>>>> implementation ran this test indefinitely. So it's somewhat >>>>> simple but it worked really well. >>>>>> In the last part of get_block where we return the unused part of >>>>>> a block retrieved from the dictionary uses compares with >>>>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>>>> size for blocks in the dictionary, not the min size for blocks to >>>>>> be reusable. I think this code can just call return_block >>>>>> unconditionally for any nonzero value of "unused" and let >>>>>> return_block deal with dark matter. >>>>> yes, I'll make that change. >>>> This change has to be conditional because I assert that >>>> BlockFreelist::return_block() is never called for < >>>> small_block_min_size. Thanks, Coleen >>>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>>> object double free"? >>>>> Yes. >>>>>> Is the problem that when a method was freed its annotations were >>>>>> deallocated as well? Could the annotations fields in the "old" >>>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>>> could be kept or is that just needless complexity? >>>>> I'd rather copy the annotations, because I don't know how the old >>>>> method, which could still be running, might use the annotations. I >>>>> don't want to mess with that but I see your point. Coleen >>>>>> Thanks /Mikael >>>>>>> Thanks, Coleen and Jon >> > From serguei.spitsyn at oracle.com Fri Oct 7 17:10:42 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 7 Oct 2016 10:10:42 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <41975ac2-41fe-42ac-457e-3d0f329dd1da@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <41975ac2-41fe-42ac-457e-3d0f329dd1da@oracle.com> Message-ID: On 10/7/16 05:32, Coleen Phillimore wrote: > > > On 10/7/16 8:02 AM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> It looks good to me. >> Some minor comments. >> >> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html >> >> 253 const static uint _small_block_max_size = >> sizeof(TreeChunk >)/HeapWordSize; 260 >> assert(word_size >= _small_block_min_size, "There are no metaspace >> objects less than %u words", _small_block_min_size); >> Extra space before FreeList and after %u. > Fixed. >> 903 // Try small_blocks first >> 904 if (word_size < SmallBlocks::small_block_max_size()) { >> 905 // Don't create small_blocks() until needed. >> It makes sense to combine both comments. > The first comment applies to like 904 and the second applies to 906. > I could add to 905, that calling small_blocks() allocates the small > block lists. It was really minor comment. It is up to you. >> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests/RedefineLeak.java.html >> >> 38 import java.lang.NoSuchFieldException; >> 39 import java.lang.NoSuchMethodException; > You're right - copied from another test. Thanks! Coleen Thanks for update! Serguei >> These imports can be removed. Thanks, Serguei On 10/6/16 10:54, >> Coleen Phillimore wrote: >>> Here is an update to the Metadata leak change. There was a bug >>> introduced when cleaning this up, which Mikael also found. open >>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >>> Changes include adding logging for report_metadata_oome, which >>> necessitated removing ResourceMarks in ClassLoaderData::dump because >>> the stream passed in already required a ResourceMark, so it got a >>> nested ResourceMark message for the stream. I changed logging for >>> tracing block allocations to log_trace(gc, metaspace, freelist, >>> blocks). In BlockFreelist::get_block and >>> BlockFreelist::return_block() assert that it's not called for a size >>> smaller than the minimum allocation (which was the bug). Renamed >>> get_raw_word_size() to get_allocation_word_size(). This rounds up >>> to the minimum allocation size which is the same as >>> small_block_min_size. Also, I added a test that takes a long time to >>> execute to verify this, and excluded it from JPRT. I could skip >>> adding this test if people don't want it. Also, the test verifies >>> that continuously redefining the same class gets memory for the new >>> class that was released because the block sizes are the same. When >>> the test exits, it gets a metaspace OOM because loading new classes >>> and allocating metadata can't use the blocks returned (wrong size). >>> There is still fragmentation in this implementation, but it's better >>> that things deallocated < 12 words are actually freed. I'll file an >>> RFE to work on a perfect algorithm, or to investigate finding a >>> better one, although I consider this a stress test that uses all of >>> metaspace to MaxMetaspaceSize, leaving allocation only to the block >>> fragments left. This isn't a typical use case. Some comments and >>> corrections to my responses to Mikael below: On 10/4/16 12:15 PM, >>> Coleen Phillimore wrote: >>>> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >>>> Mikael Gerdin wrote: >>>>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>>>> "dark matter" this change adds an array of small blocks by size, >>>>>> created lazily, to return Metablocks smaller than the >>>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>>> fixed a bug in small object double free and adds debugging code >>>>>> to check for this case. With this change, the submitted test case >>>>>> runs indefinitely. Also passed rbt tier 1-5 testing. open webrev >>>>>> at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link >>>>>> https://bugs.openjdk.java.net/browse/JDK-8164921 >>>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>>> returning its FreeLists by reference, could you change it to have >>>>> * return_chunk() * get_chunk() * num_chunks(word_size) and get rid >>>>> of list_at? >>>> Okay, I refactored this so small_chunks()->get_block() and >>>> return_block() are used rather than getting list_at. I didn't see >>>> where you got num_chunks, but list_at is hidden. >>>>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>>>> see that it's debug only but I fear that it could case problems >>>>> with running out of native memory in our internal testing of debug >>>>> builds. >>>> I thought about taking it out, but it helped me find the double >>>> free bug. I think if we add new code to metadata and have to call >>>> deallocate_contents on it, we risk re-introducting these double >>>> free bugs. I could take it out. I don't think this gets that big >>>> but I'd hate to introduce some sort of OOM bug in our testing. >>>>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>>>> the minimum size of blocks to be put on the BlockTreeDictionary, >>>>> not the minimum size of blocks which are reusable. >>>> How about min_dictionary_size() ? >>>>> Is there any particular reason behind _small_blocks being lazy >>>>> allocated and _dictionary not? >>>> We lazily create the BlockFreelists with this change. // Lazily >>>> create a block_freelist if (block_freelists() == NULL) { >>>> _block_freelists = new BlockFreelist(); } So the small_blocks are >>>> lazily created in the BlockFreelists but the dictionary is not. I >>>> guess if we're going to create the BlockFreelists here, we'll most >>>> likely need both and maybe small_blocks need not be lazily >>>> created. Was that your suggestion? My concern with this change was >>>> all the space used by the small_blocks() but if we're doing any >>>> deallocation within the metaspace, at least one of the things will >>>> be <12 words. I'll make small_blocks() not be lazily allocated >>>> since BlockFreelist are. These are pretty expensive, but should be >>>> limited to a few metaspaces. >>> Class SpaceManager doesn't need small_blocks() so I left >>> small_blocks as lazily allocated. >>>>> I would prefer if BlockFreelist::return_block would perform the >>>>> checks in reverse order instead of having a return inside the >>>>> first if block, something like if (word_size > >>>>> small_block_max_size) { dict()->return_chunk(c) } else if >>>>> (word_size > small_block_min_size) { >>>>> small_blocks()->return_chunk(c) } else { // dark matter } >>>> Why? We don't want to cast Metablock into dark matter so check if >>>> word_size < small_block_min_size first. Metablock* free_chunk = >>>> ::new (p) Metablock(word_size); if (word_size < >>>> SmallBlocks::small_block_max_size()) { >>>> small_blocks()->return_chunk(word_size); } else { >>>> dictionary()->return_chunk(free_chunk); } >>> There is no dark matter in these functions anymore. >>>>> For BlockFreelist::get_block I realize that the code is a bit more >>>>> complex but this also raises a few questions. * When we allocate >>>>> from the dictionary we search for a block which is at least as >>>>> large as we ask for and split it, returning whatever was left back >>>>> to the free list. We don't search for "large enough" blocks from >>>>> the small blocks manager, is that intentional or just to keep the >>>>> code simple (I like simple)? >>>> I'm glad you asked about this so I could give background. It turns >>>> out that we deallocate metaspace items better this way. I had a >>>> version that did exactly what you said. It was a simple sorted >>>> linked list of returned blocks < min_dictionary_size (12) where >>>> get_block returned the first block where the item would fit. It >>>> had some best fit algorithm so if the block returned was a lot >>>> bigger, it wouldn't pick it. My implementation could get through >>>> 69300 retransformations before the list didn't work anymore (too >>>> many small block fragments of the wrong size) and metaspace was >>>> exhausted (metaspace was limited to 12M in this test). Jon's >>>> implementation ran this test indefinitely. So it's somewhat simple >>>> but it worked really well. >>>>> In the last part of get_block where we return the unused part of a >>>>> block retrieved from the dictionary uses compares with >>>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>>> size for blocks in the dictionary, not the min size for blocks to >>>>> be reusable. I think this code can just call return_block >>>>> unconditionally for any nonzero value of "unused" and let >>>>> return_block deal with dark matter. >>>> yes, I'll make that change. >>> This change has to be conditional because I assert that >>> BlockFreelist::return_block() is never called for < >>> small_block_min_size. Thanks, Coleen >>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>> object double free"? >>>> Yes. >>>>> Is the problem that when a method was freed its annotations were >>>>> deallocated as well? Could the annotations fields in the "old" >>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>> could be kept or is that just needless complexity? >>>> I'd rather copy the annotations, because I don't know how the old >>>> method, which could still be running, might use the annotations. I >>>> don't want to mess with that but I see your point. Coleen >>>>> Thanks /Mikael >>>>>> Thanks, Coleen and Jon From mikael.vidstedt at oracle.com Fri Oct 7 17:21:51 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Fri, 7 Oct 2016 10:21:51 -0700 Subject: Merging jdk9/hs-comp with jdk9/hs In-Reply-To: References: Message-ID: <0C83967C-3900-4A2A-88F3-A4D956AB6049@oracle.com> Unfortunately the test results didn?t materialize in time, so we will have to postpone the consolidation. Until proven otherwise we?ll try again Friday next week - same bat-time, same bat-channel. Cheers, Mikael > On Oct 5, 2016, at 2:37 PM, Mikael Vidstedt wrote: > > > Having heard no objections, we?re looking to move forward with the consolidation. > > We of course want to make sure that what?s currently in hs-comp is ready for (the last) integration, and due to some technical difficulties we?re still waiting for the relevant test results. We?re still hoping to have everything ready to go on Friday, but if not we may have to postpone the merge. > > Cheers, > Mikael > >> On Sep 29, 2016, at 11:23 AM, Mikael Vidstedt wrote: >> >> >> All, >> >> Over the last two years we have worked towards simplifying the mercurial forest structure for Hotspot development by consolidating and removing forests. Last year we moved[1] the GC development work from jdk9/hs-gc[2] to jdk9/hs-rt[3], and earlier this year we further consolidated forests by moving[4] the jdk9/hs-rt work directly to the jdk9/hs[5] - aka. "hs main" - forest. The reduction in forests has led to less work spent on integration, faster quality feedback, more efficient hardware utilization, etc. All in all, the experience so far has been a good one. >> >> However, the Hotspot (JIT) compiler changes are still integrated through the jdk9/hs-comp[6] forest, and are tested separately before they are (bulk) integrated up to jdk9/hs. The separate forest of course comes with the expected integration/gatekeeper overhead and lead times. Since JDK 9 development is ramping down, we expect the number of changes to ramp down as well. This means that the benefit of having a separate jdk9/hs-comp forest is going to reduce over time. Now seems like a good time to look at doing a final hotspot forest consolidation. >> >> In line with this, we propose closing down jdk9/hs-comp and doing all remaining JDK 9 Hotspot development to jdk9/hs. We propose that this final forest consolidation to take place on Friday, October 7th. Much like the earlier forest consolidations the jdk9/hs-comp forest will be kept around, but will be locked down so that no accidental integrations are made to it. Any work in flight based on jdk9/hs-comp would have to be rebased on jdk9/hs. >> >> The earlier forest consolidations have gone very smoothly, and we expect this one to go smoothly as well, but if for some reason things don't work out we will of course have an option to go back to what we have today. That said, before using the escape hatch and reverting back, we will of course want to try to address any issues - thank you in advance for your patience while we work through any such issues. >> >> Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Wednesday, 5 October 2016, we will go ahead with the forest consolidation. >> >> Cheers, >> Mikael >> >> [1]http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/thread.html >> [2]http://hg.openjdk.java.net/jdk9/hs-gc >> [3]http://hg.openjdk.java.net/jdk9/hs-rt >> [4]http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-March/022218.html >> [5]http://hg.openjdk.java.net/jdk9/hs >> [6]http://hg.openjdk.java.net/jdk9/hs-comp >> > From coleen.phillimore at oracle.com Fri Oct 7 18:08:39 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 7 Oct 2016 14:08:39 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <41975ac2-41fe-42ac-457e-3d0f329dd1da@oracle.com> Message-ID: <3ff0e268-6ac3-8fac-93f5-d4c765acd4ba@oracle.com> Serguei, Thank you for the review! Coleen On 10/7/16 1:10 PM, serguei.spitsyn at oracle.com wrote: > On 10/7/16 05:32, Coleen Phillimore wrote: >> >> >> On 10/7/16 8:02 AM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> It looks good to me. >>> Some minor comments. >>> >>> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/src/share/vm/memory/metaspace.cpp.frames.html >>> >>> >>> 253 const static uint _small_block_max_size = >>> sizeof(TreeChunk >)/HeapWordSize; 260 >>> assert(word_size >= _small_block_min_size, "There are no metaspace >>> objects less than %u words", _small_block_min_size); >>> Extra space before FreeList and after %u. >> Fixed. >>> 903 // Try small_blocks first >>> 904 if (word_size < SmallBlocks::small_block_max_size()) { >>> 905 // Don't create small_blocks() until needed. >>> It makes sense to combine both comments. >> The first comment applies to like 904 and the second applies to >> 906. I could add to 905, that calling small_blocks() allocates the >> small block lists. > > It was really minor comment. > It is up to you. > > >>> http://cr.openjdk.java.net/~coleenp/8164921.02/webrev/test/runtime/RedefineTests/RedefineLeak.java.html >>> >>> 38 import java.lang.NoSuchFieldException; >>> 39 import java.lang.NoSuchMethodException; >> You're right - copied from another test. Thanks! Coleen > > Thanks for update! > Serguei > > >>> These imports can be removed. Thanks, Serguei On 10/6/16 10:54, >>> Coleen Phillimore wrote: >>>> Here is an update to the Metadata leak change. There was a bug >>>> introduced when cleaning this up, which Mikael also found. open >>>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >>>> Changes include adding logging for report_metadata_oome, which >>>> necessitated removing ResourceMarks in ClassLoaderData::dump >>>> because the stream passed in already required a ResourceMark, so it >>>> got a nested ResourceMark message for the stream. I changed logging >>>> for tracing block allocations to log_trace(gc, metaspace, freelist, >>>> blocks). In BlockFreelist::get_block and >>>> BlockFreelist::return_block() assert that it's not called for a >>>> size smaller than the minimum allocation (which was the bug). >>>> Renamed get_raw_word_size() to get_allocation_word_size(). This >>>> rounds up to the minimum allocation size which is the same as >>>> small_block_min_size. Also, I added a test that takes a long time >>>> to execute to verify this, and excluded it from JPRT. I could skip >>>> adding this test if people don't want it. Also, the test verifies >>>> that continuously redefining the same class gets memory for the new >>>> class that was released because the block sizes are the same. >>>> When the test exits, it gets a metaspace OOM because loading new >>>> classes and allocating metadata can't use the blocks returned >>>> (wrong size). There is still fragmentation in this implementation, >>>> but it's better that things deallocated < 12 words are actually >>>> freed. I'll file an RFE to work on a perfect algorithm, or to >>>> investigate finding a better one, although I consider this a stress >>>> test that uses all of metaspace to MaxMetaspaceSize, leaving >>>> allocation only to the block fragments left. This isn't a typical >>>> use case. Some comments and corrections to my responses to Mikael >>>> below: On 10/4/16 12:15 PM, Coleen Phillimore wrote: >>>>> Hi Mikael, Thanks for looking at this change. On 10/4/16 8:32 AM, >>>>> Mikael Gerdin wrote: >>>>>> Hi Coleen, On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>>>> Summary: Return Metablocks smaller than dictionary's dark >>>>>>> matter. This change contributed by Jon Masamitsu and myself. To >>>>>>> reclaim "dark matter" this change adds an array of small blocks >>>>>>> by size, created lazily, to return Metablocks smaller than the >>>>>>> BinaryTreeDictionary entry's minimum size. This change also >>>>>>> fixed a bug in small object double free and adds debugging code >>>>>>> to check for this case. With this change, the submitted test >>>>>>> case runs indefinitely. Also passed rbt tier 1-5 testing. open >>>>>>> webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >>>>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>>>> returning its FreeLists by reference, could you change it to have >>>>>> * return_chunk() * get_chunk() * num_chunks(word_size) and get >>>>>> rid of list_at? >>>>> Okay, I refactored this so small_chunks()->get_block() and >>>>> return_block() are used rather than getting list_at. I didn't see >>>>> where you got num_chunks, but list_at is hidden. >>>>>> -- For how long do you plan to keep BlockFreelist::_all_blocks? I >>>>>> see that it's debug only but I fear that it could case problems >>>>>> with running out of native memory in our internal testing of >>>>>> debug builds. >>>>> I thought about taking it out, but it helped me find the double >>>>> free bug. I think if we add new code to metadata and have to >>>>> call deallocate_contents on it, we risk re-introducting these >>>>> double free bugs. I could take it out. I don't think this gets >>>>> that big but I'd hate to introduce some sort of OOM bug in our >>>>> testing. >>>>>> BlockFreelist::min_size() is a bit of a misnomer since it returns >>>>>> the minimum size of blocks to be put on the BlockTreeDictionary, >>>>>> not the minimum size of blocks which are reusable. >>>>> How about min_dictionary_size() ? >>>>>> Is there any particular reason behind _small_blocks being lazy >>>>>> allocated and _dictionary not? >>>>> We lazily create the BlockFreelists with this change. // Lazily >>>>> create a block_freelist if (block_freelists() == NULL) { >>>>> _block_freelists = new BlockFreelist(); } So the small_blocks >>>>> are lazily created in the BlockFreelists but the dictionary is >>>>> not. I guess if we're going to create the BlockFreelists here, >>>>> we'll most likely need both and maybe small_blocks need not be >>>>> lazily created. Was that your suggestion? My concern with this >>>>> change was all the space used by the small_blocks() but if we're >>>>> doing any deallocation within the metaspace, at least one of the >>>>> things will be <12 words. I'll make small_blocks() not be lazily >>>>> allocated since BlockFreelist are. These are pretty expensive, >>>>> but should be limited to a few metaspaces. >>>> Class SpaceManager doesn't need small_blocks() so I left >>>> small_blocks as lazily allocated. >>>>>> I would prefer if BlockFreelist::return_block would perform the >>>>>> checks in reverse order instead of having a return inside the >>>>>> first if block, something like if (word_size > >>>>>> small_block_max_size) { dict()->return_chunk(c) } else if >>>>>> (word_size > small_block_min_size) { >>>>>> small_blocks()->return_chunk(c) } else { // dark matter } >>>>> Why? We don't want to cast Metablock into dark matter so check if >>>>> word_size < small_block_min_size first. Metablock* free_chunk = >>>>> ::new (p) Metablock(word_size); if (word_size < >>>>> SmallBlocks::small_block_max_size()) { >>>>> small_blocks()->return_chunk(word_size); } else { >>>>> dictionary()->return_chunk(free_chunk); } >>>> There is no dark matter in these functions anymore. >>>>>> For BlockFreelist::get_block I realize that the code is a bit >>>>>> more complex but this also raises a few questions. * When we >>>>>> allocate from the dictionary we search for a block which is at >>>>>> least as large as we ask for and split it, returning whatever was >>>>>> left back to the free list. We don't search for "large enough" >>>>>> blocks from the small blocks manager, is that intentional or just >>>>>> to keep the code simple (I like simple)? >>>>> I'm glad you asked about this so I could give background. It turns >>>>> out that we deallocate metaspace items better this way. I had a >>>>> version that did exactly what you said. It was a simple sorted >>>>> linked list of returned blocks < min_dictionary_size (12) where >>>>> get_block returned the first block where the item would fit. It >>>>> had some best fit algorithm so if the block returned was a lot >>>>> bigger, it wouldn't pick it. My implementation could get through >>>>> 69300 retransformations before the list didn't work anymore (too >>>>> many small block fragments of the wrong size) and metaspace was >>>>> exhausted (metaspace was limited to 12M in this test). Jon's >>>>> implementation ran this test indefinitely. So it's somewhat >>>>> simple but it worked really well. >>>>>> In the last part of get_block where we return the unused part of >>>>>> a block retrieved from the dictionary uses compares with >>>>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>>>> size for blocks in the dictionary, not the min size for blocks to >>>>>> be reusable. I think this code can just call return_block >>>>>> unconditionally for any nonzero value of "unused" and let >>>>>> return_block deal with dark matter. >>>>> yes, I'll make that change. >>>> This change has to be conditional because I assert that >>>> BlockFreelist::return_block() is never called for < >>>> small_block_min_size. Thanks, Coleen >>>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>>> object double free"? >>>>> Yes. >>>>>> Is the problem that when a method was freed its annotations were >>>>>> deallocated as well? Could the annotations fields in the "old" >>>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>>> could be kept or is that just needless complexity? >>>>> I'd rather copy the annotations, because I don't know how the old >>>>> method, which could still be running, might use the annotations. I >>>>> don't want to mess with that but I see your point. Coleen >>>>>> Thanks /Mikael >>>>>>> Thanks, Coleen and Jon > > From brent.christian at oracle.com Fri Oct 7 18:25:53 2016 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 7 Oct 2016 11:25:53 -0700 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: <508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com> References: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com> <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com> <56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com> <508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com> Message-ID: <582a2109-b3bb-984d-0090-00d19ad14940@oracle.com> On 10/5/16 4:43 PM, David Holmes wrote: >> >>> Okay but this will still affect the lifecycle of the PDs because >>> without the strong reference in L, those weak references in the VM >>> will quickly be cleared. There's also a strong reference held by the Class object itself (on the VM side [1]). Thanks for having a look, folks. I've applied Naoto and Mandy's suggestions for the test case and updated the webrev in place with what I plan to push. http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/ -Brent 1. http://hg.openjdk.java.net/jdk9/dev/hotspot/file/fec31089c2ef/src/share/vm/classfile/javaClasses.cpp#l917 From coleen.phillimore at oracle.com Fri Oct 7 18:32:00 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 7 Oct 2016 14:32:00 -0400 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: <582a2109-b3bb-984d-0090-00d19ad14940@oracle.com> References: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com> <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com> <56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com> <508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com> <582a2109-b3bb-984d-0090-00d19ad14940@oracle.com> Message-ID: This looks good, Brent. Really nice job figuring out the cause of the leak, and thank you for fixing it! Coleen On 10/7/16 2:25 PM, Brent Christian wrote: > On 10/5/16 4:43 PM, David Holmes wrote: >>> >>>> Okay but this will still affect the lifecycle of the PDs because >>>> without the strong reference in L, those weak references in the VM >>>> will quickly be cleared. > > There's also a strong reference held by the Class object itself (on > the VM side [1]). > > Thanks for having a look, folks. I've applied Naoto and Mandy's > suggestions for the test case and updated the webrev in place with > what I plan to push. > > http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/ > > -Brent > > 1. > http://hg.openjdk.java.net/jdk9/dev/hotspot/file/fec31089c2ef/src/share/vm/classfile/javaClasses.cpp#l917 From mandy.chung at oracle.com Fri Oct 7 18:42:11 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 7 Oct 2016 11:42:11 -0700 Subject: RFR 8151486 : Class.forName causes memory leak In-Reply-To: <582a2109-b3bb-984d-0090-00d19ad14940@oracle.com> References: <70a4c247-b306-16d6-8975-2129dd214ad4@oracle.com> <1E19F25C-98A6-49E4-BEDF-07881B25B9D4@oracle.com> <300c0596-59db-b4d8-af01-8cf5e2d0b41a@oracle.com> <56D485BF-0332-4F71-9952-5B2CFE326E8A@oracle.com> <508ffe78-ccd6-ec7d-1675-a148a17d5cb3@oracle.com> <582a2109-b3bb-984d-0090-00d19ad14940@oracle.com> Message-ID: > On Oct 7, 2016, at 11:25 AM, Brent Christian wrote: > > On 10/5/16 4:43 PM, David Holmes wrote: >>> >>>> Okay but this will still affect the lifecycle of the PDs because >>>> without the strong reference in L, those weak references in the VM >>>> will quickly be cleared. > > There's also a strong reference held by the Class object itself (on the VM side [1]). > Yes this is the protection domain of the Class itself (i.e. PD associated with its code source) that was set when the class is defined. > Thanks for having a look, folks. I've applied Naoto and Mandy's suggestions for the test case and updated the webrev in place with what I plan to push. > > http://cr.openjdk.java.net/~bchristi/8151486/webrev.00/ It might be good to refactor System.getProperty("test.classes", ".?) as a static final field. Otherwise looks good. No need for a new webrev. Mandy From sangheon.kim at oracle.com Fri Oct 7 20:48:55 2016 From: sangheon.kim at oracle.com (sangheon) Date: Fri, 7 Oct 2016 13:48:55 -0700 Subject: RFR(S): 8166461: Deprecate UseAutoGCSelectPolicy Message-ID: Hi all, Can I have some reviews for this change to deprecate UseAutoGCSelectPolicy command-line option? The intent of UseAutoGCSelectPolicy option is to provide a way to automatically select GC based on target pause time. This option is introduced with AutoGCSelectPauseMillis and if MaxGCPauseMillis is smaller than or equal to AutoGCSelectPauseMillis(and if GC is not specified), low pause collector will be selected. I think it is enough to accomplish auto-GC-selection without UseAutoGCSelectPolicy. This patch only includes adding the deprecation message for UseAutoGCSelectPolicy option. CR: https://bugs.openjdk.java.net/browse/JDK-8166461 Webrev: http://cr.openjdk.java.net/~sangheki/8166461/webrev.0 Testing: JPRT Thanks, Sangheon From kim.barrett at oracle.com Fri Oct 7 22:30:44 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Oct 2016 18:30:44 -0400 Subject: RFR(S): 8166461: Deprecate UseAutoGCSelectPolicy In-Reply-To: References: Message-ID: > On Oct 7, 2016, at 4:48 PM, sangheon wrote: > > Hi all, > > Can I have some reviews for this change to deprecate UseAutoGCSelectPolicy command-line option? > > The intent of UseAutoGCSelectPolicy option is to provide a way to automatically select GC based on target pause time. This option is introduced with AutoGCSelectPauseMillis and if MaxGCPauseMillis is smaller than or equal to AutoGCSelectPauseMillis(and if GC is not specified), low pause collector will be selected. I think it is enough to accomplish auto-GC-selection without UseAutoGCSelectPolicy. > > This patch only includes adding the deprecation message for UseAutoGCSelectPolicy option. > > CR: https://bugs.openjdk.java.net/browse/JDK-8166461 > Webrev: http://cr.openjdk.java.net/~sangheki/8166461/webrev.0 > Testing: JPRT > > Thanks, > Sangheon I thought the plan was to both deprecate the option, and to revert its behavior to selecting between a throughput collector (Parallel) and a low pause collector (probably CMS as before, though an argument could be made for G1). My understanding of the internal discussion was that the deprecation of the option was sufficient to address the concern of ergonomically selecting CMS if it is deprecated. From dmitry.samersoff at oracle.com Sat Oct 8 17:22:37 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Sat, 8 Oct 2016 20:22:37 +0300 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: <40baa78a-690f-12cd-7402-341919d215ad@oracle.com> Marcus, logConfiguration.cpp: 107: size_t is unsigned 282: Please add a comment that output[0] and output[1] is stdout/stderr -Dmitry On 2016-10-07 17:26, Marcus Larsson wrote: > Hi, > > Making another attempt to fix this issue. > > Summary: > The following patch resolves a problem where the VM would crash during > shutdown due to static log related memory being de-initialized before > the last use of the logging framework. The solution involves parts of > the Nifty Counter idiom [0] to control static initialization and > de-initialization of stdout's and stderr's LogOutputs. Both objects are > now allocated using placement new, and avoids destructor calls during > de-initialization. The LogStreamInitializer makes sure both objects are > initialized before first use. > > Because the LogOutput::Stdout/err pointers could no longer be kept in > LogOutput, I've replaced all usages of them with the new references > instead. > > The patch includes a regression test for this issue, contributed by > Michail Chernov. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8146009 > > Testing: > JPRT testset hotspot, included test on supported platforms. > > Thanks, > Marcus > > [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From david.holmes at oracle.com Sun Oct 9 23:31:50 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Oct 2016 09:31:50 +1000 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: Hi Marcus, On 8/10/2016 12:26 AM, Marcus Larsson wrote: > Hi, > > Making another attempt to fix this issue. > > Summary: > The following patch resolves a problem where the VM would crash during > shutdown due to static log related memory being de-initialized before > the last use of the logging framework. The solution involves parts of > the Nifty Counter idiom [0] to control static initialization and > de-initialization of stdout's and stderr's LogOutputs. Both objects are > now allocated using placement new, and avoids destructor calls during > de-initialization. The LogStreamInitializer makes sure both objects are > initialized before first use. Exactly when will this initialization take place? You have: ! LogFileStreamInitializer::LogFileStreamInitializer() { ! if (Atomic::cmpxchg(1, &initialized, 0) == 0) { but on many platforms cmpxchg is handled via a stub and so the stub generator needs to have been initialized. If you call before that you will get a non-atomic implementation ... in which case no point having this pretend to be thread-safe. I think I've queried this before, but does this have to rely on static initialization? It seems fragile and error prone. Thanks, David > Because the LogOutput::Stdout/err pointers could no longer be kept in > LogOutput, I've replaced all usages of them with the new references > instead. > > The patch includes a regression test for this issue, contributed by > Michail Chernov. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8146009 > > Testing: > JPRT testset hotspot, included test on supported platforms. > > Thanks, > Marcus > > [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter From iotsakp at gmail.com Mon Oct 10 00:34:33 2016 From: iotsakp at gmail.com (Ioannis Tsakpinis) Date: Mon, 10 Oct 2016 03:34:33 +0300 Subject: [PATCH] Invalid critical JNI function lookup (Windows x86) Message-ID: This patch fixes the lookup of critical JNI functions on Windows x86. There are two problems with the argument size calculation in the lookup_critical_entry function: 1) Critical natives do not have a JNIEnv parameter. Critical natives are always static, but do not have a jclass parameter. The current code assumes that both parameters exist and counts them against the total argument size. 2) For each Java array parameter, the critical native gets an additional length parameter for that array. The current code does not count them. On the 32-bit VM, the argument size is used to apply stdcall decorations to the function name. A wrong size is calculated with the current code, so the name used for the lookup is invalid (unless the function happens to have exactly two array parameters). diff -r fec31089c2ef src/share/vm/prims/nativeLookup.cpp --- a/src/share/vm/prims/nativeLookup.cpp Thu Oct 06 18:05:53 2016 -0700 +++ b/src/share/vm/prims/nativeLookup.cpp Sun Oct 09 22:44:54 2016 +0300 @@ -293,10 +293,12 @@ char* critical_name = critical_jni_name(method); // Compute argument size - int args_size = 1 // JNIEnv - + (method->is_static() ? 1 : 0) // class for static methods - + method->size_of_parameters(); // actual parameters - + int args_size = method->size_of_parameters(); // actual parameters + for (SignatureStream ss(signature); !ss.at_return_type(); ss.next()) { + if (ss.is_array()) { + args_size += T_INT_size; // array length parameter + } + } // 1) Try JNI short style entry = lookup_critical_style(method, critical_name, "", args_size, true); In steps 3 and 4 the function lookup is done without a prefix/suffix, so a workaround is available. On msvc JNI functions can be exported without decorations, but it's not without pain: it requires pragmas or a .DEF file. Regards, Ioannis From iotsakp at gmail.com Mon Oct 10 00:36:27 2016 From: iotsakp at gmail.com (Ioannis Tsakpinis) Date: Mon, 10 Oct 2016 03:36:27 +0300 Subject: [PATCH] Invalid value passed to critical JNI function (System V AMD64) Message-ID: This patch fixes invalid values passed to critical JNI functions on the System V AMD64 ABI (tested on Linux and MacOS). Specifically, this bug occurs when the arguments of a JNI method with 6 or more parameters are shuffled to match the critical native function order. The ABI provides 6 registers for passing arguments, so a temporary register must be used to break the shuffling cycle (handled in ComputeMoveOrder). If the temp register happens to be used for a 64bit argument, the target register will receive an invalid value. The fix is to call set2 when initializing the temp register to rbx. This marks the VMRegPair as a single physical register, which is important when copying 64bit values. diff -r fec31089c2ef src/cpu/x86/vm/sharedRuntime_x86_64.cpp --- a/src/cpu/x86/vm/sharedRuntime_x86_64.cpp Thu Oct 06 18:05:53 2016 -0700 +++ b/src/cpu/x86/vm/sharedRuntime_x86_64.cpp Sun Oct 09 22:44:54 2016 +0300 @@ -2178,7 +2178,7 @@ // critical natives they are offset down. GrowableArray arg_order(2 * total_in_args); VMRegPair tmp_vmreg; - tmp_vmreg.set1(rbx->as_VMReg()); + tmp_vmreg.set2(rbx->as_VMReg()); if (!is_critical_native) { for (int i = total_in_args - 1, c_arg = total_c_args - 1; i >= 0; i--, c_arg--) { The shuffling implementation on AArch64 is similar and the same fix may be required there, but I have not tested it. Regards, Ioannis From david.holmes at oracle.com Mon Oct 10 01:08:25 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Oct 2016 11:08:25 +1000 Subject: [PATCH] Invalid critical JNI function lookup (Windows x86) In-Reply-To: References: Message-ID: <8df0a742-1d22-c303-d2ea-efafdb93124f@oracle.com> Hi Ioannis, I have filed: https://bugs.openjdk.java.net/browse/JDK-8167408 for this. If you haven't already, please see: http://openjdk.java.net/contribute/ on how to contribute. While small simple patches can be accepted from anyone, more complex patches require you to sign the OCA. BTW I don't see anything Windows x86 specific here, as this is shared code. Thanks, David On 10/10/2016 10:34 AM, Ioannis Tsakpinis wrote: > This patch fixes the lookup of critical JNI functions on Windows x86. > > There are two problems with the argument size calculation in the > lookup_critical_entry function: > > 1) Critical natives do not have a JNIEnv parameter. Critical natives are > always static, but do not have a jclass parameter. The current code assumes > that both parameters exist and counts them against the total argument size. > > 2) For each Java array parameter, the critical native gets an additional > length parameter for that array. The current code does not count them. > > On the 32-bit VM, the argument size is used to apply stdcall decorations to > the function name. A wrong size is calculated with the current code, so the > name used for the lookup is invalid (unless the function happens to have > exactly two array parameters). > > diff -r fec31089c2ef src/share/vm/prims/nativeLookup.cpp > --- a/src/share/vm/prims/nativeLookup.cpp Thu Oct 06 18:05:53 2016 -0700 > +++ b/src/share/vm/prims/nativeLookup.cpp Sun Oct 09 22:44:54 2016 +0300 > @@ -293,10 +293,12 @@ > char* critical_name = critical_jni_name(method); > > // Compute argument size > - int args_size = 1 // JNIEnv > - + (method->is_static() ? 1 : 0) // class for static methods > - + method->size_of_parameters(); // actual parameters > - > + int args_size = method->size_of_parameters(); // actual parameters > + for (SignatureStream ss(signature); !ss.at_return_type(); ss.next()) { > + if (ss.is_array()) { > + args_size += T_INT_size; // array length parameter > + } > + } > > // 1) Try JNI short style > entry = lookup_critical_style(method, critical_name, "", > args_size, true); > > In steps 3 and 4 the function lookup is done without a prefix/suffix, so a > workaround is available. On msvc JNI functions can be exported without > decorations, but it's not without pain: it requires pragmas or a .DEF file. > > Regards, > Ioannis > From david.holmes at oracle.com Mon Oct 10 01:15:23 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Oct 2016 11:15:23 +1000 Subject: [PATCH] Invalid value passed to critical JNI function (System V AMD64) In-Reply-To: References: Message-ID: <4c6672f1-32e7-40ee-397e-fb93cd6bb603@oracle.com> Thanks Ioannis. I have filed: https://bugs.openjdk.java.net/browse/JDK-8167409 David On 10/10/2016 10:36 AM, Ioannis Tsakpinis wrote: > This patch fixes invalid values passed to critical JNI functions on the > System V AMD64 ABI (tested on Linux and MacOS). > > Specifically, this bug occurs when the arguments of a JNI method with 6 or > more parameters are shuffled to match the critical native function order. > The ABI provides 6 registers for passing arguments, so a temporary register > must be used to break the shuffling cycle (handled in ComputeMoveOrder). If > the temp register happens to be used for a 64bit argument, the target > register will receive an invalid value. > > The fix is to call set2 when initializing the temp register to rbx. This > marks the VMRegPair as a single physical register, which is important when > copying 64bit values. > > diff -r fec31089c2ef src/cpu/x86/vm/sharedRuntime_x86_64.cpp > --- a/src/cpu/x86/vm/sharedRuntime_x86_64.cpp Thu Oct 06 18:05:53 2016 -0700 > +++ b/src/cpu/x86/vm/sharedRuntime_x86_64.cpp Sun Oct 09 22:44:54 2016 +0300 > @@ -2178,7 +2178,7 @@ > // critical natives they are offset down. > GrowableArray arg_order(2 * total_in_args); > VMRegPair tmp_vmreg; > - tmp_vmreg.set1(rbx->as_VMReg()); > + tmp_vmreg.set2(rbx->as_VMReg()); > > if (!is_critical_native) { > for (int i = total_in_args - 1, c_arg = total_c_args - 1; i >= 0; > i--, c_arg--) { > > The shuffling implementation on AArch64 is similar and the same fix may be > required there, but I have not tested it. > > Regards, > Ioannis > From iotsakp at gmail.com Mon Oct 10 01:24:25 2016 From: iotsakp at gmail.com (Ioannis Tsakpinis) Date: Mon, 10 Oct 2016 04:24:25 +0300 Subject: [PATCH] Invalid critical JNI function lookup (Windows x86) In-Reply-To: <8df0a742-1d22-c303-d2ea-efafdb93124f@oracle.com> References: <8df0a742-1d22-c303-d2ea-efafdb93124f@oracle.com> Message-ID: Hi David, On Mon, Oct 10, 2016 at 4:08 AM, David Holmes wrote: > Hi Ioannis, > > I have filed: > > https://bugs.openjdk.java.net/browse/JDK-8167408 > > for this. Thanks! > If you haven't already, please see: > > http://openjdk.java.net/contribute/ > > on how to contribute. While small simple patches can be accepted from > anyone, more complex patches require you to sign the OCA. Yes, I'm aware of the OCA requirement and plan to have it signed soon. This is a simple patch (and my first contribution to OpenJDK), so I decided to go ahead and submit it as a participant. > BTW I don't see anything Windows x86 specific here, as this is shared code. The OS-specific bit comes in lookup_critical_style, which passes args_size to os::print_jni_name_prefix_on and os::print_jni_name_suffix_on. Afaict, these don't do anything except on Windows x86. Thanks, Ioannis From goetz.lindenmaier at sap.com Mon Oct 10 06:37:59 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 10 Oct 2016 06:37:59 +0000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: References: <4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap> <41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com> Message-ID: <0ad821fd56524e938c5dd65bf5bba617@DEWDFE13DE50.global.corp.sap> Hi David, ok, I removed the comment ... http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr04/ Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Mittwoch, 5. Oktober 2016 04:17 > To: Lindenmaier, Goetz ; Volker Simonis > > Cc: hotspot-dev at openjdk.java.net; core-libs-dev dev at openjdk.java.net> > Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > > On 5/10/2016 2:52 AM, Lindenmaier, Goetz wrote: > > Hi David, > > > > I fixed the change accordingly: > > http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr03/ > > > > But do you really want me to incorporate a bugfix for ARM in the port? > > Sure - it is only an error message and we've probably never encountered > it. All the other code has the case for Aarch64 first so this is no > different - and no need to comment it as it isn't commented upon > elsewhere. > > Thanks, > David > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Dienstag, 4. Oktober 2016 09:56 > >> To: Lindenmaier, Goetz ; Volker Simonis > >> > >> Cc: hotspot-dev at openjdk.java.net; core-libs-dev >> dev at openjdk.java.net> > >> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > >> > >> Hi Goetz, > >> > >> On 28/09/2016 8:26 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> here a new webrev for this change: > >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >> basic_s390/hotspot.wr02/ > >>> > >>> I reverted the major reorderings in macros.hpp and os_linux.cpp. > >>> David asked me to do so, and I guess it makes reviewing more simple. > >> > >> Thanks. That said ... > >> > >>> Also this fixes the issue spotted by David which actually was wrong. > >>> The other renaming of ARM to ARM32 was correct, though, as > >>> AARCH64 is defined in both ARM 64-bit ports, and is checked before. > >>> So the existing case checking ARM is only reached if !LP64. > >>> I documented this ... > >> > >> ... We actually have a bug with the Elf32_* defines in os_linux.cpp > >> > >> 1769 #elif (defined ARM) // ARM must come before AARCH64 because > the > >> closed 64-bit port requires so. > >> 1770 static Elf32_Half running_arch_code=EM_ARM; > >> > >> Aarch64 should come first otherwise we will set the wrong value. I've > >> been told it would only affect an error message but still ... can I ask > >> you to fix this please. Thanks. > >> > >> --- > >> src/share/vm/code/codeCache.cpp > >> > >> I'd prefer to just see something like: > >> > >> S390_ONLY(if (_heaps == NULL) return;) // May be true when NMT detail > is > >> used > >> > >> --- > >> > >> Otherwise seems okay. > >> > >> Thanks, > >> David > >> > >>> Also I removed the change in mutex.hpp. Maybe we contribute > >>> the full change this was part of, but independent of the s390 port. > >>> > >>> I withdraw the part of the change to jdk introducing jvm.cfg. Volker > wants > >> to > >>> do a separate change for this. > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>> > >>>> -----Original Message----- > >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com] > >>>> Sent: Dienstag, 27. September 2016 19:58 > >>>> To: David Holmes > >>>> Cc: Lindenmaier, Goetz ; hotspot- > >>>> dev at openjdk.java.net; core-libs-dev dev at openjdk.java.net> > >>>> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > >>>> > >>>> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes > >> > >>>> wrote: > >>>>> Hi Goetz, > >>>>> > >>>>> I see a change not related directly to S390 ie change from ARM to > ARM32 > >> in > >>>>> src/os/linux/vm/os_linux.cpp > >>>>> > >>>> > >>>> The change looks a little confusing because Goetz reordered the ifdef > >>>> cascades alphabetically (which I think is good). > >>>> > >>>> Besides that, the only real change not related to s390 is indeed the > >>>> change from ARM to ARM32 which happend two times in the file. > >>>> > >>>> @Goetz: have you done this intentionally? > >>>> > >>>>> It will be a while before I can go through this in any detail. > >>>>> > >>>>> David > >>>>> > >>>>> > >>>>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> This change is part of the s390 port. It contains some basic adaptions > >>>>>> needed for a full hotspot port for linux s390x. > >>>>>> > >>>>>> It defines the required macros, platform names and includes. > >>>>>> > >>>>>> The s390 port calles CodeCache::contains() in current_frame(), which > is > >>>>>> used in NMT. As NMT already collects stack traces before the > >> CodeCache > >>>> is > >>>>>> initialized, contains() needs a check for this. > >>>>>> > >>>>>> Wherever a row of platforms are listed, I sorted them alphabetically. > >>>>>> > >>>>>> The jdk requires the file jvm.cfg. > >>>>>> > >>>>>> Please review. I please need a sponsor. > >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >>>> basic_s390/hotspot.wr01/ > >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >> basic_s390/jdk.wr01/ > >>>>>> > >>>>>> Best regards, > >>>>>> Goetz. > >>>>>> > >>>>> From goetz.lindenmaier at sap.com Mon Oct 10 06:38:54 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 10 Oct 2016 06:38:54 +0000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: <7b9201ce-a417-e489-7b46-c0488e8fda8f@oracle.com> References: <7b9201ce-a417-e489-7b46-c0488e8fda8f@oracle.com> Message-ID: <14a5210f9626419db46ee453159bb0f0@DEWDFE13DE50.global.corp.sap> Hi Coleen, can I consider this reviewed? I removed the change to mutex. http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr04/ Thanks and best regards, Goetz. > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Coleen Phillimore > Sent: Dienstag, 27. September 2016 20:30 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > > > Hi, I'm surprised there aren't more changes, probably because of the > #include work that was done by you previously. Which is great. > > http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr01/src/share/vm/runtime/mutex.hpp.udiff.html > > Can you describe this change? This makes all of the mutex's take a lot > of space, and some don't have very long names. Was this added to avoid > false sharing as the comment implies or to support longer names in > monitors? > > thanks, > Coleen > > > > On 9/27/16 1:57 PM, Volker Simonis wrote: > > On Fri, Sep 23, 2016 at 8:11 AM, David Holmes > wrote: > >> Hi Goetz, > >> > >> I see a change not related directly to S390 ie change from ARM to ARM32 > in > >> src/os/linux/vm/os_linux.cpp > >> > > The change looks a little confusing because Goetz reordered the ifdef > > cascades alphabetically (which I think is good). > > > > Besides that, the only real change not related to s390 is indeed the > > change from ARM to ARM32 which happend two times in the file. > > > > @Goetz: have you done this intentionally? > > > >> It will be a while before I can go through this in any detail. > >> > >> David > >> > >> > >> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> This change is part of the s390 port. It contains some basic adaptions > >>> needed for a full hotspot port for linux s390x. > >>> > >>> It defines the required macros, platform names and includes. > >>> > >>> The s390 port calles CodeCache::contains() in current_frame(), which is > >>> used in NMT. As NMT already collects stack traces before the CodeCache > is > >>> initialized, contains() needs a check for this. > >>> > >>> Wherever a row of platforms are listed, I sorted them alphabetically. > >>> > >>> The jdk requires the file jvm.cfg. > >>> > >>> Please review. I please need a sponsor. > >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr01/ > >>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ > >>> > >>> Best regards, > >>> Goetz. > >>> From david.holmes at oracle.com Mon Oct 10 06:47:20 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Oct 2016 16:47:20 +1000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: <0ad821fd56524e938c5dd65bf5bba617@DEWDFE13DE50.global.corp.sap> References: <4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap> <41f35197-91fc-bc28-baaa-a375e5301a1e@oracle.com> <0ad821fd56524e938c5dd65bf5bba617@DEWDFE13DE50.global.corp.sap> Message-ID: <87e02882-c822-a2f9-d026-9c060772b936@oracle.com> Thanks Goetz! David On 10/10/2016 4:37 PM, Lindenmaier, Goetz wrote: > Hi David, > > ok, I removed the comment ... > http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr04/ > > Best regards, > Goetz. > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Mittwoch, 5. Oktober 2016 04:17 >> To: Lindenmaier, Goetz ; Volker Simonis >> >> Cc: hotspot-dev at openjdk.java.net; core-libs-dev > dev at openjdk.java.net> >> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. >> >> On 5/10/2016 2:52 AM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> I fixed the change accordingly: >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/hotspot.wr03/ >>> >>> But do you really want me to incorporate a bugfix for ARM in the port? >> >> Sure - it is only an error message and we've probably never encountered >> it. All the other code has the case for Aarch64 first so this is no >> different - and no need to comment it as it isn't commented upon >> elsewhere. >> >> Thanks, >> David >> >>> Best regards, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Dienstag, 4. Oktober 2016 09:56 >>>> To: Lindenmaier, Goetz ; Volker Simonis >>>> >>>> Cc: hotspot-dev at openjdk.java.net; core-libs-dev >>> dev at openjdk.java.net> >>>> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. >>>> >>>> Hi Goetz, >>>> >>>> On 28/09/2016 8:26 PM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> here a new webrev for this change: >>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>>> basic_s390/hotspot.wr02/ >>>>> >>>>> I reverted the major reorderings in macros.hpp and os_linux.cpp. >>>>> David asked me to do so, and I guess it makes reviewing more simple. >>>> >>>> Thanks. That said ... >>>> >>>>> Also this fixes the issue spotted by David which actually was wrong. >>>>> The other renaming of ARM to ARM32 was correct, though, as >>>>> AARCH64 is defined in both ARM 64-bit ports, and is checked before. >>>>> So the existing case checking ARM is only reached if !LP64. >>>>> I documented this ... >>>> >>>> ... We actually have a bug with the Elf32_* defines in os_linux.cpp >>>> >>>> 1769 #elif (defined ARM) // ARM must come before AARCH64 because >> the >>>> closed 64-bit port requires so. >>>> 1770 static Elf32_Half running_arch_code=EM_ARM; >>>> >>>> Aarch64 should come first otherwise we will set the wrong value. I've >>>> been told it would only affect an error message but still ... can I ask >>>> you to fix this please. Thanks. >>>> >>>> --- >>>> src/share/vm/code/codeCache.cpp >>>> >>>> I'd prefer to just see something like: >>>> >>>> S390_ONLY(if (_heaps == NULL) return;) // May be true when NMT detail >> is >>>> used >>>> >>>> --- >>>> >>>> Otherwise seems okay. >>>> >>>> Thanks, >>>> David >>>> >>>>> Also I removed the change in mutex.hpp. Maybe we contribute >>>>> the full change this was part of, but independent of the s390 port. >>>>> >>>>> I withdraw the part of the change to jdk introducing jvm.cfg. Volker >> wants >>>> to >>>>> do a separate change for this. >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Volker Simonis [mailto:volker.simonis at gmail.com] >>>>>> Sent: Dienstag, 27. September 2016 19:58 >>>>>> To: David Holmes >>>>>> Cc: Lindenmaier, Goetz ; hotspot- >>>>>> dev at openjdk.java.net; core-libs-dev > dev at openjdk.java.net> >>>>>> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. >>>>>> >>>>>> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes >>>> >>>>>> wrote: >>>>>>> Hi Goetz, >>>>>>> >>>>>>> I see a change not related directly to S390 ie change from ARM to >> ARM32 >>>> in >>>>>>> src/os/linux/vm/os_linux.cpp >>>>>>> >>>>>> >>>>>> The change looks a little confusing because Goetz reordered the ifdef >>>>>> cascades alphabetically (which I think is good). >>>>>> >>>>>> Besides that, the only real change not related to s390 is indeed the >>>>>> change from ARM to ARM32 which happend two times in the file. >>>>>> >>>>>> @Goetz: have you done this intentionally? >>>>>> >>>>>>> It will be a while before I can go through this in any detail. >>>>>>> >>>>>>> David >>>>>>> >>>>>>> >>>>>>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> This change is part of the s390 port. It contains some basic adaptions >>>>>>>> needed for a full hotspot port for linux s390x. >>>>>>>> >>>>>>>> It defines the required macros, platform names and includes. >>>>>>>> >>>>>>>> The s390 port calles CodeCache::contains() in current_frame(), which >> is >>>>>>>> used in NMT. As NMT already collects stack traces before the >>>> CodeCache >>>>>> is >>>>>>>> initialized, contains() needs a check for this. >>>>>>>> >>>>>>>> Wherever a row of platforms are listed, I sorted them alphabetically. >>>>>>>> >>>>>>>> The jdk requires the file jvm.cfg. >>>>>>>> >>>>>>>> Please review. I please need a sponsor. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>>>>> basic_s390/hotspot.wr01/ >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>>> basic_s390/jdk.wr01/ >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Goetz. >>>>>>>> >>>>>>> From marcus.larsson at oracle.com Mon Oct 10 07:24:44 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 10 Oct 2016 09:24:44 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: <6ee40cf5-15f5-d89e-7653-bcec9cfe76ef@oracle.com> References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> <6ee40cf5-15f5-d89e-7653-bcec9cfe76ef@oracle.com> Message-ID: <3c121184-8fc4-ffa8-0269-474589c6399f@oracle.com> Hi, Thanks for looking at this. On 10/07/2016 06:05 PM, Michail Chernov wrote: > Hi Marcus, > > Thank you for fixing this. > > Could you please change @requires in the test? > > It should be > > * @requires vm.gc.G1 > > instead of > > * @requires vm.gc=="G1" | vm.gc=="null" > > New form of tag is applicable for platforms where G1 are not supported > and allows to exclude this test on those platforms. Older version of > tag can cause to the test failure. Will fix! Marcus > > Thanks, > Michail > > > On 10/07/2016 05:26 PM, Marcus Larsson wrote: >> Hi, >> >> Making another attempt to fix this issue. >> >> Summary: >> The following patch resolves a problem where the VM would crash >> during shutdown due to static log related memory being de-initialized >> before the last use of the logging framework. The solution involves >> parts of the Nifty Counter idiom [0] to control static initialization >> and de-initialization of stdout's and stderr's LogOutputs. Both >> objects are now allocated using placement new, and avoids destructor >> calls during de-initialization. The LogStreamInitializer makes sure >> both objects are initialized before first use. >> >> Because the LogOutput::Stdout/err pointers could no longer be kept in >> LogOutput, I've replaced all usages of them with the new references >> instead. >> >> The patch includes a regression test for this issue, contributed by >> Michail Chernov. >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8146009 >> >> Testing: >> JPRT testset hotspot, included test on supported platforms. >> >> Thanks, >> Marcus >> >> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter > From marcus.larsson at oracle.com Mon Oct 10 07:31:45 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 10 Oct 2016 09:31:45 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: <40baa78a-690f-12cd-7402-341919d215ad@oracle.com> References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> <40baa78a-690f-12cd-7402-341919d215ad@oracle.com> Message-ID: Hi Dmitry, Thanks for looking at this. On 10/08/2016 07:22 PM, Dmitry Samersoff wrote: > Marcus, > > logConfiguration.cpp: > > 107: size_t is unsigned The loop breaks when i = 0, not when i < 0, so it should be fine. > 282: Please add a comment that output[0] and output[1] is stdout/stderr Will fix. Marcus > > -Dmitry > > > On 2016-10-07 17:26, Marcus Larsson wrote: >> Hi, >> >> Making another attempt to fix this issue. >> >> Summary: >> The following patch resolves a problem where the VM would crash during >> shutdown due to static log related memory being de-initialized before >> the last use of the logging framework. The solution involves parts of >> the Nifty Counter idiom [0] to control static initialization and >> de-initialization of stdout's and stderr's LogOutputs. Both objects are >> now allocated using placement new, and avoids destructor calls during >> de-initialization. The LogStreamInitializer makes sure both objects are >> initialized before first use. >> >> Because the LogOutput::Stdout/err pointers could no longer be kept in >> LogOutput, I've replaced all usages of them with the new references >> instead. >> >> The patch includes a regression test for this issue, contributed by >> Michail Chernov. >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8146009 >> >> Testing: >> JPRT testset hotspot, included test on supported platforms. >> >> Thanks, >> Marcus >> >> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter > From marcus.larsson at oracle.com Mon Oct 10 08:08:15 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 10 Oct 2016 10:08:15 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: <1050d015-8fbf-cf9e-e210-3baef9e44e97@oracle.com> Hi David, Thanks for looking at this. On 10/10/2016 01:31 AM, David Holmes wrote: > Hi Marcus, > > On 8/10/2016 12:26 AM, Marcus Larsson wrote: >> Hi, >> >> Making another attempt to fix this issue. >> >> Summary: >> The following patch resolves a problem where the VM would crash during >> shutdown due to static log related memory being de-initialized before >> the last use of the logging framework. The solution involves parts of >> the Nifty Counter idiom [0] to control static initialization and >> de-initialization of stdout's and stderr's LogOutputs. Both objects are >> now allocated using placement new, and avoids destructor calls during >> de-initialization. The LogStreamInitializer makes sure both objects are >> initialized before first use. > > Exactly when will this initialization take place? It will happen during static initialization. The initializer object will ensure it happens before any use of the outputs. > You have: > > ! LogFileStreamInitializer::LogFileStreamInitializer() { > ! if (Atomic::cmpxchg(1, &initialized, 0) == 0) { > > but on many platforms cmpxchg is handled via a stub and so the stub > generator needs to have been initialized. If you call before that you > will get a non-atomic implementation ... in which case no point having > this pretend to be thread-safe. Hmm, good point. I think I might've been too defensive here actually. Static initialization should be single threaded so the atomics are unnecessary, I think. > > I think I've queried this before, but does this have to rely on static > initialization? It seems fragile and error prone. We want logging available during static initialization, so we're kind of stuck with it. Possibly something we can improve in the future though. Thanks, Marcus > > Thanks, > David > >> Because the LogOutput::Stdout/err pointers could no longer be kept in >> LogOutput, I've replaced all usages of them with the new references >> instead. >> >> The patch includes a regression test for this issue, contributed by >> Michail Chernov. >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8146009 >> >> Testing: >> JPRT testset hotspot, included test on supported platforms. >> >> Thanks, >> Marcus >> >> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter From mikael.gerdin at oracle.com Mon Oct 10 08:28:55 2016 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 10 Oct 2016 10:28:55 +0200 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <6bf01692-a37f-6ff2-8a92-27e379143ebe@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <11f3d2d6-acac-fc06-3d03-96728b7905e9@oracle.com> <6bf01692-a37f-6ff2-8a92-27e379143ebe@oracle.com> Message-ID: Hi Coleen, On 2016-10-07 17:24, Coleen Phillimore wrote: > Mikael, thanks again ... embedded replies and 2 questions: > > On 10/7/16 10:39 AM, Mikael Gerdin wrote: >> Hi Coleen, >> >> On 2016-10-06 19:54, Coleen Phillimore wrote: >>> >>> Here is an update to the Metadata leak change. There was a bug >>> introduced when cleaning this up, which Mikael also found. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.02/webrev >>> >>> Changes include adding logging for report_metadata_oome, which >>> necessitated removing ResourceMarks in ClassLoaderData::dump because the >>> stream passed in already required a ResourceMark, so it got a nested >>> ResourceMark message for the stream. >>> >>> I changed logging for tracing block allocations to log_trace(gc, >>> metaspace, freelist, blocks). >>> >>> In BlockFreelist::get_block and BlockFreelist::return_block() assert >>> that it's not called for a size smaller than the minimum allocation >>> (which was the bug). Renamed get_raw_word_size() to >>> get_allocation_word_size(). This rounds up to the minimum allocation >>> size which is the same as small_block_min_size. >>> >>> Also, I added a test that takes a long time to execute to verify this, >>> and excluded it from JPRT. I could skip adding this test if people >>> don't want it. Also, the test verifies that continuously redefining the >>> same class gets memory for the new class that was released because the >>> block sizes are the same. When the test exits, it gets a metaspace OOM >>> because loading new classes and allocating metadata can't use the blocks >>> returned (wrong size). There is still fragmentation in this >>> implementation, but it's better that things deallocated < 12 words are >>> actually freed. I'll file an RFE to work on a perfect algorithm, or to >>> investigate finding a better one, although I consider this a stress test >>> that uses all of metaspace to MaxMetaspaceSize, leaving allocation only >>> to the block fragments left. This isn't a typical use case. >>> >>> Some comments and corrections to my responses to Mikael below: >>> >>> On 10/4/16 12:15 PM, Coleen Phillimore wrote: >>>> >>>> Hi Mikael, >>>> >>>> Thanks for looking at this change. >>>> >>>> On 10/4/16 8:32 AM, Mikael Gerdin wrote: >>>>> Hi Coleen, >>>>> >>>>> On 2016-09-30 21:02, Coleen Phillimore wrote: >>>>>> Summary: Return Metablocks smaller than dictionary's dark matter. >>>>>> >>>>>> This change contributed by Jon Masamitsu and myself. To reclaim >>>>>> "dark >>>>>> matter" this change adds an array of small blocks by size, created >>>>>> lazily, to return Metablocks smaller than the BinaryTreeDictionary >>>>>> entry's minimum size. This change also fixed a bug in small object >>>>>> double free and adds debugging code to check for this case. >>>>>> >>>>>> With this change, the submitted test case runs indefinitely. Also >>>>>> passed rbt tier 1-5 testing. >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >>>>> >>>>> I'd prefer it if SmallBlocks didn't expose its implementation by >>>>> returning its FreeLists by reference, could you change it to have >>>>> * return_chunk() >>>>> * get_chunk() >>>>> * num_chunks(word_size) >>>>> >>>>> and get rid of list_at? >>>> >>>> Okay, I refactored this so small_chunks()->get_block() and >>>> return_block() are used rather than getting list_at. I didn't see >>>> where you got num_chunks, but list_at is hidden. >> >> Thanks, that's much better! >> >>>>> >>>>> -- >>>>> >>>>> For how long do you plan to keep BlockFreelist::_all_blocks? >>>>> I see that it's debug only but I fear that it could case problems >>>>> with running out of native memory in our internal testing of debug >>>>> builds. >>>> >>>> I thought about taking it out, but it helped me find the double free >>>> bug. I think if we add new code to metadata and have to call >>>> deallocate_contents on it, we risk re-introducting these double free >>>> bugs. I could take it out. I don't think this gets that big but I'd >>>> hate to introduce some sort of OOM bug in our testing. >> >> Perhaps a trade off could be to have a bound on the size of the array >> and deallocate and disable it if it grows too large? > > I just removed it. I had run all the tests with the debugging code and > they passed. If we have to, we can add it back again pretty easily. Ok. >> >>>>> >>>>> BlockFreelist::min_size() is a bit of a misnomer since it returns the >>>>> minimum size of blocks to be put on the BlockTreeDictionary, not the >>>>> minimum size of blocks which are reusable. >>>> >>>> How about min_dictionary_size() ? >> >> That's fine. >> >>>> >>>>> >>>>> Is there any particular reason behind _small_blocks being lazy >>>>> allocated and _dictionary not? >>>> >>>> We lazily create the BlockFreelists with this change. >>>> >>>> // Lazily create a block_freelist >>>> if (block_freelists() == NULL) { >>>> _block_freelists = new BlockFreelist(); >>>> } >>>> >>>> So the small_blocks are lazily created in the BlockFreelists but the >>>> dictionary is not. I guess if we're going to create the >>>> BlockFreelists here, we'll most likely need both and maybe >>>> small_blocks need not be lazily created. Was that your suggestion? >>>> >>>> My concern with this change was all the space used by the >>>> small_blocks() but if we're doing any deallocation within the >>>> metaspace, at least one of the things will be <12 words. >>>> >>>> I'll make small_blocks() not be lazily allocated since BlockFreelist >>>> are. These are pretty expensive, but should be limited to a few >>>> metaspaces. >>> >>> Class SpaceManager doesn't need small_blocks() so I left small_blocks as >>> lazily allocated. >> >> Ok, I agree that the size of small_blocks() motivates its lazy >> allocation scheme. >> >>>>> >>>>> I would prefer if BlockFreelist::return_block would perform the >>>>> checks in reverse order instead of having a return inside the first >>>>> if block, something like >>>>> >>>>> if (word_size > small_block_max_size) { >>>>> dict()->return_chunk(c) >>>>> } else if (word_size > small_block_min_size) { >>>>> small_blocks()->return_chunk(c) >>>>> } else { >>>>> // dark matter >>>>> } >>>> >>>> Why? We don't want to cast Metablock into dark matter so check if >>>> word_size < small_block_min_size first. >>>> >>>> Metablock* free_chunk = ::new (p) Metablock(word_size); >>>> if (word_size < SmallBlocks::small_block_max_size()) { >>>> small_blocks()->return_chunk(word_size); >>>> } else { >>>> dictionary()->return_chunk(free_chunk); >>>> } >>>> >>> >>> There is no dark matter in these functions anymore. >> >> Oh! Right. >> >> >>>>> >>>>> >>>>> For BlockFreelist::get_block I realize that the code is a bit more >>>>> complex but this also raises a few questions. >>>>> * When we allocate from the dictionary we search for a block which is >>>>> at least as large as we ask for and split it, returning whatever was >>>>> left back to the free list. We don't search for "large enough" blocks >>>>> from the small blocks manager, is that intentional or just to keep >>>>> the code simple (I like simple)? >>>> >>>> I'm glad you asked about this so I could give background. It turns >>>> out that we deallocate metaspace items better this way. I had a >>>> version that did exactly what you said. It was a simple sorted linked >>>> list of returned blocks < min_dictionary_size (12) where get_block >>>> returned the first block where the item would fit. It had some best >>>> fit algorithm so if the block returned was a lot bigger, it wouldn't >>>> pick it. >>>> >>>> My implementation could get through 69300 retransformations before the >>>> list didn't work anymore (too many small block fragments of the wrong >>>> size) and metaspace was exhausted (metaspace was limited to 12M in >>>> this test). Jon's implementation ran this test indefinitely. So it's >>>> somewhat simple but it worked really well. >>>> >>>>> >>>>> In the last part of get_block where we return the unused part of a >>>>> block retrieved from the dictionary uses compares with >>>>> BlockFreelist::min_size() which, as I mentioned above, is the min >>>>> size for blocks in the dictionary, not the min size for blocks to be >>>>> reusable. >>>>> I think this code can just call return_block unconditionally for any >>>>> nonzero value of "unused" and let return_block deal with dark matter. >>>>> >>>> yes, I'll make that change. >>> >>> This change has to be conditional because I assert that >>> BlockFreelist::return_block() is never called for < >>> small_block_min_size. >> >> Ok. >> >> >> There's one more thing, though. >> In SpaceManager::retire_current_chunk the last piece of the current >> allocation chunk is made reusable through a call to >> SpaceManager::deallocate. There are two problems with this: >> >> * retire_current_chunk only calls deallocate for wasted blocks larger >> than the min dictionary size, this means that wasted blocks which >> would fit in SmallBlocks will be wasted instead of reusable. > > Yeah. I didn't know if this was worth doing for 12 words or less. Do you? To be honest: I don't know. We added the SmallBlocks free list for a reason, I guess, so maybe we should try a bit harder to conserve metaspace memory. > >> * the remaining_words parameter which retire_current_chunk passes to >> deallocate is _exact_ and must not be aligned up through >> get_allocation_word_size. >> > > Since it's already greater than the allocation_word_size, this isn't a > problem. Well, if in a 32 bit VM a deallocation is done for an object of 13 words then get_allocation_word_size will align up by 8 to 14 and the block returned could be too large. I just realized that this cannot happen since allocations are always aligned up by 8, but I wanted to share my reasoning at least. > >> >> Also, >> are you sure about logging the output of ClassLoaderData::dump on the >> info level? Perhaps logging the cld dump on the debug level is a good >> trade-off? It looks like the trace level causes the VSM to print the >> entire free lists to the log so that's probably a bit much. > > It is a lot of stuff I didn't want to see. I could change it back to > trace though because the thing I really wanted to see in > MetaspaceAux::dump() is guarded by trace > > if (log_is_enabled(Trace, gc, metaspace, freelist)) { > if (block_freelists() != NULL) block_freelists()->print_on(out); > } > > It seems like debug would be a better level though for both. Yeah, I think that's a good trade-off. /Mikael >> >> >>> >>> Thanks, >>> Coleen >>> >>>>> >>>>> >>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>> object double free"? >>>> >>>> Yes. >>>>> Is the problem that when a method was freed its annotations were >>>>> deallocated as well? Could the annotations fields in the "old" >>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>> could be kept or is that just needless complexity? >>>> >>>> I'd rather copy the annotations, because I don't know how the old >>>> method, which could still be running, might use the annotations. I >>>> don't want to mess with that but I see your point. >> >> That's fine I think, I wouldn't want to mess around with it either. I >> just wanted to understand the reasoning behind the change, thanks for >> confirming my suspicion. >> > > thanks, > Coleen > >> /Mikael >> >>>> >>>> Coleen >>>> >>>>> >>>>> Thanks >>>>> /Mikael >>>>> >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> Coleen and Jon >>>> >>> > From thomas.stuefe at gmail.com Mon Oct 10 08:56:29 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Oct 2016 10:56:29 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. Message-ID: Hi all, May I have please some feedback for this enhancement proposal? https://bugs.openjdk.java.net/browse/JDK-8166690 In one very short sentence it proposes a better allocation scheme for Metaspace Chunks in order to reduce fragmentation and metaspace waste. I also added a short presentation which describes the problem and how we solved it in our VM port. https://bugs.openjdk.java.net/secure/attachment/63894/Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf Thank you very much! Kind Regards, Thomas On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe wrote: > Dear all, > > please take a look at this Enhancement Proposal for the metaspace > allocator. I hope these are the right groups for this discussion. > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > Background: > > We at SAP see at times at customer installations OOMs in Metaspace > (usually, with compressed class pointers enabled, in Compressed Class > Space). The VM attempts to allocate metaspace and fails, hitting the > CompressedClassSpaceSize limit. Note that we usually set the limit lower > than the default, typically at 256M. > > When analyzing, we observed that a large part of the metaspace is indeed > free but "locked in" into metaspace chunks of the wrong size: often we > would find a lot of free small chunks, but the allocation request was for > medium chunks, and failed. > > The reason was that if at some point in time a lot of class loaders were > alive, each with only a few small classes loaded. This would lead to the > metaspace being swamped with lots of small chunks. This is because each > SpaceManager first allocates small chunks, only after a certain amount of > allocation requests switches to larger chunks. > > These small chunks are free and wait in the freelist, but cannot be reused > for allocation requests which require larger chunks, even if they are > physically adjacent in the virtual space. > > We (at SAP) added a patch which allows on-the-fly metaspace chunk merging > - to merge multiple adjacent smaller chunk to form a larger chunk. This, in > combination with the reverse direction - splitting a large chunk to get > smaller chunks - partly negates the "chunks-are-locked-in-into-their-size" > limitation and provides for better reuse of metaspace chunks. It also > provides better defragmentation as well. > > I discussed this fix off-list with Coleen Phillimore and Jon Masamitsu, > and instead of just offering this as a fix, both recommended to open a JEP > for this, because its scope would be beyond that of a simple fix. > > So here is my first JEP :) I hope it follows the right form. Please, if > you have time, take a look and tell us what you think. > > Thank you, and Kind Regards, > > Thomas St?fe > > > > From marcus.larsson at oracle.com Mon Oct 10 09:20:32 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 10 Oct 2016 11:20:32 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: Updated webrev after feedback: http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 Incremental: http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 On 10/07/2016 04:26 PM, Marcus Larsson wrote: > Hi, > > Making another attempt to fix this issue. > > Summary: > The following patch resolves a problem where the VM would crash during > shutdown due to static log related memory being de-initialized before > the last use of the logging framework. The solution involves parts of > the Nifty Counter idiom [0] to control static initialization and > de-initialization of stdout's and stderr's LogOutputs. Both objects > are now allocated using placement new, and avoids destructor calls > during de-initialization. The LogStreamInitializer makes sure both > objects are initialized before first use. > > Because the LogOutput::Stdout/err pointers could no longer be kept in > LogOutput, I've replaced all usages of them with the new references > instead. > > The patch includes a regression test for this issue, contributed by > Michail Chernov. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8146009 > > Testing: > JPRT testset hotspot, included test on supported platforms. > > Thanks, > Marcus > > [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter From martin.walsh at oracle.com Mon Oct 10 10:05:14 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Mon, 10 Oct 2016 11:05:14 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> <34f41756-7ac7-684e-c896-9b8babf8f419@oracle.com> Message-ID: Could I get a second reviewer for these changes please? Thanks, Martin On 10/ 4/16 10:06 AM, David Holmes wrote: > On 4/10/2016 7:03 PM, Martin Walsh wrote: >> (dropped build-dev alias as there are no longer any build changes in the >> revised fix) >> >> On 04/10/2016 08:16, David Holmes wrote: >>> Hi Martin, >> >>>> All Oracle SPARC LDOMS use the "sun4-cpu" string as the CPU >>>> implementation, therefore I think the best short term fix is to add an >>>> additional match that checks for this string. Long term, this could do >>>> with a re-write, but that is a JDK10 project. >>>> >>>> This updated fix also means there are no build changes. >>>> >>>> Updated the webrev accordingly. >>>> >>>> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ >>> >>> These new changes seem okay to me. >>> >>> Only one nit in src/os_cpu/solaris_sparc/vm/vm_version_solaris_sparc.cpp >>> >>> 467 } >>> 468 } >>> 469 } >>> 470 } >>> >>> the indent on line 469 is wrong. >> >> >> Fixed. Thanks David. I have updated the webrev and the patch attached >> to the bug. >> >> Would anybody be willing to sponsor this changeset? > > I can sponsor, but we still need a second review of this version. > > Thanks, > David > >> Thanks, >> >> Martin >> From erik.helin at oracle.com Mon Oct 10 13:54:36 2016 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 10 Oct 2016 15:54:36 +0200 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> Message-ID: <20161010135436.GB11489@ehelin.jrpg.bea.com> On 2016-09-29, Kim Barrett wrote: > Please review this change to G1 to use klass_or_null_acquire where > needed. > > [Reviewing on hotspot-dev; only GC code is involved, but some non-GC > folks have expressed interest.] > > G1 BOT uses of klass_or_null have all been changed to instead use > klass_or_null_acquire. This isn't necessary for the call in the > revised version of oops_on_card_seq_iterate_careful (discussed below), > but there can be (and are) other callers. The performance impact of > unnecessary barriers in oops_on_card_xxx will hopefully not be too bad > on non-TSO systems; the number reduces to one as the BOT gets filled > in. Rather than splitting or otherwise conditionalizing, for now we > just make it safe. [The CollectedHeap block_start &etc API needs some > TLC in the face of collectors that use klass_or_null therein, since > the answer to a call to block_start could change instantaneously. And > some (all?) of the existing not-GC-specific callers probably should be > using block_start_const rather than block_start.] > > Changed oops_on_card_seq_iterate_careful to use different code paths > when the region is humongous or not. > > The humongous case is simplified because there can only be the one > (potential) object in the region. Use klass_or_null_acquire to > determine whether the object has been "published" or allocation is > still in-progress. The acquire barrier ensures processing the > humongous object is safe. When the klass is NULL the card marking > must be stale and can be ignored. > > In the non-humongous case, we know we're dealing with an old-gen > region. Because of that, we know we don't need to deal with > in-progress allocations here. That allows many simplifications. > > As a result of the changes in oops_on_card_seq_iterate_careful, we now > almost never fail to process the card. The only place where that can > occur is a stale card in a humongous region with an in-progress > allocation, where we can just ignore it. So the only caller, > refine_card, no longer needs to examine the result of the call and > enqueue the card for later reconsideration. I'm not sure this assumption is correct. Due to JDK-8166711 [0], can't we have the following scenario: 1. The compiler (and/or CPU) reorders the loads for HeapRegion::_top and HeapRegion::_type in oops_on_card_seq_iterate_careful so that HeapRegion::_type is read before HeapRegion::_top. 2. There is a stale card pointing into a free region 3. The concurrent refinement thread begin executing oops_on_card_seq_iterate_careful fob the card in step two. 4. The first check is now (due to reordering), if (is_young()). This is false, since HeapRegion::_type is free. 5. A mutator thread now allocates a new object in the above free region, turning it into a young region. The mutator increases HeapRegion::_top, but has not yet finished writing the new object. 6. The concurrent refinement thread now performs the second check, if (mr.is_empty()). Since _top no longer equals bottom, and the stale card might point at the beginning of the region, this check might be false as well. 7. The humongous check will fail since this not a humongous region 8. The code below the humongous must now deal with a not yet "published" object in a young region, but isn't prepared to do that. What do you think? This is very tricky code to analyze, but I think this patch won't work in the above scenario. When JDK-8166711 [0] is solved however, then I do think that this patch will work. The important part for getting this patch to work is to ensure that the two first checks, mr.is_empty() and is_young(), aren't reordered. Thanks, Erik [0]: https://bugs.openjdk.java.net/browse/JDK-8166811 > Note that some of the complexity in refine_card / oops_on_card_xxx > arises from the possibility of "stale" marked cards, especially during > concurrent refinement. These are cards in regions that have been > freed and then reallocated. At present the only source of stale cards > in the concurrent case seems to be HCC eviction. (Eager reclaim of > humongous objects flushes relevant remembered sets (possibly > containing stale entries) for reconsideration during that GC, but > in-GC card clipping to scan_top deals with that.) Doing HCC cleanup > when freeing regions might remove the need for klass_or_null checking > in the humongous case for concurrent refinement, so might be worth > looking into later. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166607 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166607/webrev.01/ > > Testing: > Nightly test equiv. via rbt. (in progress) > Local specjbb2015 (non-perf). > Local jtreg hotspot_all. > From coleen.phillimore at oracle.com Mon Oct 10 17:12:03 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 10 Oct 2016 13:12:03 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <11f3d2d6-acac-fc06-3d03-96728b7905e9@oracle.com> <6bf01692-a37f-6ff2-8a92-27e379143ebe@oracle.com> Message-ID: On 10/10/16 4:28 AM, Mikael Gerdin wrote: > Hi Coleen, > > On 2016-10-07 17:24, Coleen Phillimore wrote: >> Mikael, thanks again ... embedded replies and 2 questions: >> --- cut 8< --- >> There's one more thing, though. >> In SpaceManager::retire_current_chunk the last piece of the current >> allocation chunk is made reusable through a call to >> SpaceManager::deallocate. There are two problems with this: >> >> * retire_current_chunk only calls deallocate for wasted blocks larger >> than the min dictionary size, this means that wasted blocks which >> would fit in SmallBlocks will be wasted instead of reusable. >> >> Yeah. I didn't know if this was worth doing for 12 words or less. >> Do you? > > To be honest: I don't know. > We added the SmallBlocks free list for a reason, I guess, so maybe we > should try a bit harder to conserve metaspace memory. The class metaspace goes through this retire_chunk code too and we *don't* want to create small blocks for class metaspace. Since I tested it this way and fixes the immediate problem, I'd like to leave it as is. > >> >>> * the remaining_words parameter which retire_current_chunk passes to >>> deallocate is _exact_ and must not be aligned up through >>> get_allocation_word_size. >>> >> >> Since it's already greater than the allocation_word_size, this isn't a >> problem. > > Well, if in a 32 bit VM a deallocation is done for an object of 13 > words then get_allocation_word_size will align up by 8 to 14 and the > block returned could be too large. > I just realized that this cannot happen since allocations are always > aligned up by 8, but I wanted to share my reasoning at least. > That's something I forget about is 32 bit aligns allocations to 8. I think that's all the more reason I want retire_current_chunk() to only return >12 words. >> >>> >>> Also, >>> are you sure about logging the output of ClassLoaderData::dump on the >>> info level? Perhaps logging the cld dump on the debug level is a good >>> trade-off? It looks like the trace level causes the VSM to print the >>> entire free lists to the log so that's probably a bit much. >> >> It is a lot of stuff I didn't want to see. I could change it back to >> trace though because the thing I really wanted to see in >> MetaspaceAux::dump() is guarded by trace >> >> if (log_is_enabled(Trace, gc, metaspace, freelist)) { >> if (block_freelists() != NULL) block_freelists()->print_on(out); >> } >> >> It seems like debug would be a better level though for both. > > Yeah, I think that's a good trade-off. Thanks, I made that change. Coleen > > /Mikael > >>> >>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>>>> >>>>>> >>>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>>> object double free"? >>>>> >>>>> Yes. >>>>>> Is the problem that when a method was freed its annotations were >>>>>> deallocated as well? Could the annotations fields in the "old" >>>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>>> could be kept or is that just needless complexity? >>>>> >>>>> I'd rather copy the annotations, because I don't know how the old >>>>> method, which could still be running, might use the annotations. I >>>>> don't want to mess with that but I see your point. >>> >>> That's fine I think, I wouldn't want to mess around with it either. I >>> just wanted to understand the reasoning behind the change, thanks for >>> confirming my suspicion. >>> >> >> thanks, >> Coleen >> >>> /Mikael >>> >>>>> >>>>> Coleen >>>>> >>>>>> >>>>>> Thanks >>>>>> /Mikael >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen and Jon >>>>> >>>> >> From kim.barrett at oracle.com Mon Oct 10 19:10:52 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Oct 2016 15:10:52 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <20161010135436.GB11489@ehelin.jrpg.bea.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> Message-ID: > On Oct 10, 2016, at 9:54 AM, Erik Helin wrote: > > On 2016-09-29, Kim Barrett wrote: >> Please review this change to G1 to use klass_or_null_acquire where >> needed. >> >> [?] >> >> Changed oops_on_card_seq_iterate_careful to use different code paths >> when the region is humongous or not. >> >> The humongous case is simplified because there can only be the one >> (potential) object in the region. Use klass_or_null_acquire to >> determine whether the object has been "published" or allocation is >> still in-progress. The acquire barrier ensures processing the >> humongous object is safe. When the klass is NULL the card marking >> must be stale and can be ignored. >> >> In the non-humongous case, we know we're dealing with an old-gen >> region. Because of that, we know we don't need to deal with >> in-progress allocations here. That allows many simplifications. >> >> As a result of the changes in oops_on_card_seq_iterate_careful, we now >> almost never fail to process the card. The only place where that can >> occur is a stale card in a humongous region with an in-progress >> allocation, where we can just ignore it. So the only caller, >> refine_card, no longer needs to examine the result of the call and >> enqueue the card for later reconsideration. > > I'm not sure this assumption is correct. Due to JDK-8166711 [0], can't > we have the following scenario: > > [?] > What do you think? This is very tricky code to analyze, but I think this > patch won't work in the above scenario. > > When JDK-8166711 [0] is solved however, then I do think that this patch > will work. The important part for getting this patch to work is to ensure > that the two first checks, mr.is_empty() and is_young(), aren't > reordered. I think the existing code can also fail in such a JDK-8166711 scenario. I don?t think my proposed changes make that situation any worse, and will enable changes I think can be made to fix 8166711 that will both simplify things further and might not require any additional barriers. Note that 8166711 is the next thing I plan to work on. I could add a loadload barrier as a temporary stopgap; that didn?t seem necessary since the failures are pretty low probability (not sure we?ve ever seen one) and I?m going to work on what I think will be a better fix for that problem next. From coleen.phillimore at oracle.com Mon Oct 10 19:29:01 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 10 Oct 2016 15:29:01 -0400 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: <14a5210f9626419db46ee453159bb0f0@DEWDFE13DE50.global.corp.sap> References: <7b9201ce-a417-e489-7b46-c0488e8fda8f@oracle.com> <14a5210f9626419db46ee453159bb0f0@DEWDFE13DE50.global.corp.sap> Message-ID: <73a7011e-5715-33f7-733c-5e2f90ec2ed2@oracle.com> Yes, this change seems fine. http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr04/src/share/vm/runtime/globals.hpp.udiff.html It might make sense to have CodeCacheSegmentSize develop_pd() though, since I can't really read this expression with all the macros in it. Your choice though. Coleen On 10/10/16 2:38 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > can I consider this reviewed? I removed the change to mutex. > http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr04/ > > Thanks and best regards, > Goetz. > >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> Behalf Of Coleen Phillimore >> Sent: Dienstag, 27. September 2016 20:30 >> To: hotspot-dev at openjdk.java.net >> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. >> >> >> Hi, I'm surprised there aren't more changes, probably because of the >> #include work that was done by you previously. Which is great. >> >> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/hotspot.wr01/src/share/vm/runtime/mutex.hpp.udiff.html >> >> Can you describe this change? This makes all of the mutex's take a lot >> of space, and some don't have very long names. Was this added to avoid >> false sharing as the comment implies or to support longer names in >> monitors? >> >> thanks, >> Coleen >> >> >> >> On 9/27/16 1:57 PM, Volker Simonis wrote: >>> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes >> wrote: >>>> Hi Goetz, >>>> >>>> I see a change not related directly to S390 ie change from ARM to ARM32 >> in >>>> src/os/linux/vm/os_linux.cpp >>>> >>> The change looks a little confusing because Goetz reordered the ifdef >>> cascades alphabetically (which I think is good). >>> >>> Besides that, the only real change not related to s390 is indeed the >>> change from ARM to ARM32 which happend two times in the file. >>> >>> @Goetz: have you done this intentionally? >>> >>>> It will be a while before I can go through this in any detail. >>>> >>>> David >>>> >>>> >>>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> This change is part of the s390 port. It contains some basic adaptions >>>>> needed for a full hotspot port for linux s390x. >>>>> >>>>> It defines the required macros, platform names and includes. >>>>> >>>>> The s390 port calles CodeCache::contains() in current_frame(), which is >>>>> used in NMT. As NMT already collects stack traces before the CodeCache >> is >>>>> initialized, contains() needs a check for this. >>>>> >>>>> Wherever a row of platforms are listed, I sorted them alphabetically. >>>>> >>>>> The jdk requires the file jvm.cfg. >>>>> >>>>> Please review. I please need a sponsor. >>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/hotspot.wr01/ >>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> From goetz.lindenmaier at sap.com Tue Oct 11 06:19:12 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 11 Oct 2016 06:19:12 +0000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: <73a7011e-5715-33f7-733c-5e2f90ec2ed2@oracle.com> References: <7b9201ce-a417-e489-7b46-c0488e8fda8f@oracle.com> <14a5210f9626419db46ee453159bb0f0@DEWDFE13DE50.global.corp.sap> <73a7011e-5715-33f7-733c-5e2f90ec2ed2@oracle.com> Message-ID: <5a93a1a402dd4094a0f8733c08ebb0f6@DEWDFE13DE50.global.corp.sap> Hi Coleen, your're right, this is the ideal candidate for a _pd flag: http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr05 Thanks for reviewing, Goetz. > -----Original Message----- > From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] > Sent: Montag, 10. Oktober 2016 21:29 > To: Lindenmaier, Goetz ; hotspot- > dev at openjdk.java.net > Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > > > Yes, this change seems fine. > > http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr04/src/share/vm/runtime/globals.hpp.udiff.html > > It might make sense to have CodeCacheSegmentSize develop_pd() though, > since I can't really read this expression with all the macros in it. > Your choice though. > > Coleen > > On 10/10/16 2:38 AM, Lindenmaier, Goetz wrote: > > Hi Coleen, > > > > can I consider this reviewed? I removed the change to mutex. > > http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr04/ > > > > Thanks and best regards, > > Goetz. > > > >> -----Original Message----- > >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > >> Behalf Of Coleen Phillimore > >> Sent: Dienstag, 27. September 2016 20:30 > >> To: hotspot-dev at openjdk.java.net > >> Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > >> > >> > >> Hi, I'm surprised there aren't more changes, probably because of the > >> #include work that was done by you previously. Which is great. > >> > >> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >> basic_s390/hotspot.wr01/src/share/vm/runtime/mutex.hpp.udiff.html > >> > >> Can you describe this change? This makes all of the mutex's take a lot > >> of space, and some don't have very long names. Was this added to avoid > >> false sharing as the comment implies or to support longer names in > >> monitors? > >> > >> thanks, > >> Coleen > >> > >> > >> > >> On 9/27/16 1:57 PM, Volker Simonis wrote: > >>> On Fri, Sep 23, 2016 at 8:11 AM, David Holmes > > >> wrote: > >>>> Hi Goetz, > >>>> > >>>> I see a change not related directly to S390 ie change from ARM to > ARM32 > >> in > >>>> src/os/linux/vm/os_linux.cpp > >>>> > >>> The change looks a little confusing because Goetz reordered the ifdef > >>> cascades alphabetically (which I think is good). > >>> > >>> Besides that, the only real change not related to s390 is indeed the > >>> change from ARM to ARM32 which happend two times in the file. > >>> > >>> @Goetz: have you done this intentionally? > >>> > >>>> It will be a while before I can go through this in any detail. > >>>> > >>>> David > >>>> > >>>> > >>>> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: > >>>>> Hi, > >>>>> > >>>>> This change is part of the s390 port. It contains some basic adaptions > >>>>> needed for a full hotspot port for linux s390x. > >>>>> > >>>>> It defines the required macros, platform names and includes. > >>>>> > >>>>> The s390 port calles CodeCache::contains() in current_frame(), which > is > >>>>> used in NMT. As NMT already collects stack traces before the > CodeCache > >> is > >>>>> initialized, contains() needs a check for this. > >>>>> > >>>>> Wherever a row of platforms are listed, I sorted them alphabetically. > >>>>> > >>>>> The jdk requires the file jvm.cfg. > >>>>> > >>>>> Please review. I please need a sponsor. > >>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >> basic_s390/hotspot.wr01/ > >>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/jdk.wr01/ > >>>>> > >>>>> Best regards, > >>>>> Goetz. > >>>>> From mikael.gerdin at oracle.com Tue Oct 11 07:16:05 2016 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 11 Oct 2016 09:16:05 +0200 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <11f3d2d6-acac-fc06-3d03-96728b7905e9@oracle.com> <6bf01692-a37f-6ff2-8a92-27e379143ebe@oracle.com> Message-ID: <1a18ed92-fcf8-fea2-5a1a-a137145456b9@oracle.com> Hi Coleen, On 2016-10-10 19:12, Coleen Phillimore wrote: > > > > On 10/10/16 4:28 AM, Mikael Gerdin wrote: >> Hi Coleen, >> >> On 2016-10-07 17:24, Coleen Phillimore wrote: >>> Mikael, thanks again ... embedded replies and 2 questions: >>> > > --- cut 8< --- >>> There's one more thing, though. >>> In SpaceManager::retire_current_chunk the last piece of the current >>> allocation chunk is made reusable through a call to >>> SpaceManager::deallocate. There are two problems with this: >>> >>> * retire_current_chunk only calls deallocate for wasted blocks larger >>> than the min dictionary size, this means that wasted blocks which >>> would fit in SmallBlocks will be wasted instead of reusable. >>> >>> Yeah. I didn't know if this was worth doing for 12 words or less. >>> Do you? >> >> To be honest: I don't know. >> We added the SmallBlocks free list for a reason, I guess, so maybe we >> should try a bit harder to conserve metaspace memory. > > The class metaspace goes through this retire_chunk code too and we > *don't* want to create small blocks for class metaspace. Since I tested > it this way and fixes the immediate problem, I'd like to leave it as is. Ok. >> >>> >>>> * the remaining_words parameter which retire_current_chunk passes to >>>> deallocate is _exact_ and must not be aligned up through >>>> get_allocation_word_size. >>>> >>> >>> Since it's already greater than the allocation_word_size, this isn't a >>> problem. >> >> Well, if in a 32 bit VM a deallocation is done for an object of 13 >> words then get_allocation_word_size will align up by 8 to 14 and the >> block returned could be too large. >> I just realized that this cannot happen since allocations are always >> aligned up by 8, but I wanted to share my reasoning at least. >> > > That's something I forget about is 32 bit aligns allocations to 8. I > think that's all the more reason I want retire_current_chunk() to only > return >12 words. Ok >>> >>>> >>>> Also, >>>> are you sure about logging the output of ClassLoaderData::dump on the >>>> info level? Perhaps logging the cld dump on the debug level is a good >>>> trade-off? It looks like the trace level causes the VSM to print the >>>> entire free lists to the log so that's probably a bit much. >>> >>> It is a lot of stuff I didn't want to see. I could change it back to >>> trace though because the thing I really wanted to see in >>> MetaspaceAux::dump() is guarded by trace >>> >>> if (log_is_enabled(Trace, gc, metaspace, freelist)) { >>> if (block_freelists() != NULL) block_freelists()->print_on(out); >>> } >>> >>> It seems like debug would be a better level though for both. >> >> Yeah, I think that's a good trade-off. > > Thanks, I made that change. Then I'm all done, you can add me as a Reviewer. Thanks /Mikael > > Coleen > >> >> /Mikael >> >>>> >>>> >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>>>> >>>>>>> >>>>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>>>> object double free"? >>>>>> >>>>>> Yes. >>>>>>> Is the problem that when a method was freed its annotations were >>>>>>> deallocated as well? Could the annotations fields in the "old" >>>>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>>>> could be kept or is that just needless complexity? >>>>>> >>>>>> I'd rather copy the annotations, because I don't know how the old >>>>>> method, which could still be running, might use the annotations. I >>>>>> don't want to mess with that but I see your point. >>>> >>>> That's fine I think, I wouldn't want to mess around with it either. I >>>> just wanted to understand the reasoning behind the change, thanks for >>>> confirming my suspicion. >>>> >>> >>> thanks, >>> Coleen >>> >>>> /Mikael >>>> >>>>>> >>>>>> Coleen >>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> /Mikael >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen and Jon >>>>>> >>>>> >>> > From thomas.stuefe at gmail.com Tue Oct 11 09:13:23 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 11 Oct 2016 11:13:23 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: Hi Markus, just a question whether I understand this correctly: Logging works already from static initialization on, but as no arguments are parsed yet, the default is to print out all warning and error level messages to stderr? If true, this may be potentially confusing, because there will always be a small window at startup where logging does not behave as expected from the command line arguments given. Like, if I redirect log output to a file, the first burst of log output would still go to stderr. This may also confuse tools which grep the output of a VM. Additionally, during initialization one may actually want precise control of the logging system before arguments are parsed, e.g. in os::init(). We have a lot of tracing in the AIX os layer initialization which currently goes to nil because it runs before Xlog argument parsing is done. Both situations could be solved if there were a possibility to specify logging options right from the start, e.g. via a file or an environment variable, which could be parsed in a static initializer without relying on any VM initialization. Kind Regards, Thomas On Mon, Oct 10, 2016 at 11:20 AM, Marcus Larsson wrote: > Updated webrev after feedback: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 > > Incremental: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 > > > > On 10/07/2016 04:26 PM, Marcus Larsson wrote: > >> Hi, >> >> Making another attempt to fix this issue. >> >> Summary: >> The following patch resolves a problem where the VM would crash during >> shutdown due to static log related memory being de-initialized before the >> last use of the logging framework. The solution involves parts of the Nifty >> Counter idiom [0] to control static initialization and de-initialization of >> stdout's and stderr's LogOutputs. Both objects are now allocated using >> placement new, and avoids destructor calls during de-initialization. The >> LogStreamInitializer makes sure both objects are initialized before first >> use. >> >> Because the LogOutput::Stdout/err pointers could no longer be kept in >> LogOutput, I've replaced all usages of them with the new references instead. >> >> The patch includes a regression test for this issue, contributed by >> Michail Chernov. >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8146009 >> >> Testing: >> JPRT testset hotspot, included test on supported platforms. >> >> Thanks, >> Marcus >> >> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter >> > > From mikael.gerdin at oracle.com Tue Oct 11 09:26:56 2016 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 11 Oct 2016 11:26:56 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: References: Message-ID: <3d5b2e54-2124-1716-b087-e23accfd7b5e@oracle.com> Hi Thomas, I have some questions about how your suggested change affects the allocation policy. Currently, once a SpaceManager has reached the _small_chunk_limit it will only allocate medium chunks even if an allocation request could be satisfied with a smaller variant. If metaspace is sufficiently fragmented by small chunks such that a medium chunk cannot be coalesced then we can still fail to satisfy an allocation request with large amounts of free memory, is this something that you have observed? Thanks /Mikael On 2016-10-10 10:56, Thomas St?fe wrote: > Hi all, > > May I have please some feedback for this enhancement proposal? > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > In one very short sentence it proposes a better allocation scheme for > Metaspace Chunks in order to reduce fragmentation and metaspace waste. > > I also added a short presentation which describes the problem and how we > solved it in our VM port. > > https://bugs.openjdk.java.net/secure/attachment/63894/Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf > > Thank you very much! > > Kind Regards, Thomas > > > On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe > wrote: > > Dear all, > > please take a look at this Enhancement Proposal for the metaspace > allocator. I hope these are the right groups for this discussion. > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > Background: > > We at SAP see at times at customer installations OOMs in Metaspace > (usually, with compressed class pointers enabled, in Compressed > Class Space). The VM attempts to allocate metaspace and fails, > hitting the CompressedClassSpaceSize limit. Note that we usually set > the limit lower than the default, typically at 256M. > > When analyzing, we observed that a large part of the metaspace is > indeed free but "locked in" into metaspace chunks of the wrong size: > often we would find a lot of free small chunks, but the allocation > request was for medium chunks, and failed. > > The reason was that if at some point in time a lot of class loaders > were alive, each with only a few small classes loaded. This would > lead to the metaspace being swamped with lots of small chunks. This > is because each SpaceManager first allocates small chunks, only > after a certain amount of allocation requests switches to larger chunks. > > These small chunks are free and wait in the freelist, but cannot be > reused for allocation requests which require larger chunks, even if > they are physically adjacent in the virtual space. > > We (at SAP) added a patch which allows on-the-fly metaspace chunk > merging - to merge multiple adjacent smaller chunk to form a larger > chunk. This, in combination with the reverse direction - splitting a > large chunk to get smaller chunks - partly negates the > "chunks-are-locked-in-into-their-size" limitation and provides for > better reuse of metaspace chunks. It also provides better > defragmentation as well. > > I discussed this fix off-list with Coleen Phillimore and Jon > Masamitsu, and instead of just offering this as a fix, both > recommended to open a JEP for this, because its scope would be > beyond that of a simple fix. > > So here is my first JEP :) I hope it follows the right form. Please, > if you have time, take a look and tell us what you think. > > Thank you, and Kind Regards, > > Thomas St?fe > > > > From marcus.larsson at oracle.com Tue Oct 11 09:41:52 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Tue, 11 Oct 2016 11:41:52 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: Hi Thomas, On 10/11/2016 11:13 AM, Thomas St?fe wrote: > Hi Markus, > > just a question whether I understand this correctly: Logging works > already from static initialization on, but as no arguments are parsed > yet, the default is to print out all warning and error level messages > to stderr? > > If true, this may be potentially confusing, because there will always > be a small window at startup where logging does not behave as expected > from the command line arguments given. Like, if I redirect log output > to a file, the first burst of log output would still go to stderr. > This may also confuse tools which grep the output of a VM. Yes you're right, it's a limitation of UL at the moment. Logging before argument parsing means you can only use warning and error level, and can only expect to see it on stdout. > > Additionally, during initialization one may actually want precise > control of the logging system before arguments are parsed, e.g. in > os::init(). We have a lot of tracing in the AIX os layer > initialization which currently goes to nil because it runs before Xlog > argument parsing is done. > > Both situations could be solved if there were a possibility to specify > logging options right from the start, e.g. via a file or an > environment variable, which could be parsed in a static initializer > without relying on any VM initialization. It's either that or we implement some early logging buffering that saves all logging happening before UL initialization and logs it after UL is fully initialized. We haven't had the need for that early logging previously, but given that you seem to require it, it seems like a worthwhile enhancement of UL now. Thanks, Marcus > > Kind Regards, Thomas > > > > On Mon, Oct 10, 2016 at 11:20 AM, Marcus Larsson > > wrote: > > Updated webrev after feedback: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 > > > Incremental: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 > > > > > On 10/07/2016 04:26 PM, Marcus Larsson wrote: > > Hi, > > Making another attempt to fix this issue. > > Summary: > The following patch resolves a problem where the VM would > crash during shutdown due to static log related memory being > de-initialized before the last use of the logging framework. > The solution involves parts of the Nifty Counter idiom [0] to > control static initialization and de-initialization of > stdout's and stderr's LogOutputs. Both objects are now > allocated using placement new, and avoids destructor calls > during de-initialization. The LogStreamInitializer makes sure > both objects are initialized before first use. > > Because the LogOutput::Stdout/err pointers could no longer be > kept in LogOutput, I've replaced all usages of them with the > new references instead. > > The patch includes a regression test for this issue, > contributed by Michail Chernov. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 > > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8146009 > > > Testing: > JPRT testset hotspot, included test on supported platforms. > > Thanks, > Marcus > > [0] > https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter > > > > From thomas.stuefe at gmail.com Tue Oct 11 09:53:10 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 11 Oct 2016 11:53:10 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: Hi Marcus, On Tue, Oct 11, 2016 at 11:41 AM, Marcus Larsson wrote: > Hi Thomas, > > On 10/11/2016 11:13 AM, Thomas St?fe wrote: > > Hi Markus, > > just a question whether I understand this correctly: Logging works already > from static initialization on, but as no arguments are parsed yet, the > default is to print out all warning and error level messages to stderr? > > If true, this may be potentially confusing, because there will always be a > small window at startup where logging does not behave as expected from the > command line arguments given. Like, if I redirect log output to a file, the > first burst of log output would still go to stderr. This may also confuse > tools which grep the output of a VM. > > > Yes you're right, it's a limitation of UL at the moment. Logging before > argument parsing means you can only use warning and error level, and can > only expect to see it on stdout. > > > Additionally, during initialization one may actually want precise control > of the logging system before arguments are parsed, e.g. in os::init(). We > have a lot of tracing in the AIX os layer initialization which currently > goes to nil because it runs before Xlog argument parsing is done. > > Both situations could be solved if there were a possibility to specify > logging options right from the start, e.g. via a file or an environment > variable, which could be parsed in a static initializer without relying on > any VM initialization. > > > It's either that or we implement some early logging buffering that saves > all logging happening before UL initialization and logs it after UL is > fully initialized. We haven't had the need for that early logging > previously, but given that you seem to require it, it seems like a > worthwhile enhancement of UL now. > > Thank you! This would be helpful. I think that buffering is not a good solution. For one, it requires you execute each and every log call and buffer the resulting string, because you do not know yet which of the log calls will be active. This may be cpu and memory intensive, which especially at startup may hurt. Also, there is no guarantee the VM even lives long enough to parse the Xlog arguments to print out the buffered messages. The situations where we use early logging are often situations where the VM crashes early. Kind Regards, Thomas > Thanks, > Marcus > > > Kind Regards, Thomas > > > > On Mon, Oct 10, 2016 at 11:20 AM, Marcus Larsson < > marcus.larsson at oracle.com> wrote: > >> Updated webrev after feedback: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 >> >> Incremental: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 >> >> >> >> On 10/07/2016 04:26 PM, Marcus Larsson wrote: >> >>> Hi, >>> >>> Making another attempt to fix this issue. >>> >>> Summary: >>> The following patch resolves a problem where the VM would crash during >>> shutdown due to static log related memory being de-initialized before the >>> last use of the logging framework. The solution involves parts of the Nifty >>> Counter idiom [0] to control static initialization and de-initialization of >>> stdout's and stderr's LogOutputs. Both objects are now allocated using >>> placement new, and avoids destructor calls during de-initialization. The >>> LogStreamInitializer makes sure both objects are initialized before first >>> use. >>> >>> Because the LogOutput::Stdout/err pointers could no longer be kept in >>> LogOutput, I've replaced all usages of them with the new references instead. >>> >>> The patch includes a regression test for this issue, contributed by >>> Michail Chernov. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8146009 >>> >>> Testing: >>> JPRT testset hotspot, included test on supported platforms. >>> >>> Thanks, >>> Marcus >>> >>> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter >>> >> >> > > From ningsheng.jian at linaro.org Tue Oct 11 09:56:13 2016 From: ningsheng.jian at linaro.org (Ningsheng Jian) Date: Tue, 11 Oct 2016 17:56:13 +0800 Subject: [aarch64-port-dev ] RFR: AArch64: SEGV in stub code cipherBlockChaining_decryptAESCrypt In-Reply-To: <92c29040-7a82-3b73-7079-6b23b781dcd2@redhat.com> References: <92c29040-7a82-3b73-7079-6b23b781dcd2@redhat.com> Message-ID: Hi, Any comments on this? I think we need a JIRA entry to track it. Thanks, Ningsheng On 21 September 2016 at 17:35, Andrew Dinn wrote: > On 20/09/16 11:09, Ningsheng Jian wrote: >> Jtreg test jdk/test/com/sun/crypto/provider/Cipher/CTS/CTSMode.java >> failed with SIGSEGV on option "-Xcomp -XX:-TieredCompilation". >> >> The crash happens at stub code cipherBlockChaining_decryptAESCrypt. >> >> Checking the jdk source code CipherTextStealing.decryptFinal and its >> callee CipherBlockChaining.decrypt/implDecrypt, we can see that >> cipherLen which is passed to the stub code can be 0. The unexpected >> len_reg value makes the load from invalid address. >> >> The following patch could fix this issue: >> >> http://people.linaro.org/~ningsheng.jian/webrev/cbc-stub-fix/webrev.00/ >> >> It aligns with the Java code implementation and passed JTreg tests. To >> make code consistent, I also update the encrypt part. >> >> Could someone please help to take a look at it? > > I think this patch looks like it will suffice to fix the AArch64 code. > However, I don't understand why this is only an AArch64 issue. For > example, it looks to me as if the x86_64 code is also susceptible to the > same problem should input value 0 be passed in len_reg (c_rarg4). So, > this might need fixing for all archs. Alternatively, it might be better > fixed in the jdk (Java) code. > > Could someone from the hotspot compiler/runtime team confirm: > > i) whether an input len_reg value of zero will cause a problem on x86 > (or, indeed, other archs)? > > ii) whether that case should be caught in Java code or stub code? > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From marcus.larsson at oracle.com Tue Oct 11 10:12:01 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Tue, 11 Oct 2016 12:12:01 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: Hi, On 10/11/2016 11:53 AM, Thomas St?fe wrote: > Hi Marcus, > > On Tue, Oct 11, 2016 at 11:41 AM, Marcus Larsson > > wrote: > > Hi Thomas, > > > On 10/11/2016 11:13 AM, Thomas St?fe wrote: >> Hi Markus, >> >> just a question whether I understand this correctly: Logging >> works already from static initialization on, but as no arguments >> are parsed yet, the default is to print out all warning and error >> level messages to stderr? >> >> If true, this may be potentially confusing, because there will >> always be a small window at startup where logging does not behave >> as expected from the command line arguments given. Like, if I >> redirect log output to a file, the first burst of log output >> would still go to stderr. This may also confuse tools which grep >> the output of a VM. > > Yes you're right, it's a limitation of UL at the moment. Logging > before argument parsing means you can only use warning and error > level, and can only expect to see it on stdout. > >> >> Additionally, during initialization one may actually want precise >> control of the logging system before arguments are parsed, e.g. >> in os::init(). We have a lot of tracing in the AIX os layer >> initialization which currently goes to nil because it runs before >> Xlog argument parsing is done. >> >> Both situations could be solved if there were a possibility to >> specify logging options right from the start, e.g. via a file or >> an environment variable, which could be parsed in a static >> initializer without relying on any VM initialization. > > It's either that or we implement some early logging buffering that > saves all logging happening before UL initialization and logs it > after UL is fully initialized. We haven't had the need for that > early logging previously, but given that you seem to require it, > it seems like a worthwhile enhancement of UL now. > > > Thank you! This would be helpful. > > I think that buffering is not a good solution. For one, it requires > you execute each and every log call and buffer the resulting string, > because you do not know yet which of the log calls will be active. > This may be cpu and memory intensive, which especially at startup may > hurt. That's a fair point. Although I don't think we should have enough such logging (before UL init) to make a significant difference. At least I hope not. Some of it could perhaps be debug-only logging (log_develop... etc)? > > Also, there is no guarantee the VM even lives long enough to parse the > Xlog arguments to print out the buffered messages. The situations > where we use early logging are often situations where the VM crashes > early. Unprocessed log buffers could possibly be printed in the hs_err output to avoid logging losses when the VM crashes early. Having an additional early parsing step seems more complicated to me (at least at first glance). Thanks, Marcus > > Kind Regards, Thomas > > Thanks, > Marcus > >> >> Kind Regards, Thomas >> >> >> >> On Mon, Oct 10, 2016 at 11:20 AM, Marcus Larsson >> > wrote: >> >> Updated webrev after feedback: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 >> >> >> Incremental: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 >> >> >> >> >> On 10/07/2016 04:26 PM, Marcus Larsson wrote: >> >> Hi, >> >> Making another attempt to fix this issue. >> >> Summary: >> The following patch resolves a problem where the VM would >> crash during shutdown due to static log related memory >> being de-initialized before the last use of the logging >> framework. The solution involves parts of the Nifty >> Counter idiom [0] to control static initialization and >> de-initialization of stdout's and stderr's LogOutputs. >> Both objects are now allocated using placement new, and >> avoids destructor calls during de-initialization. The >> LogStreamInitializer makes sure both objects are >> initialized before first use. >> >> Because the LogOutput::Stdout/err pointers could no >> longer be kept in LogOutput, I've replaced all usages of >> them with the new references instead. >> >> The patch includes a regression test for this issue, >> contributed by Michail Chernov. >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >> >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8146009 >> >> >> Testing: >> JPRT testset hotspot, included test on supported platforms. >> >> Thanks, >> Marcus >> >> [0] >> https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter >> >> >> >> > > From thomas.stuefe at gmail.com Tue Oct 11 12:42:06 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 11 Oct 2016 14:42:06 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: <3d5b2e54-2124-1716-b087-e23accfd7b5e@oracle.com> References: <3d5b2e54-2124-1716-b087-e23accfd7b5e@oracle.com> Message-ID: Hi Mikael, On Tue, Oct 11, 2016 at 11:26 AM, Mikael Gerdin wrote: > Hi Thomas, > > I have some questions about how your suggested change affects the > allocation policy. > > Currently, once a SpaceManager has reached the _small_chunk_limit it will > only allocate medium chunks even if an allocation request could be > satisfied with a smaller variant. > > If metaspace is sufficiently fragmented by small chunks such that a medium > chunk cannot be coalesced then we can still fail to satisfy an allocation > request with large amounts of free memory, is this something that you have > observed? > I did not see this in my tests, but in theory this is still possible even with my patch. If free and in-use small chunks are tightly interspersed, this would of course still prevent merging them to medium chunks. I think to prevent this the chunk would have to be movable, so that they can be compacted. Or, as a quick fix, the size ratio between small and medium chunks could be made smaller, so that the chance that one stray in-use small chunk prevents its free neighbors from being coalesced is smaller. In the scenarios we observed we saw mostly free small chunks resting in the freelist, because the class loaders which allocated them were all unloaded. I try to understand how probable the scenario is you describe: in order to fill up metaspace with small chunks - so that allocation of new medium chunks fails - and to intermix free and in-use small chunks such that coalescation to a medium chunk is not possible but there are still enough free small "uncoalescable" chunks to matter (what you call "large amounts of free memory"), one would have to have a quite "unlucky" allocation pattern. I think it is certainly possible, but less probable than the scenario this patch tries to address: that metaspace was allocated in many small chunks by class loaders which are now all dead and the small chunks are all free. Kind Regards, Thomas > Thanks > /Mikael > > On 2016-10-10 10:56, Thomas St?fe wrote: > >> Hi all, >> >> May I have please some feedback for this enhancement proposal? >> >> https://bugs.openjdk.java.net/browse/JDK-8166690 >> >> >> >> In one very short sentence it proposes a better allocation scheme for >> Metaspace Chunks in order to reduce fragmentation and metaspace waste. >> >> I also added a short presentation which describes the problem and how we >> solved it in our VM port. >> >> https://bugs.openjdk.java.net/secure/attachment/63894/Metasp >> ace%20Coalescation%20in%20the%20SAP%20JVM.pdf >> >> Thank you very much! >> >> Kind Regards, Thomas >> >> >> On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe > > wrote: >> >> Dear all, >> >> please take a look at this Enhancement Proposal for the metaspace >> allocator. I hope these are the right groups for this discussion. >> >> https://bugs.openjdk.java.net/browse/JDK-8166690 >> >> >> Background: >> >> We at SAP see at times at customer installations OOMs in Metaspace >> (usually, with compressed class pointers enabled, in Compressed >> Class Space). The VM attempts to allocate metaspace and fails, >> hitting the CompressedClassSpaceSize limit. Note that we usually set >> the limit lower than the default, typically at 256M. >> >> When analyzing, we observed that a large part of the metaspace is >> indeed free but "locked in" into metaspace chunks of the wrong size: >> often we would find a lot of free small chunks, but the allocation >> request was for medium chunks, and failed. >> >> The reason was that if at some point in time a lot of class loaders >> were alive, each with only a few small classes loaded. This would >> lead to the metaspace being swamped with lots of small chunks. This >> is because each SpaceManager first allocates small chunks, only >> after a certain amount of allocation requests switches to larger >> chunks. >> >> These small chunks are free and wait in the freelist, but cannot be >> reused for allocation requests which require larger chunks, even if >> they are physically adjacent in the virtual space. >> >> We (at SAP) added a patch which allows on-the-fly metaspace chunk >> merging - to merge multiple adjacent smaller chunk to form a larger >> chunk. This, in combination with the reverse direction - splitting a >> large chunk to get smaller chunks - partly negates the >> "chunks-are-locked-in-into-their-size" limitation and provides for >> better reuse of metaspace chunks. It also provides better >> defragmentation as well. >> >> I discussed this fix off-list with Coleen Phillimore and Jon >> Masamitsu, and instead of just offering this as a fix, both >> recommended to open a JEP for this, because its scope would be >> beyond that of a simple fix. >> >> So here is my first JEP :) I hope it follows the right form. Please, >> if you have time, take a look and tell us what you think. >> >> Thank you, and Kind Regards, >> >> Thomas St?fe >> >> >> >> >> From aph at redhat.com Tue Oct 11 12:46:01 2016 From: aph at redhat.com (Andrew Haley) Date: Tue, 11 Oct 2016 13:46:01 +0100 Subject: [aarch64-port-dev ] RFR: AArch64: SEGV in stub code cipherBlockChaining_decryptAESCrypt In-Reply-To: References: <92c29040-7a82-3b73-7079-6b23b781dcd2@redhat.com> Message-ID: <227edb20-660f-8bed-dd2a-2d29b607dab0@redhat.com> On 11/10/16 10:56, Ningsheng Jian wrote: > Hi, > > Any comments on this? I think we need a JIRA entry to track it. I'm looking at it. Andrew. From thomas.stuefe at gmail.com Tue Oct 11 12:52:29 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 11 Oct 2016 14:52:29 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: Hi Marcus, On Tue, Oct 11, 2016 at 12:12 PM, Marcus Larsson wrote: > Hi, > > On 10/11/2016 11:53 AM, Thomas St?fe wrote: > > Hi Marcus, > > On Tue, Oct 11, 2016 at 11:41 AM, Marcus Larsson < > marcus.larsson at oracle.com> wrote: > >> Hi Thomas, >> >> On 10/11/2016 11:13 AM, Thomas St?fe wrote: >> >> Hi Markus, >> >> just a question whether I understand this correctly: Logging works >> already from static initialization on, but as no arguments are parsed yet, >> the default is to print out all warning and error level messages to stderr? >> >> If true, this may be potentially confusing, because there will always be >> a small window at startup where logging does not behave as expected from >> the command line arguments given. Like, if I redirect log output to a file, >> the first burst of log output would still go to stderr. This may also >> confuse tools which grep the output of a VM. >> >> >> Yes you're right, it's a limitation of UL at the moment. Logging before >> argument parsing means you can only use warning and error level, and can >> only expect to see it on stdout. >> >> >> Additionally, during initialization one may actually want precise control >> of the logging system before arguments are parsed, e.g. in os::init(). We >> have a lot of tracing in the AIX os layer initialization which currently >> goes to nil because it runs before Xlog argument parsing is done. >> >> Both situations could be solved if there were a possibility to specify >> logging options right from the start, e.g. via a file or an environment >> variable, which could be parsed in a static initializer without relying on >> any VM initialization. >> >> >> It's either that or we implement some early logging buffering that saves >> all logging happening before UL initialization and logs it after UL is >> fully initialized. We haven't had the need for that early logging >> previously, but given that you seem to require it, it seems like a >> worthwhile enhancement of UL now. >> >> > Thank you! This would be helpful. > > I think that buffering is not a good solution. For one, it requires you > execute each and every log call and buffer the resulting string, because > you do not know yet which of the log calls will be active. This may be cpu > and memory intensive, which especially at startup may hurt. > > > That's a fair point. Although I don't think we should have enough such > logging (before UL init) to make a significant difference. > You sometimes do not know, it could be a log call in a small inconspicuous function which gets called a lot more than the original author intended. Plus, you still have the cost of retrieving the information you try to print, which you can completely avoid if you know the log site is not active. > At least I hope not. Some of it could perhaps be debug-only logging > (log_develop... etc)? > > Our logging usually does not differ between develop and product build. Simply because logging is valuable for our support people at customer sites, and they usually have product builds. > > Also, there is no guarantee the VM even lives long enough to parse the > Xlog arguments to print out the buffered messages. The situations where we > use early logging are often situations where the VM crashes early. > > > Unprocessed log buffers could possibly be printed in the hs_err output to > avoid logging losses when the VM crashes early. Having an additional early > parsing step seems more complicated to me (at least at first glance). > If argument parsing for -Xlog could be factored out, it could be called for arguments from various sources - including the hypothetical environment variable - from a very early point on. But I know, every patch is easy from a high level :) ..Thomas > > > Thanks, > Marcus > > > Kind Regards, Thomas > > > >> Thanks, >> Marcus >> >> >> Kind Regards, Thomas >> >> >> >> On Mon, Oct 10, 2016 at 11:20 AM, Marcus Larsson < >> marcus.larsson at oracle.com> wrote: >> >>> Updated webrev after feedback: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 >>> >>> Incremental: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 >>> >>> >>> >>> On 10/07/2016 04:26 PM, Marcus Larsson wrote: >>> >>>> Hi, >>>> >>>> Making another attempt to fix this issue. >>>> >>>> Summary: >>>> The following patch resolves a problem where the VM would crash during >>>> shutdown due to static log related memory being de-initialized before the >>>> last use of the logging framework. The solution involves parts of the Nifty >>>> Counter idiom [0] to control static initialization and de-initialization of >>>> stdout's and stderr's LogOutputs. Both objects are now allocated using >>>> placement new, and avoids destructor calls during de-initialization. The >>>> LogStreamInitializer makes sure both objects are initialized before first >>>> use. >>>> >>>> Because the LogOutput::Stdout/err pointers could no longer be kept in >>>> LogOutput, I've replaced all usages of them with the new references instead. >>>> >>>> The patch includes a regression test for this issue, contributed by >>>> Michail Chernov. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8146009 >>>> >>>> Testing: >>>> JPRT testset hotspot, included test on supported platforms. >>>> >>>> Thanks, >>>> Marcus >>>> >>>> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter >>>> >>> >>> >> >> > > From volker.simonis at gmail.com Tue Oct 11 14:04:46 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 11 Oct 2016 16:04:46 +0200 Subject: jdk9/hs is CLOSED In-Reply-To: References: Message-ID: Hi Jesper, is jdk9/hs still closed? And does this apply to the whole forest or just to jdk9/hs/hotspot? If it is still closed, when do expect it will be opened again? Thanks, Volker On Thu, Oct 6, 2016 at 3:24 PM, Jesper Wilhelmsson wrote: > Hi, > > Due to issues with our nightly testing we have decided to close jdk9/hs for > new changes. Please do not push anything to hs until further notice. > > Thanks, > /Jesper From jesper.wilhelmsson at oracle.com Tue Oct 11 14:22:36 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 11 Oct 2016 16:22:36 +0200 Subject: jdk9/hs is CLOSED In-Reply-To: References: Message-ID: <64a3232c-4738-5cb1-570f-03ba962b31ef@oracle.com> Hi Volker, Yes, jdk9/hs is still closed. There are issues with our nightly testing that are blocking us. I have not heard any timeline for when the issues will be resolved. Critical changes are approved on a case by case basis. If you are pushing to parts of the JDK that are not covered by the nightly tests I think we can approve these changes. /Jesper Den 11/10/16 kl. 16:04, skrev Volker Simonis: > Hi Jesper, > > is jdk9/hs still closed? And does this apply to the whole forest or > just to jdk9/hs/hotspot? > If it is still closed, when do expect it will be opened again? > > Thanks, > Volker > > > > On Thu, Oct 6, 2016 at 3:24 PM, Jesper Wilhelmsson > wrote: >> Hi, >> >> Due to issues with our nightly testing we have decided to close jdk9/hs for >> new changes. Please do not push anything to hs until further notice. >> >> Thanks, >> /Jesper From volker.simonis at gmail.com Tue Oct 11 14:33:03 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 11 Oct 2016 16:33:03 +0200 Subject: jdk9/hs is CLOSED In-Reply-To: <64a3232c-4738-5cb1-570f-03ba962b31ef@oracle.com> References: <64a3232c-4738-5cb1-570f-03ba962b31ef@oracle.com> Message-ID: Hi Jesper, thanks for the prompt response. I have "8166801: [s390] Add jvm.cfg file for Linux/s390x" in the queue: http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166801/ which shouldn't do any harm to anybody :) If you don't mind, I'd like to push it to jdk9/hs/jdk before I'm leaving until next week. Please let me know if that would be OK for you? Thank you and best regards, Volker On Tue, Oct 11, 2016 at 4:22 PM, Jesper Wilhelmsson wrote: > Hi Volker, > > Yes, jdk9/hs is still closed. There are issues with our nightly testing that > are blocking us. I have not heard any timeline for when the issues will be > resolved. > > Critical changes are approved on a case by case basis. > > If you are pushing to parts of the JDK that are not covered by the nightly > tests I think we can approve these changes. > /Jesper > > > Den 11/10/16 kl. 16:04, skrev Volker Simonis: > >> Hi Jesper, >> >> is jdk9/hs still closed? And does this apply to the whole forest or >> just to jdk9/hs/hotspot? >> If it is still closed, when do expect it will be opened again? >> >> Thanks, >> Volker >> >> >> >> On Thu, Oct 6, 2016 at 3:24 PM, Jesper Wilhelmsson >> wrote: >>> >>> Hi, >>> >>> Due to issues with our nightly testing we have decided to close jdk9/hs >>> for >>> new changes. Please do not push anything to hs until further notice. >>> >>> Thanks, >>> /Jesper From jesper.wilhelmsson at oracle.com Tue Oct 11 14:52:08 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 11 Oct 2016 16:52:08 +0200 Subject: jdk9/hs is CLOSED In-Reply-To: References: <64a3232c-4738-5cb1-570f-03ba962b31ef@oracle.com> Message-ID: <2dad23af-c93b-6ae9-b396-3c82422dcc5f@oracle.com> Yes, I'm fine with you pushing this change. /Jesper Den 11/10/16 kl. 16:33, skrev Volker Simonis: > Hi Jesper, > > thanks for the prompt response. > > I have "8166801: [s390] Add jvm.cfg file for Linux/s390x" in the queue: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166801/ > > which shouldn't do any harm to anybody :) > > If you don't mind, I'd like to push it to jdk9/hs/jdk before I'm > leaving until next week. > Please let me know if that would be OK for you? > > Thank you and best regards, > Volker > > > On Tue, Oct 11, 2016 at 4:22 PM, Jesper Wilhelmsson > wrote: >> Hi Volker, >> >> Yes, jdk9/hs is still closed. There are issues with our nightly testing that >> are blocking us. I have not heard any timeline for when the issues will be >> resolved. >> >> Critical changes are approved on a case by case basis. >> >> If you are pushing to parts of the JDK that are not covered by the nightly >> tests I think we can approve these changes. >> /Jesper >> >> >> Den 11/10/16 kl. 16:04, skrev Volker Simonis: >> >>> Hi Jesper, >>> >>> is jdk9/hs still closed? And does this apply to the whole forest or >>> just to jdk9/hs/hotspot? >>> If it is still closed, when do expect it will be opened again? >>> >>> Thanks, >>> Volker >>> >>> >>> >>> On Thu, Oct 6, 2016 at 3:24 PM, Jesper Wilhelmsson >>> wrote: >>>> >>>> Hi, >>>> >>>> Due to issues with our nightly testing we have decided to close jdk9/hs >>>> for >>>> new changes. Please do not push anything to hs until further notice. >>>> >>>> Thanks, >>>> /Jesper From volker.simonis at gmail.com Tue Oct 11 15:05:31 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 11 Oct 2016 17:05:31 +0200 Subject: jdk9/hs is CLOSED In-Reply-To: <2dad23af-c93b-6ae9-b396-3c82422dcc5f@oracle.com> References: <64a3232c-4738-5cb1-570f-03ba962b31ef@oracle.com> <2dad23af-c93b-6ae9-b396-3c82422dcc5f@oracle.com> Message-ID: Thanks Jesper! Pushed... On Tue, Oct 11, 2016 at 4:52 PM, Jesper Wilhelmsson wrote: > Yes, I'm fine with you pushing this change. > /Jesper > > > Den 11/10/16 kl. 16:33, skrev Volker Simonis: > >> Hi Jesper, >> >> thanks for the prompt response. >> >> I have "8166801: [s390] Add jvm.cfg file for Linux/s390x" in the queue: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/8166801/ >> >> which shouldn't do any harm to anybody :) >> >> If you don't mind, I'd like to push it to jdk9/hs/jdk before I'm >> leaving until next week. >> Please let me know if that would be OK for you? >> >> Thank you and best regards, >> Volker >> >> >> On Tue, Oct 11, 2016 at 4:22 PM, Jesper Wilhelmsson >> wrote: >>> >>> Hi Volker, >>> >>> Yes, jdk9/hs is still closed. There are issues with our nightly testing >>> that >>> are blocking us. I have not heard any timeline for when the issues will >>> be >>> resolved. >>> >>> Critical changes are approved on a case by case basis. >>> >>> If you are pushing to parts of the JDK that are not covered by the >>> nightly >>> tests I think we can approve these changes. >>> /Jesper >>> >>> >>> Den 11/10/16 kl. 16:04, skrev Volker Simonis: >>> >>>> Hi Jesper, >>>> >>>> is jdk9/hs still closed? And does this apply to the whole forest or >>>> just to jdk9/hs/hotspot? >>>> If it is still closed, when do expect it will be opened again? >>>> >>>> Thanks, >>>> Volker >>>> >>>> >>>> >>>> On Thu, Oct 6, 2016 at 3:24 PM, Jesper Wilhelmsson >>>> wrote: >>>>> >>>>> >>>>> Hi, >>>>> >>>>> Due to issues with our nightly testing we have decided to close jdk9/hs >>>>> for >>>>> new changes. Please do not push anything to hs until further notice. >>>>> >>>>> Thanks, >>>>> /Jesper From coleen.phillimore at oracle.com Tue Oct 11 17:38:23 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Oct 2016 13:38:23 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <1a18ed92-fcf8-fea2-5a1a-a137145456b9@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <49af9f75-ae63-e9e4-5631-a8efe392e4ae@oracle.com> <2e819502-45b9-5e05-549e-5c1c37809b97@oracle.com> <11f3d2d6-acac-fc06-3d03-96728b7905e9@oracle.com> <6bf01692-a37f-6ff2-8a92-27e379143ebe@oracle.com> <1a18ed92-fcf8-fea2-5a1a-a137145456b9@oracle.com> Message-ID: <2a56011e-e6c2-cb73-c8bd-1152a0ead4fd@oracle.com> Mikael, Thank you for reviewing this code. Coleen On 10/11/16 3:16 AM, Mikael Gerdin wrote: > Hi Coleen, > > On 2016-10-10 19:12, Coleen Phillimore wrote: >> >> >> >> On 10/10/16 4:28 AM, Mikael Gerdin wrote: >>> Hi Coleen, >>> >>> On 2016-10-07 17:24, Coleen Phillimore wrote: >>>> Mikael, thanks again ... embedded replies and 2 questions: >>>> >> >> --- cut 8< --- >>>> There's one more thing, though. >>>> In SpaceManager::retire_current_chunk the last piece of the current >>>> allocation chunk is made reusable through a call to >>>> SpaceManager::deallocate. There are two problems with this: >>>> >>>> * retire_current_chunk only calls deallocate for wasted blocks larger >>>> than the min dictionary size, this means that wasted blocks which >>>> would fit in SmallBlocks will be wasted instead of reusable. >>>> >>>> Yeah. I didn't know if this was worth doing for 12 words or less. >>>> Do you? >>> >>> To be honest: I don't know. >>> We added the SmallBlocks free list for a reason, I guess, so maybe we >>> should try a bit harder to conserve metaspace memory. >> >> The class metaspace goes through this retire_chunk code too and we >> *don't* want to create small blocks for class metaspace. Since I tested >> it this way and fixes the immediate problem, I'd like to leave it as is. > > > Ok. > >>> >>>> >>>>> * the remaining_words parameter which retire_current_chunk passes to >>>>> deallocate is _exact_ and must not be aligned up through >>>>> get_allocation_word_size. >>>>> >>>> >>>> Since it's already greater than the allocation_word_size, this isn't a >>>> problem. >>> >>> Well, if in a 32 bit VM a deallocation is done for an object of 13 >>> words then get_allocation_word_size will align up by 8 to 14 and the >>> block returned could be too large. >>> I just realized that this cannot happen since allocations are always >>> aligned up by 8, but I wanted to share my reasoning at least. >>> >> >> That's something I forget about is 32 bit aligns allocations to 8. I >> think that's all the more reason I want retire_current_chunk() to only >> return >12 words. > > Ok > >>>> >>>>> >>>>> Also, >>>>> are you sure about logging the output of ClassLoaderData::dump on the >>>>> info level? Perhaps logging the cld dump on the debug level is a good >>>>> trade-off? It looks like the trace level causes the VSM to print the >>>>> entire free lists to the log so that's probably a bit much. >>>> >>>> It is a lot of stuff I didn't want to see. I could change it back to >>>> trace though because the thing I really wanted to see in >>>> MetaspaceAux::dump() is guarded by trace >>>> >>>> if (log_is_enabled(Trace, gc, metaspace, freelist)) { >>>> if (block_freelists() != NULL) block_freelists()->print_on(out); >>>> } >>>> >>>> It seems like debug would be a better level though for both. >>> >>> Yeah, I think that's a good trade-off. >> >> Thanks, I made that change. > > Then I'm all done, you can add me as a Reviewer. > Thanks > /Mikael > > > >> >> Coleen >> >>> >>> /Mikael >>> >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Are the changes to Method and ConstMethod the "bug fix to small >>>>>>>> object double free"? >>>>>>> >>>>>>> Yes. >>>>>>>> Is the problem that when a method was freed its annotations were >>>>>>>> deallocated as well? Could the annotations fields in the "old" >>>>>>>> ConstMethod be cleared instead so that the old annotation arrays >>>>>>>> could be kept or is that just needless complexity? >>>>>>> >>>>>>> I'd rather copy the annotations, because I don't know how the old >>>>>>> method, which could still be running, might use the annotations. I >>>>>>> don't want to mess with that but I see your point. >>>>> >>>>> That's fine I think, I wouldn't want to mess around with it either. I >>>>> just wanted to understand the reasoning behind the change, thanks for >>>>> confirming my suspicion. >>>>> >>>> >>>> thanks, >>>> Coleen >>>> >>>>> /Mikael >>>>> >>>>>>> >>>>>>> Coleen >>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> /Mikael >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen and Jon >>>>>>> >>>>>> >>>> >> From max.ockner at oracle.com Tue Oct 11 17:44:01 2016 From: max.ockner at oracle.com (Max Ockner) Date: Tue, 11 Oct 2016 13:44:01 -0400 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize Message-ID: <57FD24E1.1020905@oracle.com> Hello, Please Review this small change. http://cr.openjdk.java.net/~mockner/8167446/ https://bugs.openjdk.java.net/browse/JDK-8167446 Added PermSize and MaxPermSize to obsolete flags table. They are recognized but ignored. The bug title says "PermGenSize" and "MaxPermGenSize" but the comment says "PermSize" and MaxPermSize". Which one is correct? Tested both options on java -version. Nothing happens, but I don't get an "Unrecognized ..." error. Thanks, Max From harold.seigel at oracle.com Tue Oct 11 18:31:23 2016 From: harold.seigel at oracle.com (harold seigel) Date: Tue, 11 Oct 2016 14:31:23 -0400 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize In-Reply-To: <57FD24E1.1020905@oracle.com> References: <57FD24E1.1020905@oracle.com> Message-ID: Hi Max, Can you add a small test? It would prevent someone else from removing the options. Thanks, Harold On 10/11/2016 1:44 PM, Max Ockner wrote: > Hello, > Please Review this small change. > > http://cr.openjdk.java.net/~mockner/8167446/ > https://bugs.openjdk.java.net/browse/JDK-8167446 > > Added PermSize and MaxPermSize to obsolete flags table. They are > recognized but ignored. > > The bug title says "PermGenSize" and "MaxPermGenSize" but the comment > says "PermSize" and MaxPermSize". Which one is correct? > > Tested both options on java -version. Nothing happens, but I don't get > an "Unrecognized ..." error. > > Thanks, > Max From max.ockner at oracle.com Tue Oct 11 21:29:21 2016 From: max.ockner at oracle.com (Max Ockner) Date: Tue, 11 Oct 2016 17:29:21 -0400 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize In-Reply-To: References: <57FD24E1.1020905@oracle.com> Message-ID: <57FD59B1.6070703@oracle.com> New webrev: http://oklahoma.us.oracle.com/~mockner/webrev/8167446.02 - Added a test - Adjusted the obsolete field on both flags. On 10/11/2016 2:31 PM, harold seigel wrote: > Hi Max, > > Can you add a small test? It would prevent someone else from removing > the options. > I have added a simple test. > Thanks, Harold > > > On 10/11/2016 1:44 PM, Max Ockner wrote: >> Hello, >> Please Review this small change. >> >> http://cr.openjdk.java.net/~mockner/8167446/ >> https://bugs.openjdk.java.net/browse/JDK-8167446 >> >> Added PermSize and MaxPermSize to obsolete flags table. They are >> recognized but ignored. >> >> The bug title says "PermGenSize" and "MaxPermGenSize" but the comment >> says "PermSize" and MaxPermSize". Which one is correct? >> >> Tested both options on java -version. Nothing happens, but I don't >> get an "Unrecognized ..." error. >> >> Thanks, >> Max > From coleen.phillimore at oracle.com Tue Oct 11 22:10:14 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Oct 2016 18:10:14 -0400 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize In-Reply-To: <57FD59B1.6070703@oracle.com> References: <57FD24E1.1020905@oracle.com> <57FD59B1.6070703@oracle.com> Message-ID: <918df3e4-2b38-63ab-a1be-904e73b2f808@oracle.com> I think this looks great. If we decide to never remove these options at some point in the future, we can change the last element in the array to undefined. Thanks for doing this so quickly, Max. Coleen On 10/11/16 5:29 PM, Max Ockner wrote: > > New webrev: http://oklahoma.us.oracle.com/~mockner/webrev/8167446.02 > - Added a test > - Adjusted the obsolete field on both flags. > > On 10/11/2016 2:31 PM, harold seigel wrote: >> Hi Max, >> >> Can you add a small test? It would prevent someone else from >> removing the options. >> > I have added a simple test. > >> Thanks, Harold >> >> >> On 10/11/2016 1:44 PM, Max Ockner wrote: >>> Hello, >>> Please Review this small change. >>> >>> http://cr.openjdk.java.net/~mockner/8167446/ >>> https://bugs.openjdk.java.net/browse/JDK-8167446 >>> >>> Added PermSize and MaxPermSize to obsolete flags table. They are >>> recognized but ignored. >>> >>> The bug title says "PermGenSize" and "MaxPermGenSize" but the >>> comment says "PermSize" and MaxPermSize". Which one is correct? >>> >>> Tested both options on java -version. Nothing happens, but I don't >>> get an "Unrecognized ..." error. >>> >>> Thanks, >>> Max >> > From jeremymanson at google.com Wed Oct 12 00:03:55 2016 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 11 Oct 2016 17:03:55 -0700 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize In-Reply-To: <918df3e4-2b38-63ab-a1be-904e73b2f808@oracle.com> References: <57FD24E1.1020905@oracle.com> <57FD59B1.6070703@oracle.com> <918df3e4-2b38-63ab-a1be-904e73b2f808@oracle.com> Message-ID: FYI: At Google, to ease the transition, we just made those flags *mean* MetaspaceSize and MaxMetaspaceSize. We got no complaints (as far as I know). Jeremy On Tue, Oct 11, 2016 at 3:10 PM, Coleen Phillimore < coleen.phillimore at oracle.com> wrote: > > I think this looks great. If we decide to never remove these options at > some point in the future, we can change the last element in the array to > undefined. > > Thanks for doing this so quickly, Max. > Coleen > > > > On 10/11/16 5:29 PM, Max Ockner wrote: > >> >> New webrev: http://oklahoma.us.oracle.com/~mockner/webrev/8167446.02 >> - Added a test >> - Adjusted the obsolete field on both flags. >> >> On 10/11/2016 2:31 PM, harold seigel wrote: >> >>> Hi Max, >>> >>> Can you add a small test? It would prevent someone else from removing >>> the options. >>> >>> I have added a simple test. >> >> Thanks, Harold >>> >>> >>> On 10/11/2016 1:44 PM, Max Ockner wrote: >>> >>>> Hello, >>>> Please Review this small change. >>>> >>>> http://cr.openjdk.java.net/~mockner/8167446/ >>>> https://bugs.openjdk.java.net/browse/JDK-8167446 >>>> >>>> Added PermSize and MaxPermSize to obsolete flags table. They are >>>> recognized but ignored. >>>> >>>> The bug title says "PermGenSize" and "MaxPermGenSize" but the comment >>>> says "PermSize" and MaxPermSize". Which one is correct? >>>> >>>> Tested both options on java -version. Nothing happens, but I don't get >>>> an "Unrecognized ..." error. >>>> >>>> Thanks, >>>> Max >>>> >>> >>> >> > From marcus.larsson at oracle.com Wed Oct 12 08:41:52 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 12 Oct 2016 10:41:52 +0200 Subject: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: <69b49bf4-708a-9d9e-876c-841dd6bd6e49@oracle.com> Hi, On 10/11/2016 02:52 PM, Thomas St?fe wrote: > Hi Marcus, > > On Tue, Oct 11, 2016 at 12:12 PM, Marcus Larsson > > wrote: > > Hi, > > > On 10/11/2016 11:53 AM, Thomas St?fe wrote: >> Hi Marcus, >> >> On Tue, Oct 11, 2016 at 11:41 AM, Marcus Larsson >> > wrote: >> >> Hi Thomas, >> >> >> On 10/11/2016 11:13 AM, Thomas St?fe wrote: >>> Hi Markus, >>> >>> just a question whether I understand this correctly: Logging >>> works already from static initialization on, but as no >>> arguments are parsed yet, the default is to print out all >>> warning and error level messages to stderr? >>> >>> If true, this may be potentially confusing, because there >>> will always be a small window at startup where logging does >>> not behave as expected from the command line arguments >>> given. Like, if I redirect log output to a file, the first >>> burst of log output would still go to stderr. This may also >>> confuse tools which grep the output of a VM. >> >> Yes you're right, it's a limitation of UL at the moment. >> Logging before argument parsing means you can only use >> warning and error level, and can only expect to see it on stdout. >> >>> >>> Additionally, during initialization one may actually want >>> precise control of the logging system before arguments are >>> parsed, e.g. in os::init(). We have a lot of tracing in the >>> AIX os layer initialization which currently goes to nil >>> because it runs before Xlog argument parsing is done. >>> >>> Both situations could be solved if there were a possibility >>> to specify logging options right from the start, e.g. via a >>> file or an environment variable, which could be parsed in a >>> static initializer without relying on any VM initialization. >> >> It's either that or we implement some early logging buffering >> that saves all logging happening before UL initialization and >> logs it after UL is fully initialized. We haven't had the >> need for that early logging previously, but given that you >> seem to require it, it seems like a worthwhile enhancement of >> UL now. >> >> >> Thank you! This would be helpful. >> >> I think that buffering is not a good solution. For one, it >> requires you execute each and every log call and buffer the >> resulting string, because you do not know yet which of the log >> calls will be active. This may be cpu and memory intensive, which >> especially at startup may hurt. > > That's a fair point. Although I don't think we should have enough > such logging (before UL init) to make a significant difference. > > > You sometimes do not know, it could be a log call in a small > inconspicuous function which gets called a lot more than the original > author intended. Plus, you still have the cost of retrieving the > information you try to print, which you can completely avoid if you > know the log site is not active. > > At least I hope not. Some of it could perhaps be debug-only > logging (log_develop... etc)? > > > Our logging usually does not differ between develop and product build. > Simply because logging is valuable for our support people at customer > sites, and they usually have product builds. > >> >> Also, there is no guarantee the VM even lives long enough to >> parse the Xlog arguments to print out the buffered messages. The >> situations where we use early logging are often situations where >> the VM crashes early. > > Unprocessed log buffers could possibly be printed in the hs_err > output to avoid logging losses when the VM crashes early. Having > an additional early parsing step seems more complicated to me (at > least at first glance). > > > If argument parsing for -Xlog could be factored out, it could be > called for arguments from various sources - including the hypothetical > environment variable - from a very early point on. But I know, every > patch is easy from a high level :) Yeah, I guess we'll have to look at the alternatives and details as we go. I've filed https://bugs.openjdk.java.net/browse/JDK-8167571 for this. I appreciate the input! Thanks, Marcus > > ..Thomas > > > > Thanks, > Marcus > >> >> Kind Regards, Thomas >> >> Thanks, >> Marcus >> >>> >>> Kind Regards, Thomas >>> >>> >>> >>> On Mon, Oct 10, 2016 at 11:20 AM, Marcus Larsson >>> >> > wrote: >>> >>> Updated webrev after feedback: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 >>> >>> >>> Incremental: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 >>> >>> >>> >>> >>> >>> On 10/07/2016 04:26 PM, Marcus Larsson wrote: >>> >>> Hi, >>> >>> Making another attempt to fix this issue. >>> >>> Summary: >>> The following patch resolves a problem where the VM >>> would crash during shutdown due to static log >>> related memory being de-initialized before the last >>> use of the logging framework. The solution involves >>> parts of the Nifty Counter idiom [0] to control >>> static initialization and de-initialization of >>> stdout's and stderr's LogOutputs. Both objects are >>> now allocated using placement new, and avoids >>> destructor calls during de-initialization. The >>> LogStreamInitializer makes sure both objects are >>> initialized before first use. >>> >>> Because the LogOutput::Stdout/err pointers could no >>> longer be kept in LogOutput, I've replaced all >>> usages of them with the new references instead. >>> >>> The patch includes a regression test for this issue, >>> contributed by Michail Chernov. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >>> >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8146009 >>> >>> >>> Testing: >>> JPRT testset hotspot, included test on supported >>> platforms. >>> >>> Thanks, >>> Marcus >>> >>> [0] >>> https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter >>> >>> >>> >>> >> >> > > From marcus.larsson at oracle.com Wed Oct 12 08:42:52 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 12 Oct 2016 10:42:52 +0200 Subject: [PING] Re: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> Message-ID: <9ddbfabe-4602-6028-ed60-929225708b95@oracle.com> Any further comments on the updated patch? Marcus On 10/10/2016 11:20 AM, Marcus Larsson wrote: > Updated webrev after feedback: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 > > Incremental: > http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 > > > On 10/07/2016 04:26 PM, Marcus Larsson wrote: >> Hi, >> >> Making another attempt to fix this issue. >> >> Summary: >> The following patch resolves a problem where the VM would crash >> during shutdown due to static log related memory being de-initialized >> before the last use of the logging framework. The solution involves >> parts of the Nifty Counter idiom [0] to control static initialization >> and de-initialization of stdout's and stderr's LogOutputs. Both >> objects are now allocated using placement new, and avoids destructor >> calls during de-initialization. The LogStreamInitializer makes sure >> both objects are initialized before first use. >> >> Because the LogOutput::Stdout/err pointers could no longer be kept in >> LogOutput, I've replaced all usages of them with the new references >> instead. >> >> The patch includes a regression test for this issue, contributed by >> Michail Chernov. >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8146009 >> >> Testing: >> JPRT testset hotspot, included test on supported platforms. >> >> Thanks, >> Marcus >> >> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter > From dmitry.samersoff at oracle.com Wed Oct 12 09:00:55 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 12 Oct 2016 12:00:55 +0300 Subject: [PING] Re: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: <9ddbfabe-4602-6028-ed60-929225708b95@oracle.com> References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> <9ddbfabe-4602-6028-ed60-929225708b95@oracle.com> Message-ID: <8f5c5530-fb54-10ff-4658-ec7f422bba46@oracle.com> Marcus, This version looks good for me. -Dmitry On 2016-10-12 11:42, Marcus Larsson wrote: > Any further comments on the updated patch? > > Marcus > > > On 10/10/2016 11:20 AM, Marcus Larsson wrote: >> Updated webrev after feedback: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 >> >> Incremental: >> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 >> >> >> On 10/07/2016 04:26 PM, Marcus Larsson wrote: >>> Hi, >>> >>> Making another attempt to fix this issue. >>> >>> Summary: >>> The following patch resolves a problem where the VM would crash >>> during shutdown due to static log related memory being de-initialized >>> before the last use of the logging framework. The solution involves >>> parts of the Nifty Counter idiom [0] to control static initialization >>> and de-initialization of stdout's and stderr's LogOutputs. Both >>> objects are now allocated using placement new, and avoids destructor >>> calls during de-initialization. The LogStreamInitializer makes sure >>> both objects are initialized before first use. >>> >>> Because the LogOutput::Stdout/err pointers could no longer be kept in >>> LogOutput, I've replaced all usages of them with the new references >>> instead. >>> >>> The patch includes a regression test for this issue, contributed by >>> Michail Chernov. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8146009 >>> >>> Testing: >>> JPRT testset hotspot, included test on supported platforms. >>> >>> Thanks, >>> Marcus >>> >>> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From marcus.larsson at oracle.com Wed Oct 12 09:12:45 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 12 Oct 2016 11:12:45 +0200 Subject: [PING] Re: RFR: 8146009: "pure virtual method called" with using new GC logging mechanism In-Reply-To: <8f5c5530-fb54-10ff-4658-ec7f422bba46@oracle.com> References: <5910605e-e5bd-f8dc-4161-d66d550749c1@oracle.com> <9ddbfabe-4602-6028-ed60-929225708b95@oracle.com> <8f5c5530-fb54-10ff-4658-ec7f422bba46@oracle.com> Message-ID: <57c903c0-2b95-179b-3440-11d5e2397cb7@oracle.com> Great, thanks Dmitry! Marcus On 10/12/2016 11:00 AM, Dmitry Samersoff wrote: > Marcus, > > This version looks good for me. > > -Dmitry > > On 2016-10-12 11:42, Marcus Larsson wrote: >> Any further comments on the updated patch? >> >> Marcus >> >> >> On 10/10/2016 11:20 AM, Marcus Larsson wrote: >>> Updated webrev after feedback: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.01 >>> >>> Incremental: >>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00-01 >>> >>> >>> On 10/07/2016 04:26 PM, Marcus Larsson wrote: >>>> Hi, >>>> >>>> Making another attempt to fix this issue. >>>> >>>> Summary: >>>> The following patch resolves a problem where the VM would crash >>>> during shutdown due to static log related memory being de-initialized >>>> before the last use of the logging framework. The solution involves >>>> parts of the Nifty Counter idiom [0] to control static initialization >>>> and de-initialization of stdout's and stderr's LogOutputs. Both >>>> objects are now allocated using placement new, and avoids destructor >>>> calls during de-initialization. The LogStreamInitializer makes sure >>>> both objects are initialized before first use. >>>> >>>> Because the LogOutput::Stdout/err pointers could no longer be kept in >>>> LogOutput, I've replaced all usages of them with the new references >>>> instead. >>>> >>>> The patch includes a regression test for this issue, contributed by >>>> Michail Chernov. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8146009/webrev.00 >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8146009 >>>> >>>> Testing: >>>> JPRT testset hotspot, included test on supported platforms. >>>> >>>> Thanks, >>>> Marcus >>>> >>>> [0] https://en.wikibooks.org/wiki/More_C%2B%2B_Idioms/Nifty_Counter > From aph at redhat.com Wed Oct 12 10:23:58 2016 From: aph at redhat.com (Andrew Haley) Date: Wed, 12 Oct 2016 11:23:58 +0100 Subject: [aarch64-port-dev ] RFR: AArch64: SEGV in stub code cipherBlockChaining_decryptAESCrypt In-Reply-To: <92c29040-7a82-3b73-7079-6b23b781dcd2@redhat.com> References: <92c29040-7a82-3b73-7079-6b23b781dcd2@redhat.com> Message-ID: <309f4e28-8bcf-37a0-b649-4285a6f44fc4@redhat.com> On 21/09/16 10:35, Andrew Dinn wrote: > I think this patch looks like it will suffice to fix the AArch64 code. > However, I don't understand why this is only an AArch64 issue. For > example, it looks to me as if the x86_64 code is also susceptible to the > same problem should input value 0 be passed in len_reg (c_rarg4). So, > this might need fixing for all archs. Alternatively, it might be better > fixed in the jdk (Java) code. x86 is OK, but the reason why is a bit hard to find. It goes like this: __ align(OptoLoopAlignment); __ BIND(L_multiBlock_loopTop[k]); __ cmpptr(len_reg, PARALLEL_FACTOR * AESBlockSize); // see if at least 4 blocks left __ jcc(Assembler::less, L_singleBlock_loopTopHead[k]); ... __ BIND(L_singleBlock_loopTopHead[k]); if (k == 1) { __ addptr(rsp, 6 * wordSize); } else if (k == 2) { __ addptr(rsp, 10 * wordSize); } __ cmpptr(len_reg, 0); // any blocks left?? __ jcc(Assembler::equal, L_exit); So it firsts tests to see if 4 blocks are left, then tests to see if there are any blocks left. From lois.foltan at oracle.com Wed Oct 12 11:20:03 2016 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 12 Oct 2016 07:20:03 -0400 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize In-Reply-To: <918df3e4-2b38-63ab-a1be-904e73b2f808@oracle.com> References: <57FD24E1.1020905@oracle.com> <57FD59B1.6070703@oracle.com> <918df3e4-2b38-63ab-a1be-904e73b2f808@oracle.com> Message-ID: <57FE1C63.5000503@oracle.com> +1. Thanks for adding the test! Lois On 10/11/2016 6:10 PM, Coleen Phillimore wrote: > > I think this looks great. If we decide to never remove these options > at some point in the future, we can change the last element in the > array to undefined. > > Thanks for doing this so quickly, Max. > Coleen > > > On 10/11/16 5:29 PM, Max Ockner wrote: >> >> New webrev: http://oklahoma.us.oracle.com/~mockner/webrev/8167446.02 >> - Added a test >> - Adjusted the obsolete field on both flags. >> >> On 10/11/2016 2:31 PM, harold seigel wrote: >>> Hi Max, >>> >>> Can you add a small test? It would prevent someone else from >>> removing the options. >>> >> I have added a simple test. >> >>> Thanks, Harold >>> >>> >>> On 10/11/2016 1:44 PM, Max Ockner wrote: >>>> Hello, >>>> Please Review this small change. >>>> >>>> http://cr.openjdk.java.net/~mockner/8167446/ >>>> https://bugs.openjdk.java.net/browse/JDK-8167446 >>>> >>>> Added PermSize and MaxPermSize to obsolete flags table. They are >>>> recognized but ignored. >>>> >>>> The bug title says "PermGenSize" and "MaxPermGenSize" but the >>>> comment says "PermSize" and MaxPermSize". Which one is correct? >>>> >>>> Tested both options on java -version. Nothing happens, but I don't >>>> get an "Unrecognized ..." error. >>>> >>>> Thanks, >>>> Max >>> >> > From vladimir.kozlov at oracle.com Wed Oct 12 23:09:07 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Oct 2016 16:09:07 -0700 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> Message-ID: Hi Goetz and Volker, Positive news is that JEP status is moving, changes are reviewed and some changes were already pushed. But because of our testing issues during past week I was not able to execute the testing. We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge hs-comp repository first. hs/ repo will be opened for other pushes soon after that. I added estimated integration date to the JEP (Oct 28). We would like to test and integrate this port before JDK 10 forest is forked. Do you think all s390 changes and new code will be ready by that date? Do you have all shared changes reviewed and approved for push? Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare changeset based on hs/ repo for changes which are not pushed yet? I will try to submit testing over weekend. Regards, Vladimir On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote: > Hi Vladimir, > > This webrev contains all the changes to hotspot needed for the port: > http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/ > > It includes > http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/ > http://cr.openjdk.java.net/~goetz/wr16/8166561-basic_C1C2_s390/webrev.01/ > http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/ > which are out for review. Further it includes > the one change to relocate the pc-relative instructions where we didn't open > a bug for yet, and the new s390-files. > > Altogether this passed all our tests that were running on the weekend > on linuxs390. > > The s390-files though are not yet fully in shape, I'm still editing them to get > rid of legacy stuff and SAP JVM specific code. E.g. all the code guarded by > #ifdef SAPJVM will go away in the end. > > I hope to have the final versions by end of this week. > > Best regards, > Goetz. > > >> -----Original Message----- >> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] >> On Behalf Of Vladimir Kozlov >> Sent: Montag, 3. Oktober 2016 23:50 >> To: Volker Simonis >> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; >> build-dev ; HotSpot Open Source Developers >> ; Java Core Libs > dev at openjdk.java.net> >> Subject: Re: RFR: JEP draft for Linux/s3990x port >> >> Hi Volker, >> >> Can you prepare combined patch (or set of patches) based on latest >> reviews together with s390 code as it will be in final push? >> >> We want to run it through our pre-integration testing to verify that it >> does not have problems. >> >> Thanks, >> Vladimir >> >> On 9/29/16 11:25 AM, Vladimir Kozlov wrote: >>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or >>> other) state: >>> >>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html >>> >>> Vladimir >>> >>> On 9/29/16 9:55 AM, Volker Simonis wrote: >>>> Hi Vladimir, >>>> >>>> thanks a lot for reviewing and endorsing the JEP. >>>> >>>> I've linked all the relevant issues to the JEP (they all have a link >>>> to a webrev) and change the state to "Submitted". >>>> >>>> There's just one more small shared change we need for the port for >>>> which we haven't opened a bug now because we are still working on >>>> simplifying it. The current version looks as follows: >>>> >>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- >> constant_table_offset.patch >>>> >>>> >>>> What are the next steps? Should I add a "jdk9-fc-request" label to t >>>> he JEP and add a corresponding "FC Extension Request" comment to it? >>>> Or will this be done automatically once I move it to "Candidate"? >>>> >>>> Is there anything left to do before I can move it to "Candidate" state? >>>> >>>> Thanks a lot and best regards, >>>> Volker >>>> >>>> >>>> >>>> >>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov >>>> wrote: >>>>> On 9/27/16 10:49 AM, Volker Simonis wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> can you please review and endorse the following draft JEP for the >>>>>> integration of the Linux/s390x port into the jkd9 master repository: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730 >>>>> >>>>> >>>>> Good. >>>>> Add links to webrevs in a comment. It will help to get umbrella FC >>>>> extension >>>>> approval. >>>>> >>>>>> >>>>>> As detailed in the JEP, the Linux/s390x requires very few shared >>>>>> changes and we therefore don't foresee any impact on the existing >>>>>> platforms at all. Following you can find a short description of the >>>>>> planned changes: >>>>>> >>>>>> hotspot: >>>>>> ======= >>>>>> >>>>>> Out for review: >>>>>> 8166560: [s390] Basic enablement of s390 port. >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/hotspot.wr01/ >>>>>> >>>>>> Reviewed: >>>>>> 8166562: C2: Suppress relocations in scratch emit. >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >> scratch_emit/webrev.01/ >>>>>> >>>>>> Will send RFR soon (depends on 8166560): >>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >> scratch_emit/webrev.01 >>>>> >>>>> >>>>> Wrong link. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> >>>>>> >>>>>> We are still investigating the need of these shared changes: >>>>>> >>>>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >> 011-pass_PC_to_retAddrOffset.patch >>>>>> >>>>>> >>>>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >> 016-constant_table_offset.patch >>>>>> >>>>>> >>>>>> And finally the patch with the s390x-only platform files. We are still >>>>>> editing these to get them into OpenJdk style and shape. >>>>>> Hotspot passes most jck, jtreg and spec tests with these. >>>>>> >>>>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >> 101-zFiles.patch >>>>>> >>>>>> >>>>>> top-level repository: >>>>>> =============== >>>>>> >>>>>> The following is just adding some s390x specific compiler flags to >>>>>> flags.m4 >>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800 >>>>>> >>>>>> jdk repository: >>>>>> ============ >>>>>> >>>>>> This one just adds a new jvm.cfg file for s390x >>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801 >>>>>> >>>>>> >>>>>> And finally we plan to do one more change which fixes the jtreg test >>>>>> on Linux/s390x. But this is mainly for the correct detection of the >>>>>> platform and for excluding the tests which are not appropriate for >>>>>> s390x. >>>>>> >>>>>> Thank you and best regards, >>>>>> Volker >>>>>> >>>>> From sangheon.kim at oracle.com Thu Oct 13 04:07:57 2016 From: sangheon.kim at oracle.com (sangheon) Date: Wed, 12 Oct 2016 21:07:57 -0700 Subject: RFR(S): 8166461: Deprecate UseAutoGCSelectPolicy In-Reply-To: References: Message-ID: <063682a0-f670-8064-a674-cda09da4743b@oracle.com> Hi all, On 10/07/2016 03:30 PM, Kim Barrett wrote: >> On Oct 7, 2016, at 4:48 PM, sangheon wrote: >> >> Hi all, >> >> Can I have some reviews for this change to deprecate UseAutoGCSelectPolicy command-line option? >> >> The intent of UseAutoGCSelectPolicy option is to provide a way to automatically select GC based on target pause time. This option is introduced with AutoGCSelectPauseMillis and if MaxGCPauseMillis is smaller than or equal to AutoGCSelectPauseMillis(and if GC is not specified), low pause collector will be selected. I think it is enough to accomplish auto-GC-selection without UseAutoGCSelectPolicy. >> >> This patch only includes adding the deprecation message for UseAutoGCSelectPolicy option. >> >> CR:https://bugs.openjdk.java.net/browse/JDK-8166461 >> Webrev:http://cr.openjdk.java.net/~sangheki/8166461/webrev.0 >> Testing: JPRT >> >> Thanks, >> Sangheon > I thought the plan was to both deprecate the option, and to revert > its behavior to selecting between a throughput collector (Parallel) > and a low pause collector (probably CMS as before, though an argument > could be made for G1). My understanding of the internal discussion > was that the deprecation of the option was sufficient to address the > concern of ergonomically selecting CMS if it is deprecated. I had an internal discussion including Kim. And we decided to follow Kim's suggestion. This new webrev includes: 1) Reverted the code to select CMS or Parallel GC if UseAutoGCSelectPolicy(same behavior as previous version). If not, G1 will be selected as a default GC. 2) AutoGCSelectPauseMillis will be deprecated together but will be addressed by JDK-8167494. Webrev: - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1/ - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1_to_0/ Test: JPRT Thanks, Sangheon From sangheon.kim at oracle.com Thu Oct 13 04:10:36 2016 From: sangheon.kim at oracle.com (sangheon) Date: Wed, 12 Oct 2016 21:10:36 -0700 Subject: RFR(s): 8167494: Deprecate AutoGCSelectPauseMillis Message-ID: Hi all, Can I have some reviews for this patch of deprecating AutoGCSelectPauseMillis command-line option? As UseAutoGCSelectPolicy will be deprecated in JDK 9, AutoGCSelectPauseMillis which functions together with it, should be deprecated too. And AutoGCSelectPauseMillis should be addressed within "JDK-8166461: Deprecate UseAutoGCSelectPolicy", but it didn't. This is a sub-task of JDK-8166461 and the patch is based on the patch of JDK-8166461. This patch includes adding a deprecation message for AutoGCSelectPauseMillis option and a modification for existing test. CR: https://bugs.openjdk.java.net/browse/JDK-8167494 Webrev: http://cr.openjdk.java.net/~sangheki/8167494/webrev.0/ Test: JPRT Thanks, Sangheon From volker.simonis at gmail.com Thu Oct 13 06:31:31 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 13 Oct 2016 08:31:31 +0200 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> Message-ID: Hi Vladimir, thanks for keeping us updated and hanks for setting the integrations date on the JEP. We think it is reasonable to do it before the JDK 10 forest will be forked as this will obviously save us a lot of work. The remaining shared changes are "8166561: [s390] Adaptions needed for s390 port in C1 and C2." "8166560: [s390] Basic enablement of s390 port." "8167184: [s390] Extend relocations for pc-relative instructions." "8166837: [TESTBUG] Fix tests on Linux/s390x" Except the TESTBUG, they are all reviewed but I think the testbug is not so important and could be done later if we don't get it reviewd until the integration date. I'm currently traveling, but I'm sure Goetz can provide a new hs/based changeset for testing, right Goetz :) Regards, Volker On Thu, Oct 13, 2016 at 1:09 AM, Vladimir Kozlov wrote: > Hi Goetz and Volker, > > Positive news is that JEP status is moving, changes are reviewed and some > changes were already pushed. > > But because of our testing issues during past week I was not able to execute > the testing. > > We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge > hs-comp repository first. hs/ repo will be opened for other pushes soon > after that. > > I added estimated integration date to the JEP (Oct 28). We would like to > test and integrate this port before JDK 10 forest is forked. Do you think > all s390 changes and new code will be ready by that date? > > Do you have all shared changes reviewed and approved for push? > > Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare > changeset based on hs/ repo for changes which are not pushed yet? I will try > to submit testing over weekend. > > Regards, > Vladimir > > > On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote: >> >> Hi Vladimir, >> >> This webrev contains all the changes to hotspot needed for the port: >> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/ >> >> It includes >> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr03/ >> http://cr.openjdk.java.net/~goetz/wr16/8166561-basic_C1C2_s390/webrev.01/ >> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/ >> which are out for review. Further it includes >> the one change to relocate the pc-relative instructions where we didn't >> open >> a bug for yet, and the new s390-files. >> >> Altogether this passed all our tests that were running on the weekend >> on linuxs390. >> >> The s390-files though are not yet fully in shape, I'm still editing them >> to get >> rid of legacy stuff and SAP JVM specific code. E.g. all the code guarded >> by >> #ifdef SAPJVM will go away in the end. >> >> I hope to have the final versions by end of this week. >> >> Best regards, >> Goetz. >> >> >>> -----Original Message----- >>> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] >>> On Behalf Of Vladimir Kozlov >>> Sent: Montag, 3. Oktober 2016 23:50 >>> To: Volker Simonis >>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; >>> build-dev ; HotSpot Open Source Developers >>> ; Java Core Libs >> dev at openjdk.java.net> >>> Subject: Re: RFR: JEP draft for Linux/s3990x port >>> >>> Hi Volker, >>> >>> Can you prepare combined patch (or set of patches) based on latest >>> reviews together with s390 code as it will be in final push? >>> >>> We want to run it through our pre-integration testing to verify that it >>> does not have problems. >>> >>> Thanks, >>> Vladimir >>> >>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote: >>>> >>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or >>>> other) state: >>>> >>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html >>>> >>>> Vladimir >>>> >>>> On 9/29/16 9:55 AM, Volker Simonis wrote: >>>>> >>>>> Hi Vladimir, >>>>> >>>>> thanks a lot for reviewing and endorsing the JEP. >>>>> >>>>> I've linked all the relevant issues to the JEP (they all have a link >>>>> to a webrev) and change the state to "Submitted". >>>>> >>>>> There's just one more small shared change we need for the port for >>>>> which we haven't opened a bug now because we are still working on >>>>> simplifying it. The current version looks as follows: >>>>> >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- >>> >>> constant_table_offset.patch >>>>> >>>>> >>>>> >>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t >>>>> he JEP and add a corresponding "FC Extension Request" comment to it? >>>>> Or will this be done automatically once I move it to "Candidate"? >>>>> >>>>> Is there anything left to do before I can move it to "Candidate" state? >>>>> >>>>> Thanks a lot and best regards, >>>>> Volker >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov >>>>> wrote: >>>>>> >>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote: >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> can you please review and endorse the following draft JEP for the >>>>>>> integration of the Linux/s390x port into the jkd9 master repository: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730 >>>>>> >>>>>> >>>>>> >>>>>> Good. >>>>>> Add links to webrevs in a comment. It will help to get umbrella FC >>>>>> extension >>>>>> approval. >>>>>> >>>>>>> >>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared >>>>>>> changes and we therefore don't foresee any impact on the existing >>>>>>> platforms at all. Following you can find a short description of the >>>>>>> planned changes: >>>>>>> >>>>>>> hotspot: >>>>>>> ======= >>>>>>> >>>>>>> Out for review: >>>>>>> 8166560: [s390] Basic enablement of s390 port. >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>> >>> basic_s390/hotspot.wr01/ >>>>>>> >>>>>>> >>>>>>> Reviewed: >>>>>>> 8166562: C2: Suppress relocations in scratch emit. >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>> >>> scratch_emit/webrev.01/ >>>>>>> >>>>>>> >>>>>>> Will send RFR soon (depends on 8166560): >>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>> >>> scratch_emit/webrev.01 >>>>>> >>>>>> >>>>>> >>>>>> Wrong link. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> >>>>>>> >>>>>>> We are still investigating the need of these shared changes: >>>>>>> >>>>>>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>> 011-pass_PC_to_retAddrOffset.patch >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>> 016-constant_table_offset.patch >>>>>>> >>>>>>> >>>>>>> >>>>>>> And finally the patch with the s390x-only platform files. We are >>>>>>> still >>>>>>> editing these to get them into OpenJdk style and shape. >>>>>>> Hotspot passes most jck, jtreg and spec tests with these. >>>>>>> >>>>>>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>> 101-zFiles.patch >>>>>>> >>>>>>> >>>>>>> >>>>>>> top-level repository: >>>>>>> =============== >>>>>>> >>>>>>> The following is just adding some s390x specific compiler flags to >>>>>>> flags.m4 >>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800 >>>>>>> >>>>>>> jdk repository: >>>>>>> ============ >>>>>>> >>>>>>> This one just adds a new jvm.cfg file for s390x >>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801 >>>>>>> >>>>>>> >>>>>>> And finally we plan to do one more change which fixes the jtreg test >>>>>>> on Linux/s390x. But this is mainly for the correct detection of the >>>>>>> platform and for excluding the tests which are not appropriate for >>>>>>> s390x. >>>>>>> >>>>>>> Thank you and best regards, >>>>>>> Volker >>>>>>> >>>>>> > From thomas.stuefe at gmail.com Thu Oct 13 07:05:51 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 13 Oct 2016 09:05:51 +0200 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize In-Reply-To: References: <57FD24E1.1020905@oracle.com> <57FD59B1.6070703@oracle.com> <918df3e4-2b38-63ab-a1be-904e73b2f808@oracle.com> Message-ID: On Wed, Oct 12, 2016 at 2:03 AM, Jeremy Manson wrote: > FYI: At Google, to ease the transition, we just made those flags *mean* > MetaspaceSize and MaxMetaspaceSize. We got no complaints (as far as I > know). > Do you also derive CompressedClassSpaceSize ? > > Jeremy > > On Tue, Oct 11, 2016 at 3:10 PM, Coleen Phillimore < > coleen.phillimore at oracle.com> wrote: > > > > > I think this looks great. If we decide to never remove these options at > > some point in the future, we can change the last element in the array to > > undefined. > > > > Thanks for doing this so quickly, Max. > > Coleen > > > > > > > > On 10/11/16 5:29 PM, Max Ockner wrote: > > > >> > >> New webrev: http://oklahoma.us.oracle.com/~mockner/webrev/8167446.02 > >> - Added a test > >> - Adjusted the obsolete field on both flags. > >> > >> On 10/11/2016 2:31 PM, harold seigel wrote: > >> > >>> Hi Max, > >>> > >>> Can you add a small test? It would prevent someone else from removing > >>> the options. > >>> > >>> I have added a simple test. > >> > >> Thanks, Harold > >>> > >>> > >>> On 10/11/2016 1:44 PM, Max Ockner wrote: > >>> > >>>> Hello, > >>>> Please Review this small change. > >>>> > >>>> http://cr.openjdk.java.net/~mockner/8167446/ > >>>> https://bugs.openjdk.java.net/browse/JDK-8167446 > >>>> > >>>> Added PermSize and MaxPermSize to obsolete flags table. They are > >>>> recognized but ignored. > >>>> > >>>> The bug title says "PermGenSize" and "MaxPermGenSize" but the comment > >>>> says "PermSize" and MaxPermSize". Which one is correct? > >>>> > >>>> Tested both options on java -version. Nothing happens, but I don't get > >>>> an "Unrecognized ..." error. > >>>> > >>>> Thanks, > >>>> Max > >>>> > >>> > >>> > >> > > > From rkennke at redhat.com Thu Oct 13 09:40:25 2016 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 13 Oct 2016 11:40:25 +0200 Subject: RFR: 8167659: Access of mark word should use oopDesc::mark_offset_in_bytes() instead of '0' Message-ID: <1476351625.2776.7.camel@redhat.com> In several places in Hotspot's assembly routines, we're using Address(obj, 0) to access the object's mark word. This is bad style and makes it very hard to find all the places where the mark word is used. It should use Address(obj, oopDesc::mark_offset_in_bytes()) instead.? The change addresses the issue in x86 and aarch64. I'm not familiar with the othe arches, maybe somebody wants to check and fill them in? http://cr.openjdk.java.net/~rkennke/markoffset-hs/webrev.00/ Roman From erik.helin at oracle.com Thu Oct 13 10:11:49 2016 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 13 Oct 2016 12:11:49 +0200 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> Message-ID: <20161013101149.GL19291@ehelin.jrpg.bea.com> On 2016-10-10, Kim Barrett wrote: > > On Oct 10, 2016, at 9:54 AM, Erik Helin wrote: > > > > On 2016-09-29, Kim Barrett wrote: > >> Please review this change to G1 to use klass_or_null_acquire where > >> needed. > >> > >> [?] > >> > >> Changed oops_on_card_seq_iterate_careful to use different code paths > >> when the region is humongous or not. > >> > >> The humongous case is simplified because there can only be the one > >> (potential) object in the region. Use klass_or_null_acquire to > >> determine whether the object has been "published" or allocation is > >> still in-progress. The acquire barrier ensures processing the > >> humongous object is safe. When the klass is NULL the card marking > >> must be stale and can be ignored. > >> > >> In the non-humongous case, we know we're dealing with an old-gen > >> region. Because of that, we know we don't need to deal with > >> in-progress allocations here. That allows many simplifications. > >> > >> As a result of the changes in oops_on_card_seq_iterate_careful, we now > >> almost never fail to process the card. The only place where that can > >> occur is a stale card in a humongous region with an in-progress > >> allocation, where we can just ignore it. So the only caller, > >> refine_card, no longer needs to examine the result of the call and > >> enqueue the card for later reconsideration. > > > > I'm not sure this assumption is correct. Due to JDK-8166711 [0], can't > > we have the following scenario: > > > > [?] > > What do you think? This is very tricky code to analyze, but I think this > > patch won't work in the above scenario. > > > > When JDK-8166711 [0] is solved however, then I do think that this patch > > will work. The important part for getting this patch to work is to ensure > > that the two first checks, mr.is_empty() and is_young(), aren't > > reordered. > > I think the existing code can also fail in such a JDK-8166711 scenario. Yep! :) On 2016-10-10, Kim Barrett wrote: > I don?t think my proposed changes make that situation any worse, and will > enable changes I think can be made to fix 8166711 that will both simplify things > further and might not require any additional barriers. Note that 8166711 is the > next thing I plan to work on. I could add a loadload barrier as a temporary > stopgap; that didn?t seem necessary since the failures are pretty low probability > (not sure we?ve ever seen one) and I?m going to work on what I think will be > a better fix for that problem next. I agree that your change doesn't make it worse and I know that you will work on JDK-8166711. The problem is that I get headache enough from reasoning about these kind of issues, so I'm having a hard time reviewing this patch and trying to figure if it is correct as it is. If I also have to take into account that some known issues will be fixed, then it becomes *really* hard for me to verify this patch :( If you plan on working on JDK-8166711, is there any chance you can send out the patch for that issue first, and then this patch can be rebased on top of that work? I will of course review the patch for JDK-8166711 as well. Thanks, Erik From david.holmes at oracle.com Thu Oct 13 10:21:09 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Oct 2016 20:21:09 +1000 Subject: RFR: 8167659: Access of mark word should use oopDesc::mark_offset_in_bytes() instead of '0' In-Reply-To: <1476351625.2776.7.camel@redhat.com> References: <1476351625.2776.7.camel@redhat.com> Message-ID: <0e5ff47f-afa7-b9f2-3cee-e4f7767502c6@oracle.com> Hi Roman, On 13/10/2016 7:40 PM, Roman Kennke wrote: > In several places in Hotspot's assembly routines, we're using > Address(obj, 0) to access the object's mark word. This is bad style and > makes it very hard to find all the places where the mark word is used. > It should use Address(obj, oopDesc::mark_offset_in_bytes()) instead. The changes from 0 seem fine. The additional asserts make me think that something as basic as: assert(oopDesc::mark_offset_in_bytes() == 0, "assumption"); should only ever need to be checked once when the VM is initialized. > The change addresses the issue in x86 and aarch64. I'm not familiar > with the othe arches, maybe somebody wants to check and fill them in? I can't speak to that - I'm not in a position to do it. However this is an enhancement not a bug so realistically it is not a cleanup that should be being done in JDK 9 at this time. Unless you want to go through the FC extension process. Thanks, David > http://cr.openjdk.java.net/~rkennke/markoffset-hs/webrev.00/ > > Roman > From rkennke at redhat.com Thu Oct 13 10:25:26 2016 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 13 Oct 2016 12:25:26 +0200 Subject: RFR: 8167659: Access of mark word should use oopDesc::mark_offset_in_bytes() instead of '0' In-Reply-To: <0e5ff47f-afa7-b9f2-3cee-e4f7767502c6@oracle.com> References: <1476351625.2776.7.camel@redhat.com> <0e5ff47f-afa7-b9f2-3cee-e4f7767502c6@oracle.com> Message-ID: <1476354326.2776.13.camel@redhat.com> Am Donnerstag, den 13.10.2016, 20:21 +1000 schrieb David Holmes: > Hi Roman, > > On 13/10/2016 7:40 PM, Roman Kennke wrote: > > > > In several places in Hotspot's assembly routines, we're using > > Address(obj, 0) to access the object's mark word. This is bad style and > > makes it very hard to find all the places where the mark word is used. > > It should use Address(obj, oopDesc::mark_offset_in_bytes()) instead. > > The changes from 0 seem fine. > > The additional asserts make me think that something as basic as: > > assert(oopDesc::mark_offset_in_bytes() == 0, "assumption"); > > should only ever need to be checked once when the VM is initialized. I put them there because there's cmpxchg right below that uses obj directly as address. This way I reach my primary goal: identifying the places that access the mark word, while not complicating the code with additional (and pointless) lea instructions to build an obj+0 address. > However this is an enhancement not a bug so realistically it is not a? > cleanup that should be being done in JDK 9 at this time. Unless you want? > to go through the FC extension process. No, I don't. So we put that on hold until jdk10-dev is branched off? Roman From aph at redhat.com Thu Oct 13 10:44:19 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 13 Oct 2016 11:44:19 +0100 Subject: RFR: 8167659: Access of mark word should use oopDesc::mark_offset_in_bytes() instead of '0' In-Reply-To: <1476354326.2776.13.camel@redhat.com> References: <1476351625.2776.7.camel@redhat.com> <0e5ff47f-afa7-b9f2-3cee-e4f7767502c6@oracle.com> <1476354326.2776.13.camel@redhat.com> Message-ID: <255923e8-af82-e8af-d0e1-702f2e393e12@redhat.com> On 13/10/16 11:25, Roman Kennke wrote: > Am Donnerstag, den 13.10.2016, 20:21 +1000 schrieb David Holmes: >> Hi Roman, >> >> On 13/10/2016 7:40 PM, Roman Kennke wrote: >>> >>> In several places in Hotspot's assembly routines, we're using >>> Address(obj, 0) to access the object's mark word. This is bad style and >>> makes it very hard to find all the places where the mark word is used. >>> It should use Address(obj, oopDesc::mark_offset_in_bytes()) instead. >> >> The changes from 0 seem fine. >> >> The additional asserts make me think that something as basic as: >> >> assert(oopDesc::mark_offset_in_bytes() == 0, "assumption"); >> >> should only ever need to be checked once when the VM is initialized. > > I put them there because there's cmpxchg right below that uses obj > directly as address. This way I reach my primary goal: identifying > the places that access the mark word, while not complicating the > code with additional (and pointless) lea instructions to build an > obj+0 address. The problem with this approach is that it's not robust: everybody for evermore would have to remember to add such asserts. And these are checked every time in a slowdebug build. Andrew. From goetz.lindenmaier at sap.com Thu Oct 13 11:08:41 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 13 Oct 2016 11:08:41 +0000 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> Message-ID: <53905441dbdc41159896a1b143fdde70@DEWDFE13DE50.global.corp.sap> Hi Vladimir, I'm about to post the final change with the s390 files. All the others are reviewed. I'll then also send a complete webrev for testing. Best regards, Goetz > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: Thursday, October 13, 2016 8:32 AM > To: Vladimir Kozlov > Cc: Lindenmaier, Goetz ; s390x-port- > dev at openjdk.java.net; porters-dev at openjdk.java.net; build-dev dev at openjdk.java.net>; HotSpot Open Source Developers dev at openjdk.java.net>; Java Core Libs > Subject: Re: RFR: JEP draft for Linux/s3990x port > > Hi Vladimir, > > thanks for keeping us updated and hanks for setting the integrations > date on the JEP. We think it is reasonable to do it before the JDK 10 > forest will be forked as this will obviously save us a lot of work. > > The remaining shared changes are > > "8166561: [s390] Adaptions needed for s390 port in C1 and C2." > "8166560: [s390] Basic enablement of s390 port." > "8167184: [s390] Extend relocations for pc-relative instructions." > "8166837: [TESTBUG] Fix tests on Linux/s390x" > > > Except the TESTBUG, they are all reviewed but I think the testbug is > not so important and could be done later if we don't get it reviewd > until the integration date. > > I'm currently traveling, but I'm sure Goetz can provide a new hs/based > changeset for testing, right Goetz :) > > Regards, > Volker > > > On Thu, Oct 13, 2016 at 1:09 AM, Vladimir Kozlov > wrote: > > Hi Goetz and Volker, > > > > Positive news is that JEP status is moving, changes are reviewed and some > > changes were already pushed. > > > > But because of our testing issues during past week I was not able to > execute > > the testing. > > > > We hope jdk9/hs will be open soon but we want to sync jdk9/dev and > merge > > hs-comp repository first. hs/ repo will be opened for other pushes soon > > after that. > > > > I added estimated integration date to the JEP (Oct 28). We would like to > > test and integrate this port before JDK 10 forest is forked. Do you think > > all s390 changes and new code will be ready by that date? > > > > Do you have all shared changes reviewed and approved for push? > > > > Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare > > changeset based on hs/ repo for changes which are not pushed yet? I will > try > > to submit testing over weekend. > > > > Regards, > > Vladimir > > > > > > On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote: > >> > >> Hi Vladimir, > >> > >> This webrev contains all the changes to hotspot needed for the port: > >> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390- > all/hotspot.wr01/ > >> > >> It includes > >> http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr03/ > >> http://cr.openjdk.java.net/~goetz/wr16/8166561- > basic_C1C2_s390/webrev.01/ > >> http://cr.openjdk.java.net/~goetz/wr16/8166562- > scratch_emit/webrev.01/ > >> which are out for review. Further it includes > >> the one change to relocate the pc-relative instructions where we didn't > >> open > >> a bug for yet, and the new s390-files. > >> > >> Altogether this passed all our tests that were running on the weekend > >> on linuxs390. > >> > >> The s390-files though are not yet fully in shape, I'm still editing them > >> to get > >> rid of legacy stuff and SAP JVM specific code. E.g. all the code guarded > >> by > >> #ifdef SAPJVM will go away in the end. > >> > >> I hope to have the final versions by end of this week. > >> > >> Best regards, > >> Goetz. > >> > >> > >>> -----Original Message----- > >>> From: s390x-port-dev [mailto:s390x-port-dev- > bounces at openjdk.java.net] > >>> On Behalf Of Vladimir Kozlov > >>> Sent: Montag, 3. Oktober 2016 23:50 > >>> To: Volker Simonis > >>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; > >>> build-dev ; HotSpot Open Source > Developers > >>> ; Java Core Libs >>> dev at openjdk.java.net> > >>> Subject: Re: RFR: JEP draft for Linux/s3990x port > >>> > >>> Hi Volker, > >>> > >>> Can you prepare combined patch (or set of patches) based on latest > >>> reviews together with s390 code as it will be in final push? > >>> > >>> We want to run it through our pre-integration testing to verify that it > >>> does not have problems. > >>> > >>> Thanks, > >>> Vladimir > >>> > >>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote: > >>>> > >>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or > >>>> other) state: > >>>> > >>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html > >>>> > >>>> Vladimir > >>>> > >>>> On 9/29/16 9:55 AM, Volker Simonis wrote: > >>>>> > >>>>> Hi Vladimir, > >>>>> > >>>>> thanks a lot for reviewing and endorsing the JEP. > >>>>> > >>>>> I've linked all the relevant issues to the JEP (they all have a link > >>>>> to a webrev) and change the state to "Submitted". > >>>>> > >>>>> There's just one more small shared change we need for the port for > >>>>> which we haven't opened a bug now because we are still working on > >>>>> simplifying it. The current version looks as follows: > >>>>> > >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- > >>> > >>> constant_table_offset.patch > >>>>> > >>>>> > >>>>> > >>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t > >>>>> he JEP and add a corresponding "FC Extension Request" comment to it? > >>>>> Or will this be done automatically once I move it to "Candidate"? > >>>>> > >>>>> Is there anything left to do before I can move it to "Candidate" state? > >>>>> > >>>>> Thanks a lot and best regards, > >>>>> Volker > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov > >>>>> wrote: > >>>>>> > >>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote: > >>>>>>> > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> can you please review and endorse the following draft JEP for the > >>>>>>> integration of the Linux/s390x port into the jkd9 master repository: > >>>>>>> > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730 > >>>>>> > >>>>>> > >>>>>> > >>>>>> Good. > >>>>>> Add links to webrevs in a comment. It will help to get umbrella FC > >>>>>> extension > >>>>>> approval. > >>>>>> > >>>>>>> > >>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared > >>>>>>> changes and we therefore don't foresee any impact on the existing > >>>>>>> platforms at all. Following you can find a short description of the > >>>>>>> planned changes: > >>>>>>> > >>>>>>> hotspot: > >>>>>>> ======= > >>>>>>> > >>>>>>> Out for review: > >>>>>>> 8166560: [s390] Basic enablement of s390 port. > >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >>> > >>> basic_s390/hotspot.wr01/ > >>>>>>> > >>>>>>> > >>>>>>> Reviewed: > >>>>>>> 8166562: C2: Suppress relocations in scratch emit. > >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > >>> > >>> scratch_emit/webrev.01/ > >>>>>>> > >>>>>>> > >>>>>>> Will send RFR soon (depends on 8166560): > >>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. > >>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > >>> > >>> scratch_emit/webrev.01 > >>>>>> > >>>>>> > >>>>>> > >>>>>> Wrong link. > >>>>>> > >>>>>> Thanks, > >>>>>> Vladimir > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> We are still investigating the need of these shared changes: > >>>>>>> > >>>>>>> > >>> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >>> 011-pass_PC_to_retAddrOffset.patch > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >>> 016-constant_table_offset.patch > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> And finally the patch with the s390x-only platform files. We are > >>>>>>> still > >>>>>>> editing these to get them into OpenJdk style and shape. > >>>>>>> Hotspot passes most jck, jtreg and spec tests with these. > >>>>>>> > >>>>>>> > >>> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >>> 101-zFiles.patch > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> top-level repository: > >>>>>>> =============== > >>>>>>> > >>>>>>> The following is just adding some s390x specific compiler flags to > >>>>>>> flags.m4 > >>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800 > >>>>>>> > >>>>>>> jdk repository: > >>>>>>> ============ > >>>>>>> > >>>>>>> This one just adds a new jvm.cfg file for s390x > >>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801 > >>>>>>> > >>>>>>> > >>>>>>> And finally we plan to do one more change which fixes the jtreg test > >>>>>>> on Linux/s390x. But this is mainly for the correct detection of the > >>>>>>> platform and for excluding the tests which are not appropriate for > >>>>>>> s390x. > >>>>>>> > >>>>>>> Thank you and best regards, > >>>>>>> Volker > >>>>>>> > >>>>>> > > From erik.helin at oracle.com Thu Oct 13 12:15:10 2016 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 13 Oct 2016 14:15:10 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: References: Message-ID: <20161013121510.GM19291@ehelin.jrpg.bea.com> Hi Thomas, thanks for submitting the JEP and proposing this feature! On 2016-10-10, Thomas St?fe wrote: > Hi all, > > May I have please some feedback for this enhancement proposal? > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > In one very short sentence it proposes a better allocation scheme for > Metaspace Chunks in order to reduce fragmentation and metaspace waste. > > I also added a short presentation which describes the problem and how we > solved it in our VM port. > > https://bugs.openjdk.java.net/secure/attachment/63894/Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf Do we really need the flag -XX:+CoalesceMetaspace? Having two differnent ways to handle the chunk free lists in Metaspace is unfortunate, it might introduce hard to detect bugs and will also require much more testing (runnings lots of tests with the flag both on and off). Do you think your proposed solution has low enough overhead (in terms of CPU and memory) to be on "by default"? Thanks, Erik > Thank you very much! > > Kind Regards, Thomas > > > On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe > wrote: > > > Dear all, > > > > please take a look at this Enhancement Proposal for the metaspace > > allocator. I hope these are the right groups for this discussion. > > > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > Background: > > > > We at SAP see at times at customer installations OOMs in Metaspace > > (usually, with compressed class pointers enabled, in Compressed Class > > Space). The VM attempts to allocate metaspace and fails, hitting the > > CompressedClassSpaceSize limit. Note that we usually set the limit lower > > than the default, typically at 256M. > > > > When analyzing, we observed that a large part of the metaspace is indeed > > free but "locked in" into metaspace chunks of the wrong size: often we > > would find a lot of free small chunks, but the allocation request was for > > medium chunks, and failed. > > > > The reason was that if at some point in time a lot of class loaders were > > alive, each with only a few small classes loaded. This would lead to the > > metaspace being swamped with lots of small chunks. This is because each > > SpaceManager first allocates small chunks, only after a certain amount of > > allocation requests switches to larger chunks. > > > > These small chunks are free and wait in the freelist, but cannot be reused > > for allocation requests which require larger chunks, even if they are > > physically adjacent in the virtual space. > > > > We (at SAP) added a patch which allows on-the-fly metaspace chunk merging > > - to merge multiple adjacent smaller chunk to form a larger chunk. This, in > > combination with the reverse direction - splitting a large chunk to get > > smaller chunks - partly negates the "chunks-are-locked-in-into-their-size" > > limitation and provides for better reuse of metaspace chunks. It also > > provides better defragmentation as well. > > > > I discussed this fix off-list with Coleen Phillimore and Jon Masamitsu, > > and instead of just offering this as a fix, both recommended to open a JEP > > for this, because its scope would be beyond that of a simple fix. > > > > So here is my first JEP :) I hope it follows the right form. Please, if > > you have time, take a look and tell us what you think. > > > > Thank you, and Kind Regards, > > > > Thomas St?fe > > > > > > > > From coleen.phillimore at oracle.com Thu Oct 13 12:36:22 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 13 Oct 2016 08:36:22 -0400 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize In-Reply-To: References: <57FD24E1.1020905@oracle.com> <57FD59B1.6070703@oracle.com> <918df3e4-2b38-63ab-a1be-904e73b2f808@oracle.com> Message-ID: <101e4493-562b-808d-d685-31fdbbf04d96@oracle.com> On 10/11/16 8:03 PM, Jeremy Manson wrote: > FYI: At Google, to ease the transition, we just made those flags > *mean* MetaspaceSize and MaxMetaspaceSize. We got no complaints (as > far as I know). We didn't want to do that, although we suggested that initial settings could be the same as PermSize and MaxPermSize. I think PermGen was more compact than Metaspace is, so I don't think we want to set these as limits. Plus, I think you give MaxPermSize for a different reason than you give MaxMetaspaceSize. One is because the allocation is fixed so you need to specify as much as you need, and the other is not to overwhelm your machine because the allocation is not fixed at startup. I'm glad you haven't had complaints though. thanks, Coleen > > Jeremy > > On Tue, Oct 11, 2016 at 3:10 PM, Coleen Phillimore > > > wrote: > > > I think this looks great. If we decide to never remove these > options at some point in the future, we can change the last > element in the array to undefined. > > Thanks for doing this so quickly, Max. > Coleen > > > > On 10/11/16 5:29 PM, Max Ockner wrote: > > > New webrev: > http://oklahoma.us.oracle.com/~mockner/webrev/8167446.02 > > - Added a test > - Adjusted the obsolete field on both flags. > > On 10/11/2016 2:31 PM, harold seigel wrote: > > Hi Max, > > Can you add a small test? It would prevent someone else > from removing the options. > > I have added a simple test. > > Thanks, Harold > > > On 10/11/2016 1:44 PM, Max Ockner wrote: > > Hello, > Please Review this small change. > > http://cr.openjdk.java.net/~mockner/8167446/ > > https://bugs.openjdk.java.net/browse/JDK-8167446 > > > Added PermSize and MaxPermSize to obsolete flags > table. They are recognized but ignored. > > The bug title says "PermGenSize" and "MaxPermGenSize" > but the comment says "PermSize" and MaxPermSize". > Which one is correct? > > Tested both options on java -version. Nothing happens, > but I don't get an "Unrecognized ..." error. > > Thanks, > Max > > > > > From coleen.phillimore at oracle.com Thu Oct 13 12:41:40 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 13 Oct 2016 08:41:40 -0400 Subject: RFR: 8167659: Access of mark word should use oopDesc::mark_offset_in_bytes() instead of '0' In-Reply-To: <1476351625.2776.7.camel@redhat.com> References: <1476351625.2776.7.camel@redhat.com> Message-ID: Roman, I really like this change and we can do the other platforms and combine for 10. Can you file a bug report? + assert(oopDesc::mark_offset_in_bytes() == 0, "assumption"); cmpxchgptr(swap_reg, tmp_reg, obj_reg, rscratch1, here, slow_case); Maybe for aarch64 you can make this a macro assembler instruction? And either include the assert (which won't really add time during code generation), or add a comment with this assumption. Thanks, Coleen On 10/13/16 5:40 AM, Roman Kennke wrote: > In several places in Hotspot's assembly routines, we're using > Address(obj, 0) to access the object's mark word. This is bad style and > makes it very hard to find all the places where the mark word is used. > It should use Address(obj, oopDesc::mark_offset_in_bytes()) instead. > > The change addresses the issue in x86 and aarch64. I'm not familiar > with the othe arches, maybe somebody wants to check and fill them in? > > http://cr.openjdk.java.net/~rkennke/markoffset-hs/webrev.00/ > > Roman > From Alan.Burlison at oracle.com Thu Oct 13 12:47:13 2016 From: Alan.Burlison at oracle.com (Alan Burlison) Date: Thu, 13 Oct 2016 13:47:13 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> <34f41756-7ac7-684e-c896-9b8babf8f419@oracle.com> Message-ID: On 10/10/2016 11:05, Martin Walsh wrote: > Could I get a second reviewer for these changes please? LGTM. -- Alan Burlison -- From thomas.stuefe at gmail.com Thu Oct 13 13:21:04 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 13 Oct 2016 15:21:04 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: <20161013121510.GM19291@ehelin.jrpg.bea.com> References: <20161013121510.GM19291@ehelin.jrpg.bea.com> Message-ID: Hi Erik, On Thu, Oct 13, 2016 at 2:15 PM, Erik Helin wrote: > Hi Thomas, > > thanks for submitting the JEP and proposing this feature! > > On 2016-10-10, Thomas St?fe wrote: > > Hi all, > > > > May I have please some feedback for this enhancement proposal? > > > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > > > In one very short sentence it proposes a better allocation scheme for > > Metaspace Chunks in order to reduce fragmentation and metaspace waste. > > > > I also added a short presentation which describes the problem and how we > > solved it in our VM port. > > > > https://bugs.openjdk.java.net/secure/attachment/63894/ > Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf > > Do we really need the flag -XX:+CoalesceMetaspace? Having two differnent > ways to handle the chunk free lists in Metaspace is unfortunate, it > might introduce hard to detect bugs and will also require much more > testing (runnings lots of tests with the flag both on and off). > You are right. If the new allocator works well, there is no reason to keep the old allocator around. We wanted for a temporary time to be able to switch between both old and new allocator. Just to have a fallback if problems occur. But if it works, it makes sense to only have one allocator, and the "CoalesceMetaspace" flag can be removed, and also the code can be made a lot simpler because we do not need both code paths. > > Do you think your proposed solution has low enough overhead (in terms > of CPU and memory) to be on "by default"? > We decided to switch it on by default in our VM. Memory overhead can be almost exactly calculated. Bitmasks take 2 bits per specialized-chunk-sized-area. That means, for specialized-chunk-size = 1k (128 meta words): metaspace size / 8192. So, for 1G of metaspace we pay 132KB overhead for the bitmasks, or roughly 0.1%. There is some CPU overhead, but in my tests I could not measure anything above noise level. > Thanks, > Erik > > Btw, I understand that it is difficult to estimate this proposal without a prototype to play around. As I already mentioned, the patch right now only exists in our code base and not yet in the OpenJDK. If you guys are seriously interested in this JEP, I will invest the time to port the patch to the OpenJDK, so that you can check it out for yourself. Kind Regards, Thomas > > Thank you very much! > > > > Kind Regards, Thomas > > > > > > On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe > > wrote: > > > > > Dear all, > > > > > > please take a look at this Enhancement Proposal for the metaspace > > > allocator. I hope these are the right groups for this discussion. > > > > > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > > > Background: > > > > > > We at SAP see at times at customer installations OOMs in Metaspace > > > (usually, with compressed class pointers enabled, in Compressed Class > > > Space). The VM attempts to allocate metaspace and fails, hitting the > > > CompressedClassSpaceSize limit. Note that we usually set the limit > lower > > > than the default, typically at 256M. > > > > > > When analyzing, we observed that a large part of the metaspace is > indeed > > > free but "locked in" into metaspace chunks of the wrong size: often we > > > would find a lot of free small chunks, but the allocation request was > for > > > medium chunks, and failed. > > > > > > The reason was that if at some point in time a lot of class loaders > were > > > alive, each with only a few small classes loaded. This would lead to > the > > > metaspace being swamped with lots of small chunks. This is because each > > > SpaceManager first allocates small chunks, only after a certain amount > of > > > allocation requests switches to larger chunks. > > > > > > These small chunks are free and wait in the freelist, but cannot be > reused > > > for allocation requests which require larger chunks, even if they are > > > physically adjacent in the virtual space. > > > > > > We (at SAP) added a patch which allows on-the-fly metaspace chunk > merging > > > - to merge multiple adjacent smaller chunk to form a larger chunk. > This, in > > > combination with the reverse direction - splitting a large chunk to get > > > smaller chunks - partly negates the "chunks-are-locked-in-into- > their-size" > > > limitation and provides for better reuse of metaspace chunks. It also > > > provides better defragmentation as well. > > > > > > I discussed this fix off-list with Coleen Phillimore and Jon Masamitsu, > > > and instead of just offering this as a fix, both recommended to open a > JEP > > > for this, because its scope would be beyond that of a simple fix. > > > > > > So here is my first JEP :) I hope it follows the right form. Please, if > > > you have time, take a look and tell us what you think. > > > > > > Thank you, and Kind Regards, > > > > > > Thomas St?fe > > > > > > > > > > > > > From goetz.lindenmaier at sap.com Thu Oct 13 16:22:16 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 13 Oct 2016 16:22:16 +0000 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> Message-ID: <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap> Hi Vladimir, I made a new webrev containing all outstanding changes merged into one patch http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/ You probably saw my RFR with the s390 files. Best regards, Goetz. > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Thursday, October 13, 2016 1:09 AM > To: Lindenmaier, Goetz ; Volker Simonis > > Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; build- > dev ; HotSpot Open Source Developers > ; Java Core Libs dev at openjdk.java.net> > Subject: Re: RFR: JEP draft for Linux/s3990x port > > Hi Goetz and Volker, > > Positive news is that JEP status is moving, changes are reviewed and some > changes were already pushed. > > But because of our testing issues during past week I was not able to execute > the testing. > > We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge > hs-comp repository first. hs/ repo will be > opened for other pushes soon after that. > > I added estimated integration date to the JEP (Oct 28). We would like to test > and integrate this port before JDK 10 > forest is forked. Do you think all s390 changes and new code will be ready by > that date? > > Do you have all shared changes reviewed and approved for push? > > Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare > changeset based on hs/ repo for changes which > are not pushed yet? I will try to submit testing over weekend. > > Regards, > Vladimir > > On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote: > > Hi Vladimir, > > > > This webrev contains all the changes to hotspot needed for the port: > > http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390- > all/hotspot.wr01/ > > > > It includes > > http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr03/ > > http://cr.openjdk.java.net/~goetz/wr16/8166561- > basic_C1C2_s390/webrev.01/ > > http://cr.openjdk.java.net/~goetz/wr16/8166562- > scratch_emit/webrev.01/ > > which are out for review. Further it includes > > the one change to relocate the pc-relative instructions where we didn't > open > > a bug for yet, and the new s390-files. > > > > Altogether this passed all our tests that were running on the weekend > > on linuxs390. > > > > The s390-files though are not yet fully in shape, I'm still editing them to get > > rid of legacy stuff and SAP JVM specific code. E.g. all the code guarded by > > #ifdef SAPJVM will go away in the end. > > > > I hope to have the final versions by end of this week. > > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] > >> On Behalf Of Vladimir Kozlov > >> Sent: Montag, 3. Oktober 2016 23:50 > >> To: Volker Simonis > >> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; > >> build-dev ; HotSpot Open Source > Developers > >> ; Java Core Libs >> dev at openjdk.java.net> > >> Subject: Re: RFR: JEP draft for Linux/s3990x port > >> > >> Hi Volker, > >> > >> Can you prepare combined patch (or set of patches) based on latest > >> reviews together with s390 code as it will be in final push? > >> > >> We want to run it through our pre-integration testing to verify that it > >> does not have problems. > >> > >> Thanks, > >> Vladimir > >> > >> On 9/29/16 11:25 AM, Vladimir Kozlov wrote: > >>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or > >>> other) state: > >>> > >>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html > >>> > >>> Vladimir > >>> > >>> On 9/29/16 9:55 AM, Volker Simonis wrote: > >>>> Hi Vladimir, > >>>> > >>>> thanks a lot for reviewing and endorsing the JEP. > >>>> > >>>> I've linked all the relevant issues to the JEP (they all have a link > >>>> to a webrev) and change the state to "Submitted". > >>>> > >>>> There's just one more small shared change we need for the port for > >>>> which we haven't opened a bug now because we are still working on > >>>> simplifying it. The current version looks as follows: > >>>> > >>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- > >> constant_table_offset.patch > >>>> > >>>> > >>>> What are the next steps? Should I add a "jdk9-fc-request" label to t > >>>> he JEP and add a corresponding "FC Extension Request" comment to it? > >>>> Or will this be done automatically once I move it to "Candidate"? > >>>> > >>>> Is there anything left to do before I can move it to "Candidate" state? > >>>> > >>>> Thanks a lot and best regards, > >>>> Volker > >>>> > >>>> > >>>> > >>>> > >>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov > >>>> wrote: > >>>>> On 9/27/16 10:49 AM, Volker Simonis wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> can you please review and endorse the following draft JEP for the > >>>>>> integration of the Linux/s390x port into the jkd9 master repository: > >>>>>> > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730 > >>>>> > >>>>> > >>>>> Good. > >>>>> Add links to webrevs in a comment. It will help to get umbrella FC > >>>>> extension > >>>>> approval. > >>>>> > >>>>>> > >>>>>> As detailed in the JEP, the Linux/s390x requires very few shared > >>>>>> changes and we therefore don't foresee any impact on the existing > >>>>>> platforms at all. Following you can find a short description of the > >>>>>> planned changes: > >>>>>> > >>>>>> hotspot: > >>>>>> ======= > >>>>>> > >>>>>> Out for review: > >>>>>> 8166560: [s390] Basic enablement of s390 port. > >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >> basic_s390/hotspot.wr01/ > >>>>>> > >>>>>> Reviewed: > >>>>>> 8166562: C2: Suppress relocations in scratch emit. > >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > >> scratch_emit/webrev.01/ > >>>>>> > >>>>>> Will send RFR soon (depends on 8166560): > >>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. > >>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > >> scratch_emit/webrev.01 > >>>>> > >>>>> > >>>>> Wrong link. > >>>>> > >>>>> Thanks, > >>>>> Vladimir > >>>>> > >>>>> > >>>>>> > >>>>>> We are still investigating the need of these shared changes: > >>>>>> > >>>>>> > >> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >> 011-pass_PC_to_retAddrOffset.patch > >>>>>> > >>>>>> > >>>>>> > >> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >> 016-constant_table_offset.patch > >>>>>> > >>>>>> > >>>>>> And finally the patch with the s390x-only platform files. We are still > >>>>>> editing these to get them into OpenJdk style and shape. > >>>>>> Hotspot passes most jck, jtreg and spec tests with these. > >>>>>> > >>>>>> > >> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >> 101-zFiles.patch > >>>>>> > >>>>>> > >>>>>> top-level repository: > >>>>>> =============== > >>>>>> > >>>>>> The following is just adding some s390x specific compiler flags to > >>>>>> flags.m4 > >>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800 > >>>>>> > >>>>>> jdk repository: > >>>>>> ============ > >>>>>> > >>>>>> This one just adds a new jvm.cfg file for s390x > >>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801 > >>>>>> > >>>>>> > >>>>>> And finally we plan to do one more change which fixes the jtreg test > >>>>>> on Linux/s390x. But this is mainly for the correct detection of the > >>>>>> platform and for excluding the tests which are not appropriate for > >>>>>> s390x. > >>>>>> > >>>>>> Thank you and best regards, > >>>>>> Volker > >>>>>> > >>>>> From kim.barrett at oracle.com Thu Oct 13 18:20:27 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Oct 2016 14:20:27 -0400 Subject: RFR(S): 8166461: Deprecate UseAutoGCSelectPolicy In-Reply-To: <063682a0-f670-8064-a674-cda09da4743b@oracle.com> References: <063682a0-f670-8064-a674-cda09da4743b@oracle.com> Message-ID: > On Oct 13, 2016, at 12:07 AM, sangheon wrote: > > Webrev: > - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1/ > - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1_to_0/ > > Test: JPRT > > Thanks, > Sangheon Looks good. From kim.barrett at oracle.com Thu Oct 13 18:20:51 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Oct 2016 14:20:51 -0400 Subject: RFR(s): 8167494: Deprecate AutoGCSelectPauseMillis In-Reply-To: References: Message-ID: <32067A75-DB8B-499A-83B7-E7BDE731D878@oracle.com> > On Oct 13, 2016, at 12:10 AM, sangheon wrote: > > Hi all, > > Can I have some reviews for this patch of deprecating AutoGCSelectPauseMillis command-line option? > > As UseAutoGCSelectPolicy will be deprecated in JDK 9, AutoGCSelectPauseMillis which functions together with it, should be deprecated too. > And AutoGCSelectPauseMillis should be addressed within "JDK-8166461: Deprecate UseAutoGCSelectPolicy", but it didn't. This is a sub-task of JDK-8166461 and the patch is based on the patch of JDK-8166461. > > This patch includes adding a deprecation message for AutoGCSelectPauseMillis option and a modification for existing test. > > CR: https://bugs.openjdk.java.net/browse/JDK-8167494 > Webrev: http://cr.openjdk.java.net/~sangheki/8167494/webrev.0/ > Test: JPRT > > Thanks, > Sangheon Looks good. From sangheon.kim at oracle.com Thu Oct 13 18:22:05 2016 From: sangheon.kim at oracle.com (sangheon) Date: Thu, 13 Oct 2016 11:22:05 -0700 Subject: RFR(S): 8166461: Deprecate UseAutoGCSelectPolicy In-Reply-To: References: <063682a0-f670-8064-a674-cda09da4743b@oracle.com> Message-ID: Hi Kim, Thanks for the review! Sangheon On 10/13/2016 11:20 AM, Kim Barrett wrote: >> On Oct 13, 2016, at 12:07 AM, sangheon wrote: >> >> Webrev: >> - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1/ >> - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1_to_0/ >> >> Test: JPRT >> >> Thanks, >> Sangheon > Looks good. > From sangheon.kim at oracle.com Thu Oct 13 18:22:54 2016 From: sangheon.kim at oracle.com (sangheon) Date: Thu, 13 Oct 2016 11:22:54 -0700 Subject: RFR(s): 8167494: Deprecate AutoGCSelectPauseMillis In-Reply-To: <32067A75-DB8B-499A-83B7-E7BDE731D878@oracle.com> References: <32067A75-DB8B-499A-83B7-E7BDE731D878@oracle.com> Message-ID: <8a46a6f4-2497-b1d0-dbef-961ab478c783@oracle.com> Hi Kim, Thank you for this review too! Sangheon On 10/13/2016 11:20 AM, Kim Barrett wrote: >> On Oct 13, 2016, at 12:10 AM, sangheon wrote: >> >> Hi all, >> >> Can I have some reviews for this patch of deprecating AutoGCSelectPauseMillis command-line option? >> >> As UseAutoGCSelectPolicy will be deprecated in JDK 9, AutoGCSelectPauseMillis which functions together with it, should be deprecated too. >> And AutoGCSelectPauseMillis should be addressed within "JDK-8166461: Deprecate UseAutoGCSelectPolicy", but it didn't. This is a sub-task of JDK-8166461 and the patch is based on the patch of JDK-8166461. >> >> This patch includes adding a deprecation message for AutoGCSelectPauseMillis option and a modification for existing test. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8167494 >> Webrev: http://cr.openjdk.java.net/~sangheki/8167494/webrev.0/ >> Test: JPRT >> >> Thanks, >> Sangheon > Looks good. > From kim.barrett at oracle.com Thu Oct 13 20:39:25 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Oct 2016 16:39:25 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <20161013101149.GL19291@ehelin.jrpg.bea.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> <20161013101149.GL19291@ehelin.jrpg.bea.com> Message-ID: <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> > On Oct 13, 2016, at 6:11 AM, Erik Helin wrote: > I agree that your change doesn't make it worse and I know that you will > work on JDK-8166711. The problem is that I get headache enough from > reasoning about these kind of issues, so I'm having a hard time > reviewing this patch and trying to figure if it is correct as it is. If > I also have to take into account that some known issues will be fixed, > then it becomes *really* hard for me to verify this patch :( > > If you plan on working on JDK-8166711, is there any chance you can send > out the patch for that issue first, and then this patch can be > rebased on top of that work? I will of course review the patch for > JDK-8166711 as well. I'm pretty sure I don't want to analyze the changes I'm thinking about making for JDK-8166711 against the old (pre-fix for JDK-8166607) code. So re-ordering the changes doesn't really work for me. That's why I suggested I could, in the webrev under review, add an OrderAccess::loadload() somewhere between the top() access and the is_young() check to ensure the ordering there, as a temporary workaround for JDK-8166711. Or you could pretend it's there when reviewing (which is what I did when writing it). I think the best place for that loadload is right after the call to top(); it's not needed on the in-GC scan_top branch. But it can go anywere after there until before the call to is_young(). From kim.barrett at oracle.com Thu Oct 13 23:37:11 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Oct 2016 19:37:11 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> <20161013101149.GL19291@ehelin.jrpg.bea.com> <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> Message-ID: <328E1EF5-9DD6-41A9-900A-56E66DF245F3@oracle.com> > On Oct 13, 2016, at 4:39 PM, Kim Barrett wrote: >> On Oct 13, 2016, at 6:11 AM, Erik Helin wrote: >> I agree that your change doesn't make it worse and I know that you will >> work on JDK-8166711. The problem is that I get headache enough from >> reasoning about these kind of issues, so I'm having a hard time >> reviewing this patch and trying to figure if it is correct as it is. If >> I also have to take into account that some known issues will be fixed, >> then it becomes *really* hard for me to verify this patch :( >> >> If you plan on working on JDK-8166711, is there any chance you can send >> out the patch for that issue first, and then this patch can be >> rebased on top of that work? I will of course review the patch for >> JDK-8166711 as well. > > I'm pretty sure I don't want to analyze the changes I'm thinking about > making for JDK-8166711 against the old (pre-fix for JDK-8166607) code. > So re-ordering the changes doesn't really work for me. > > That's why I suggested I could, in the webrev under review, add an > OrderAccess::loadload() somewhere between the top() access and the > is_young() check to ensure the ordering there, as a temporary > workaround for JDK-8166711. Note that we?re using the wrong bug number for that other bug. It?s JDK-8166811. That loadload addition wouldn?t actually work; I forgot that the needed store-side ordering went missing at some point. > Or you could pretend it's there when > reviewing (which is what I did when writing it). So what I?d like is that you just pretend for now that the comment about the ordering in oops_on_card_xxx is actually correct. The change I?m working on for JDK-8166811 eliminates that ordering problem completely, but depends heavily on the changes here. From david.holmes at oracle.com Fri Oct 14 00:02:00 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 14 Oct 2016 10:02:00 +1000 Subject: RFR: 8167659: Access of mark word should use oopDesc::mark_offset_in_bytes() instead of '0' In-Reply-To: <1476354326.2776.13.camel@redhat.com> References: <1476351625.2776.7.camel@redhat.com> <0e5ff47f-afa7-b9f2-3cee-e4f7767502c6@oracle.com> <1476354326.2776.13.camel@redhat.com> Message-ID: <5b846065-2630-cb43-c1ff-137998c8ec69@oracle.com> On 13/10/2016 8:25 PM, Roman Kennke wrote: > Am Donnerstag, den 13.10.2016, 20:21 +1000 schrieb David Holmes: >> Hi Roman, >> >> On 13/10/2016 7:40 PM, Roman Kennke wrote: >>> >>> In several places in Hotspot's assembly routines, we're using >>> Address(obj, 0) to access the object's mark word. This is bad style and >>> makes it very hard to find all the places where the mark word is used. >>> It should use Address(obj, oopDesc::mark_offset_in_bytes()) instead. >> >> The changes from 0 seem fine. >> >> The additional asserts make me think that something as basic as: >> >> assert(oopDesc::mark_offset_in_bytes() == 0, "assumption"); >> >> should only ever need to be checked once when the VM is initialized. > > I put them there because there's cmpxchg right below that uses obj directly as address. This way I reach my primary goal: identifying the places that access the mark word, while not complicating the > code with additional (and pointless) lea instructions to build an obj+0 address. Hmmm okay. I still dislike the proliferation of that assert, and think it should be fixed where possible. I don't know if there is a better way to write the code using obj+offset() without actually generating the pointless instructions. If the mark offset must always be zero then perhaps it should some kind of compile-time constant. >> However this is an enhancement not a bug so realistically it is not a >> cleanup that should be being done in JDK 9 at this time. Unless you want >> to go through the FC extension process. > > No, I don't. So we put that on hold until jdk10-dev is branched off? Yes please. Thanks, David > Roman > From vladimir.kozlov at oracle.com Fri Oct 14 01:13:47 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Oct 2016 18:13:47 -0700 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap> References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap> Message-ID: <5800314B.5050206@oracle.com> Thank you, Goetz I submitted testing. Vladimir On 10/13/16 9:22 AM, Lindenmaier, Goetz wrote: > Hi Vladimir, > > I made a new webrev containing all outstanding changes merged into one patch > http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/ > > You probably saw my RFR with the s390 files. > > Best regards, > Goetz. > >> -----Original Message----- >> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >> Sent: Thursday, October 13, 2016 1:09 AM >> To: Lindenmaier, Goetz ; Volker Simonis >> >> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; build- >> dev ; HotSpot Open Source Developers >> ; Java Core Libs > dev at openjdk.java.net> >> Subject: Re: RFR: JEP draft for Linux/s3990x port >> >> Hi Goetz and Volker, >> >> Positive news is that JEP status is moving, changes are reviewed and some >> changes were already pushed. >> >> But because of our testing issues during past week I was not able to execute >> the testing. >> >> We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge >> hs-comp repository first. hs/ repo will be >> opened for other pushes soon after that. >> >> I added estimated integration date to the JEP (Oct 28). We would like to test >> and integrate this port before JDK 10 >> forest is forked. Do you think all s390 changes and new code will be ready by >> that date? >> >> Do you have all shared changes reviewed and approved for push? >> >> Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare >> changeset based on hs/ repo for changes which >> are not pushed yet? I will try to submit testing over weekend. >> >> Regards, >> Vladimir >> >> On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote: >>> Hi Vladimir, >>> >>> This webrev contains all the changes to hotspot needed for the port: >>> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390- >> all/hotspot.wr01/ >>> >>> It includes >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/hotspot.wr03/ >>> http://cr.openjdk.java.net/~goetz/wr16/8166561- >> basic_C1C2_s390/webrev.01/ >>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >> scratch_emit/webrev.01/ >>> which are out for review. Further it includes >>> the one change to relocate the pc-relative instructions where we didn't >> open >>> a bug for yet, and the new s390-files. >>> >>> Altogether this passed all our tests that were running on the weekend >>> on linuxs390. >>> >>> The s390-files though are not yet fully in shape, I'm still editing them to get >>> rid of legacy stuff and SAP JVM specific code. E.g. all the code guarded by >>> #ifdef SAPJVM will go away in the end. >>> >>> I hope to have the final versions by end of this week. >>> >>> Best regards, >>> Goetz. >>> >>> >>>> -----Original Message----- >>>> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] >>>> On Behalf Of Vladimir Kozlov >>>> Sent: Montag, 3. Oktober 2016 23:50 >>>> To: Volker Simonis >>>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; >>>> build-dev ; HotSpot Open Source >> Developers >>>> ; Java Core Libs >>> dev at openjdk.java.net> >>>> Subject: Re: RFR: JEP draft for Linux/s3990x port >>>> >>>> Hi Volker, >>>> >>>> Can you prepare combined patch (or set of patches) based on latest >>>> reviews together with s390 code as it will be in final push? >>>> >>>> We want to run it through our pre-integration testing to verify that it >>>> does not have problems. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote: >>>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or >>>>> other) state: >>>>> >>>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html >>>>> >>>>> Vladimir >>>>> >>>>> On 9/29/16 9:55 AM, Volker Simonis wrote: >>>>>> Hi Vladimir, >>>>>> >>>>>> thanks a lot for reviewing and endorsing the JEP. >>>>>> >>>>>> I've linked all the relevant issues to the JEP (they all have a link >>>>>> to a webrev) and change the state to "Submitted". >>>>>> >>>>>> There's just one more small shared change we need for the port for >>>>>> which we haven't opened a bug now because we are still working on >>>>>> simplifying it. The current version looks as follows: >>>>>> >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- >>>> constant_table_offset.patch >>>>>> >>>>>> >>>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t >>>>>> he JEP and add a corresponding "FC Extension Request" comment to it? >>>>>> Or will this be done automatically once I move it to "Candidate"? >>>>>> >>>>>> Is there anything left to do before I can move it to "Candidate" state? >>>>>> >>>>>> Thanks a lot and best regards, >>>>>> Volker >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov >>>>>> wrote: >>>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> can you please review and endorse the following draft JEP for the >>>>>>>> integration of the Linux/s390x port into the jkd9 master repository: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730 >>>>>>> >>>>>>> >>>>>>> Good. >>>>>>> Add links to webrevs in a comment. It will help to get umbrella FC >>>>>>> extension >>>>>>> approval. >>>>>>> >>>>>>>> >>>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared >>>>>>>> changes and we therefore don't foresee any impact on the existing >>>>>>>> platforms at all. Following you can find a short description of the >>>>>>>> planned changes: >>>>>>>> >>>>>>>> hotspot: >>>>>>>> ======= >>>>>>>> >>>>>>>> Out for review: >>>>>>>> 8166560: [s390] Basic enablement of s390 port. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>>> basic_s390/hotspot.wr01/ >>>>>>>> >>>>>>>> Reviewed: >>>>>>>> 8166562: C2: Suppress relocations in scratch emit. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>>> scratch_emit/webrev.01/ >>>>>>>> >>>>>>>> Will send RFR soon (depends on 8166560): >>>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>>> scratch_emit/webrev.01 >>>>>>> >>>>>>> >>>>>>> Wrong link. >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> We are still investigating the need of these shared changes: >>>>>>>> >>>>>>>> >>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>> 011-pass_PC_to_retAddrOffset.patch >>>>>>>> >>>>>>>> >>>>>>>> >>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>> 016-constant_table_offset.patch >>>>>>>> >>>>>>>> >>>>>>>> And finally the patch with the s390x-only platform files. We are still >>>>>>>> editing these to get them into OpenJdk style and shape. >>>>>>>> Hotspot passes most jck, jtreg and spec tests with these. >>>>>>>> >>>>>>>> >>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>> 101-zFiles.patch >>>>>>>> >>>>>>>> >>>>>>>> top-level repository: >>>>>>>> =============== >>>>>>>> >>>>>>>> The following is just adding some s390x specific compiler flags to >>>>>>>> flags.m4 >>>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800 >>>>>>>> >>>>>>>> jdk repository: >>>>>>>> ============ >>>>>>>> >>>>>>>> This one just adds a new jvm.cfg file for s390x >>>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801 >>>>>>>> >>>>>>>> >>>>>>>> And finally we plan to do one more change which fixes the jtreg test >>>>>>>> on Linux/s390x. But this is mainly for the correct detection of the >>>>>>>> platform and for excluding the tests which are not appropriate for >>>>>>>> s390x. >>>>>>>> >>>>>>>> Thank you and best regards, >>>>>>>> Volker >>>>>>>> >>>>>>> From jeremymanson at google.com Fri Oct 14 05:51:56 2016 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 13 Oct 2016 22:51:56 -0700 Subject: RFR: 8167446: Add back PermGenSize and MaxPermGenSize In-Reply-To: <101e4493-562b-808d-d685-31fdbbf04d96@oracle.com> References: <57FD24E1.1020905@oracle.com> <57FD59B1.6070703@oracle.com> <918df3e4-2b38-63ab-a1be-904e73b2f808@oracle.com> <101e4493-562b-808d-d685-31fdbbf04d96@oracle.com> Message-ID: For context, at Google, production services are carefully memory-limited. If we run over the memory limit, we get a memory limit exceeded error, which isn't very informative. Admins historically made a calculation that looked like: memory limit = max heap size + perm gen size + native memory overhead. Historically, we would carefully size the heap, the perm gen, and the service's memory limit. If you ran out of permgen space, you would get a nice clear OOME instead of a mysterious memory exceeded error. (Note that this has meant that we have many, many thousands of scripts that specify MaxPermSize. We haven't done anything about it because it's a relatively major undertaking! This thread inspired me to throw together some ideas, though.) When we transitioned to Java 8, we had two choices: - Let the PermSize flags do nothing. If there is a class leak, users who spent time doing this size calculation will get mysterious memory exceeded errors instead of a clear error. - Make the PermSize flags map to the MetaspaceSize flags. Extra room is available for the Metaspace, but if there is a class leak, then the users will get clear OOMEs. Plus, the system doesn't actually use the extra room unless it needs it, so users who have seen a drop in memory consumption can adjust their requirements downward appropriately. We went with option 2. (Also, we don't adjust the CompressedClassSpaceSize flag. I thought it was proportional to the MetaspaceSize, if that was specified?) Jeremy On Thu, Oct 13, 2016 at 5:36 AM, Coleen Phillimore < coleen.phillimore at oracle.com> wrote: > > > On 10/11/16 8:03 PM, Jeremy Manson wrote: > > FYI: At Google, to ease the transition, we just made those flags *mean* > MetaspaceSize and MaxMetaspaceSize. We got no complaints (as far as I > know). > > > We didn't want to do that, although we suggested that initial settings > could be the same as PermSize and MaxPermSize. I think PermGen was more > compact than Metaspace is, so I don't think we want to set these as > limits. Plus, I think you give MaxPermSize for a different reason than you > give MaxMetaspaceSize. One is because the allocation is fixed so you need > to specify as much as you need, and the other is not to overwhelm your > machine because the allocation is not fixed at startup. I'm glad you > haven't had complaints though. > > thanks, > Coleen > > > Jeremy > > On Tue, Oct 11, 2016 at 3:10 PM, Coleen Phillimore < > coleen.phillimore at oracle.com> wrote: > >> >> I think this looks great. If we decide to never remove these options at >> some point in the future, we can change the last element in the array to >> undefined. >> >> Thanks for doing this so quickly, Max. >> Coleen >> >> >> >> On 10/11/16 5:29 PM, Max Ockner wrote: >> >>> >>> New webrev: http://oklahoma.us.oracle.com/~mockner/webrev/8167446.02 >>> - Added a test >>> - Adjusted the obsolete field on both flags. >>> >>> On 10/11/2016 2:31 PM, harold seigel wrote: >>> >>>> Hi Max, >>>> >>>> Can you add a small test? It would prevent someone else from removing >>>> the options. >>>> >>>> I have added a simple test. >>> >>> Thanks, Harold >>>> >>>> >>>> On 10/11/2016 1:44 PM, Max Ockner wrote: >>>> >>>>> Hello, >>>>> Please Review this small change. >>>>> >>>>> http://cr.openjdk.java.net/~mockner/8167446/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8167446 >>>>> >>>>> Added PermSize and MaxPermSize to obsolete flags table. They are >>>>> recognized but ignored. >>>>> >>>>> The bug title says "PermGenSize" and "MaxPermGenSize" but the comment >>>>> says "PermSize" and MaxPermSize". Which one is correct? >>>>> >>>>> Tested both options on java -version. Nothing happens, but I don't get >>>>> an "Unrecognized ..." error. >>>>> >>>>> Thanks, >>>>> Max >>>>> >>>> >>>> >>> >> > > From aph at redhat.com Fri Oct 14 08:01:25 2016 From: aph at redhat.com (Andrew Haley) Date: Fri, 14 Oct 2016 09:01:25 +0100 Subject: RFR: 8167659: Access of mark word should use oopDesc::mark_offset_in_bytes() instead of '0' In-Reply-To: <5b846065-2630-cb43-c1ff-137998c8ec69@oracle.com> References: <1476351625.2776.7.camel@redhat.com> <0e5ff47f-afa7-b9f2-3cee-e4f7767502c6@oracle.com> <1476354326.2776.13.camel@redhat.com> <5b846065-2630-cb43-c1ff-137998c8ec69@oracle.com> Message-ID: On 14/10/16 01:02, David Holmes wrote: > Hmmm okay. I still dislike the proliferation of that assert, and > think it should be fixed where possible. I don't know if there is a > better way to write the code using obj+offset() without actually > generating the pointless instructions. If the mark offset must > always be zero then perhaps it should some kind of compile-time > constant. I don't think there is. On ARM, atomic instructions take only the register indirect addressing mode so there is literally nowhere to put an offset. It is possible to change the assembler to make these instructions take an offset and error out at runtime if that offset were ever anything other than zero. However, that's an additional complication for something I don't expect ever to change. Andrew. From jesper.wilhelmsson at oracle.com Fri Oct 14 16:09:50 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 14 Oct 2016 18:09:50 +0200 Subject: RFR: JDK-8157455 - Convert TestOS_test to GTest In-Reply-To: <851c8815-8a4d-df46-7426-3400c830f843@oracle.com> References: <851c8815-8a4d-df46-7426-3400c830f843@oracle.com> Message-ID: <0bf643bb-3c63-4fbc-900b-a69ae9705e0b@oracle.com> Hi, Kirill gave me some feedback offline and this is an updated version of the test. It's using EXPECT instead of ASSERT to allow all test cases to be executed even in the presence of failures and the test cases has been renamed. I also updated the copyright date since I learned that the copyright should follow the code, not the file, and we are moving older code into a new file. New webrev: http://cr.openjdk.java.net/~jwilhelm/8157455/webrev.01/ Thanks, /Jesper Den 25/8/16 kl. 20:46, skrev Jesper Wilhelmsson: > Hi, > > Please review this test conversion of the large page tests to GTest. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8157455 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8157455/webrev.00/ > > Thanks, > /Jesper From paul.sandoz at oracle.com Fri Oct 14 22:08:30 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 14 Oct 2016 15:08:30 -0700 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors Message-ID: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> Hi, Please review: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/ http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/ The JMV spec was recently updated to state the following in the Linkage Exceptions section of the invokedynamic instruction: https://bugs.openjdk.java.net/browse/JDK-8164693 "If resolution of the symbolic reference to the call site specifier throws an exception E, the invokedynamic instruction throws E if the type of E is Error or a subclass, else it throws a BootstrapMethodError that wraps E." "Otherwise, during the continuing resolution of the call site specifier, if invocation of the bootstrap method completes abruptly (?2.6.5) because of a throw of exception E, the invokedynamic instruction throws E if the type of E is Error or a subclass, else it throws a BootstrapMethodError that wraps E.? Namely if linkage fails due to the throwing of an Error or subclass then that Error is re-thrown by the indy. All other types of exception that may be thrown are wrapped in BootstrapMethodError that is then thrown. This means that errors such as ThreadDeath or OutOfMemoryError will pass through unwrapped. This is a behavioural change but we anticipate one that should have minimal impact. This change was motivated by updates to some classes using VarHandle for which VarHandle method linkage unduly wrapped some errors (like ThreadDeath) in LinkageError. (Note the same will apply for MethodHandle invoke linkage, or MethodHandle/MethodType resolution in the constant pool, or for string concatenation which now leverages indy). VarHandle/MethodHandle linkage error behaviour is tracked by another issue, https://bugs.openjdk.java.net/browse/JDK-8163553, which should not require any specification change. Thanks, Paul. From shafi.s.ahmad at oracle.com Mon Oct 17 10:12:31 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Mon, 17 Oct 2016 03:12:31 -0700 (PDT) Subject: [8u] RFR for JDK-8134918 - C2: Type speculation produces mismatched unsafe accesses Message-ID: Hi All, Please review the backport of JDK-8134918 - C2: Type speculation produces mismatched unsafe accesses to jdk8u-dev. Please note that backport is not clean and the conflict is due to: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/9108fab781a4#l5.165 Getting debug build failure because of: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/9108fab781a4#l5.155 The above changes are done under bug# 'JDK-8136473: failed: no mismatched stores, except on raw memory: StoreB StoreI' which is not back ported to jdk8u and the current backport is on top of above change. Please note that I am not sure if there is any dependency between these two changesets. open webrev: http://cr.openjdk.java.net/~shshahma/8134918/webrev.00/ jdk9 bug link: https://bugs.openjdk.java.net/browse/JDK-8134918 jdk9 changeset: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/79dae2cd00ef testing: Passes JPRT, jtreg not completed Regards, Shafi From erik.helin at oracle.com Mon Oct 17 14:22:28 2016 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Oct 2016 16:22:28 +0200 Subject: RFR(S): 8166461: Deprecate UseAutoGCSelectPolicy In-Reply-To: <063682a0-f670-8064-a674-cda09da4743b@oracle.com> References: <063682a0-f670-8064-a674-cda09da4743b@oracle.com> Message-ID: <20161017142228.GW19291@ehelin.jrpg.bea.com> On 2016-10-12, sangheon wrote: > Hi all, > > On 10/07/2016 03:30 PM, Kim Barrett wrote: > >>On Oct 7, 2016, at 4:48 PM, sangheon wrote: > >> > >>Hi all, > >> > >>Can I have some reviews for this change to deprecate UseAutoGCSelectPolicy command-line option? > >> > >>The intent of UseAutoGCSelectPolicy option is to provide a way to automatically select GC based on target pause time. This option is introduced with AutoGCSelectPauseMillis and if MaxGCPauseMillis is smaller than or equal to AutoGCSelectPauseMillis(and if GC is not specified), low pause collector will be selected. I think it is enough to accomplish auto-GC-selection without UseAutoGCSelectPolicy. > >> > >>This patch only includes adding the deprecation message for UseAutoGCSelectPolicy option. > >> > >>CR:https://bugs.openjdk.java.net/browse/JDK-8166461 > >>Webrev:http://cr.openjdk.java.net/~sangheki/8166461/webrev.0 > >>Testing: JPRT > >> > >>Thanks, > >>Sangheon > >I thought the plan was to both deprecate the option, and to revert > >its behavior to selecting between a throughput collector (Parallel) > >and a low pause collector (probably CMS as before, though an argument > >could be made for G1). My understanding of the internal discussion > >was that the deprecation of the option was sufficient to address the > >concern of ergonomically selecting CMS if it is deprecated. > I had an internal discussion including Kim. And we decided to follow Kim's > suggestion. > > This new webrev includes: > 1) Reverted the code to select CMS or Parallel GC if > UseAutoGCSelectPolicy(same behavior as previous version). If not, G1 will be > selected as a default GC. > 2) AutoGCSelectPauseMillis will be deprecated together but will be addressed > by JDK-8167494. > > Webrev: > - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1/ > - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1_to_0/ Looks good, Reviewed. Thanks, Erik > Test: JPRT > > Thanks, > Sangheon > > > From erik.helin at oracle.com Mon Oct 17 14:22:47 2016 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Oct 2016 16:22:47 +0200 Subject: RFR(s): 8167494: Deprecate AutoGCSelectPauseMillis In-Reply-To: References: Message-ID: <20161017142247.GX19291@ehelin.jrpg.bea.com> On 2016-10-12, sangheon wrote: > Hi all, > > Can I have some reviews for this patch of deprecating > AutoGCSelectPauseMillis command-line option? > > As UseAutoGCSelectPolicy will be deprecated in JDK 9, > AutoGCSelectPauseMillis which functions together with it, should be > deprecated too. > And AutoGCSelectPauseMillis should be addressed within "JDK-8166461: > Deprecate UseAutoGCSelectPolicy", but it didn't. This is a sub-task of > JDK-8166461 and the patch is based on the patch of JDK-8166461. > > This patch includes adding a deprecation message for AutoGCSelectPauseMillis > option and a modification for existing test. > > CR: https://bugs.openjdk.java.net/browse/JDK-8167494 > Webrev: http://cr.openjdk.java.net/~sangheki/8167494/webrev.0/ Looks good, Reviewed. Thanks, Erik > Test: JPRT > > Thanks, > Sangheon From rwestrel at redhat.com Mon Oct 17 15:58:40 2016 From: rwestrel at redhat.com (Roland Westrelin) Date: Mon, 17 Oct 2016 17:58:40 +0200 Subject: RFR(XS): 8168086: 8166869 broke jvmci build on aarch64 Message-ID: 8166869 was missing a similar aarch64 change which, I suppose, is something like this: http://cr.openjdk.java.net/~roland/8168086/webrev.00/ Roland. From vladimir.kozlov at oracle.com Mon Oct 17 16:16:21 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 17 Oct 2016 09:16:21 -0700 Subject: RFR(XS): 8168086: 8166869 broke jvmci build on aarch64 In-Reply-To: References: Message-ID: Looks good. Please, push it into jdk9/hs since hs-comp is closed and RO from now on. Thanks, Vladimir On 10/17/16 8:58 AM, Roland Westrelin wrote: > > 8166869 was missing a similar aarch64 change which, I suppose, is > something like this: > > http://cr.openjdk.java.net/~roland/8168086/webrev.00/ > > Roland. > From daniel.daugherty at oracle.com Mon Oct 17 16:30:18 2016 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 17 Oct 2016 10:30:18 -0600 Subject: RFR(XS): 8168086: 8166869 broke jvmci build on aarch64 In-Reply-To: References: Message-ID: <1d197712-ceaf-69ad-7bc9-d98acc0ea6c5@oracle.com> Jesper (as the JDK9-hs GK) needs to agree to this push since JDK9-hs is currently closed. Dan On 10/17/16 10:16 AM, Vladimir Kozlov wrote: > Looks good. > > Please, push it into jdk9/hs since hs-comp is closed and RO from now on. > > Thanks, > Vladimir > > On 10/17/16 8:58 AM, Roland Westrelin wrote: >> >> 8166869 was missing a similar aarch64 change which, I suppose, is >> something like this: >> >> http://cr.openjdk.java.net/~roland/8168086/webrev.00/ >> >> Roland. >> From vladimir.kozlov at oracle.com Mon Oct 17 16:34:32 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 17 Oct 2016 09:34:32 -0700 Subject: RFR(XS): 8168086: 8166869 broke jvmci build on aarch64 In-Reply-To: <1d197712-ceaf-69ad-7bc9-d98acc0ea6c5@oracle.com> References: <1d197712-ceaf-69ad-7bc9-d98acc0ea6c5@oracle.com> Message-ID: <0d39c485-c9bc-a350-ddf7-22df8063de19@oracle.com> Changes does not affect code which we test. But RH can't build their aarch64 port - they need to resolve it ASAP. Jesper said that we can push urgent changes. Vladimir On 10/17/16 9:30 AM, Daniel D. Daugherty wrote: > Jesper (as the JDK9-hs GK) needs to agree to this push since > JDK9-hs is currently closed. > > Dan > > > On 10/17/16 10:16 AM, Vladimir Kozlov wrote: >> Looks good. >> >> Please, push it into jdk9/hs since hs-comp is closed and RO from now on. >> >> Thanks, >> Vladimir >> >> On 10/17/16 8:58 AM, Roland Westrelin wrote: >>> >>> 8166869 was missing a similar aarch64 change which, I suppose, is >>> something like this: >>> >>> http://cr.openjdk.java.net/~roland/8168086/webrev.00/ >>> >>> Roland. >>> > From sangheon.kim at oracle.com Mon Oct 17 16:51:00 2016 From: sangheon.kim at oracle.com (sangheon) Date: Mon, 17 Oct 2016 09:51:00 -0700 Subject: RFR(S): 8166461: Deprecate UseAutoGCSelectPolicy In-Reply-To: <20161017142228.GW19291@ehelin.jrpg.bea.com> References: <063682a0-f670-8064-a674-cda09da4743b@oracle.com> <20161017142228.GW19291@ehelin.jrpg.bea.com> Message-ID: <0a2a2f92-3824-b9f0-5bb9-cf829f912457@oracle.com> Hi Erik, Thank you for the review! Sangheon On 10/17/2016 07:22 AM, Erik Helin wrote: > On 2016-10-12, sangheon wrote: >> Hi all, >> >> On 10/07/2016 03:30 PM, Kim Barrett wrote: >>>> On Oct 7, 2016, at 4:48 PM, sangheon wrote: >>>> >>>> Hi all, >>>> >>>> Can I have some reviews for this change to deprecate UseAutoGCSelectPolicy command-line option? >>>> >>>> The intent of UseAutoGCSelectPolicy option is to provide a way to automatically select GC based on target pause time. This option is introduced with AutoGCSelectPauseMillis and if MaxGCPauseMillis is smaller than or equal to AutoGCSelectPauseMillis(and if GC is not specified), low pause collector will be selected. I think it is enough to accomplish auto-GC-selection without UseAutoGCSelectPolicy. >>>> >>>> This patch only includes adding the deprecation message for UseAutoGCSelectPolicy option. >>>> >>>> CR:https://bugs.openjdk.java.net/browse/JDK-8166461 >>>> Webrev:http://cr.openjdk.java.net/~sangheki/8166461/webrev.0 >>>> Testing: JPRT >>>> >>>> Thanks, >>>> Sangheon >>> I thought the plan was to both deprecate the option, and to revert >>> its behavior to selecting between a throughput collector (Parallel) >>> and a low pause collector (probably CMS as before, though an argument >>> could be made for G1). My understanding of the internal discussion >>> was that the deprecation of the option was sufficient to address the >>> concern of ergonomically selecting CMS if it is deprecated. >> I had an internal discussion including Kim. And we decided to follow Kim's >> suggestion. >> >> This new webrev includes: >> 1) Reverted the code to select CMS or Parallel GC if >> UseAutoGCSelectPolicy(same behavior as previous version). If not, G1 will be >> selected as a default GC. >> 2) AutoGCSelectPauseMillis will be deprecated together but will be addressed >> by JDK-8167494. >> >> Webrev: >> - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1/ >> - http://cr.openjdk.java.net/~sangheki/8166461/webrev.1_to_0/ > Looks good, Reviewed. > > Thanks, > Erik > >> Test: JPRT >> >> Thanks, >> Sangheon >> >> >> From sangheon.kim at oracle.com Mon Oct 17 16:51:26 2016 From: sangheon.kim at oracle.com (sangheon) Date: Mon, 17 Oct 2016 09:51:26 -0700 Subject: RFR(s): 8167494: Deprecate AutoGCSelectPauseMillis In-Reply-To: <20161017142247.GX19291@ehelin.jrpg.bea.com> References: <20161017142247.GX19291@ehelin.jrpg.bea.com> Message-ID: Hi Erik, Thank you for reviewing this! Sangheon On 10/17/2016 07:22 AM, Erik Helin wrote: > On 2016-10-12, sangheon wrote: >> Hi all, >> >> Can I have some reviews for this patch of deprecating >> AutoGCSelectPauseMillis command-line option? >> >> As UseAutoGCSelectPolicy will be deprecated in JDK 9, >> AutoGCSelectPauseMillis which functions together with it, should be >> deprecated too. >> And AutoGCSelectPauseMillis should be addressed within "JDK-8166461: >> Deprecate UseAutoGCSelectPolicy", but it didn't. This is a sub-task of >> JDK-8166461 and the patch is based on the patch of JDK-8166461. >> >> This patch includes adding a deprecation message for AutoGCSelectPauseMillis >> option and a modification for existing test. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8167494 >> Webrev: http://cr.openjdk.java.net/~sangheki/8167494/webrev.0/ > Looks good, Reviewed. > > Thanks, > Erik > >> Test: JPRT >> >> Thanks, >> Sangheon From daniel.daugherty at oracle.com Mon Oct 17 17:06:51 2016 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 17 Oct 2016 11:06:51 -0600 Subject: RFR(XS): 8168086: 8166869 broke jvmci build on aarch64 In-Reply-To: <0d39c485-c9bc-a350-ddf7-22df8063de19@oracle.com> References: <1d197712-ceaf-69ad-7bc9-d98acc0ea6c5@oracle.com> <0d39c485-c9bc-a350-ddf7-22df8063de19@oracle.com> Message-ID: <3fbef4a0-b06a-5f9b-2572-c22762be5a39@oracle.com> So this is an AARCH64 specific change and I'm good with this change going in, but I'm not Jesper. I've re-read Jesper's internal email about allowing urgent changes to be pushed and it certainly sounds like he wants to approve the urgent changes on a case-by-case basis. I recommend waiting until you hear from Jesper, but that's just my opinion. Dan On 10/17/16 10:34 AM, Vladimir Kozlov wrote: > Changes does not affect code which we test. But RH can't build their > aarch64 port - they need to resolve it ASAP. Jesper said that we can > push urgent changes. > > Vladimir > > On 10/17/16 9:30 AM, Daniel D. Daugherty wrote: >> Jesper (as the JDK9-hs GK) needs to agree to this push since >> JDK9-hs is currently closed. >> >> Dan >> >> >> On 10/17/16 10:16 AM, Vladimir Kozlov wrote: >>> Looks good. >>> >>> Please, push it into jdk9/hs since hs-comp is closed and RO from now >>> on. >>> >>> Thanks, >>> Vladimir >>> >>> On 10/17/16 8:58 AM, Roland Westrelin wrote: >>>> >>>> 8166869 was missing a similar aarch64 change which, I suppose, is >>>> something like this: >>>> >>>> http://cr.openjdk.java.net/~roland/8168086/webrev.00/ >>>> >>>> Roland. >>>> >> From vladimir.kozlov at oracle.com Mon Oct 17 17:27:39 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 17 Oct 2016 10:27:39 -0700 Subject: RFR(XS): 8168086: 8166869 broke jvmci build on aarch64 In-Reply-To: <3fbef4a0-b06a-5f9b-2572-c22762be5a39@oracle.com> References: <1d197712-ceaf-69ad-7bc9-d98acc0ea6c5@oracle.com> <0d39c485-c9bc-a350-ddf7-22df8063de19@oracle.com> <3fbef4a0-b06a-5f9b-2572-c22762be5a39@oracle.com> Message-ID: <048baf92-b5aa-eee8-5bdd-8148737e0200@oracle.com> Okay, lets wait Jesper's approval. Vladimir On 10/17/16 10:06 AM, Daniel D. Daugherty wrote: > So this is an AARCH64 specific change and I'm good with this change > going in, but I'm not Jesper. > > I've re-read Jesper's internal email about allowing urgent changes > to be pushed and it certainly sounds like he wants to approve the > urgent changes on a case-by-case basis. > > I recommend waiting until you hear from Jesper, but that's just > my opinion. > > Dan > > > On 10/17/16 10:34 AM, Vladimir Kozlov wrote: >> Changes does not affect code which we test. But RH can't build their >> aarch64 port - they need to resolve it ASAP. Jesper said that we can >> push urgent changes. >> >> Vladimir >> >> On 10/17/16 9:30 AM, Daniel D. Daugherty wrote: >>> Jesper (as the JDK9-hs GK) needs to agree to this push since >>> JDK9-hs is currently closed. >>> >>> Dan >>> >>> >>> On 10/17/16 10:16 AM, Vladimir Kozlov wrote: >>>> Looks good. >>>> >>>> Please, push it into jdk9/hs since hs-comp is closed and RO from now >>>> on. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 10/17/16 8:58 AM, Roland Westrelin wrote: >>>>> >>>>> 8166869 was missing a similar aarch64 change which, I suppose, is >>>>> something like this: >>>>> >>>>> http://cr.openjdk.java.net/~roland/8168086/webrev.00/ >>>>> >>>>> Roland. >>>>> >>> > From rachel.protacio at oracle.com Mon Oct 17 17:57:02 2016 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Mon, 17 Oct 2016 13:57:02 -0400 Subject: RFR: JDK-8157455 - Convert TestOS_test to GTest In-Reply-To: <0bf643bb-3c63-4fbc-900b-a69ae9705e0b@oracle.com> References: <851c8815-8a4d-df46-7426-3400c830f843@oracle.com> <0bf643bb-3c63-4fbc-900b-a69ae9705e0b@oracle.com> Message-ID: You might add parentheses in test_quicksort.cpp lines 57/58 to make visual scanning slightly easier? Otherwise looks good to me. Rachel On 10/14/2016 12:09 PM, Jesper Wilhelmsson wrote: > Hi, > > Kirill gave me some feedback offline and this is an updated version of > the test. > > It's using EXPECT instead of ASSERT to allow all test cases to be > executed even in the presence of failures and the test cases has been > renamed. > > I also updated the copyright date since I learned that the copyright > should follow the code, not the file, and we are moving older code > into a new file. > > New webrev: http://cr.openjdk.java.net/~jwilhelm/8157455/webrev.01/ > > Thanks, > /Jesper > > > Den 25/8/16 kl. 20:46, skrev Jesper Wilhelmsson: >> Hi, >> >> Please review this test conversion of the large page tests to GTest. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8157455 >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8157455/webrev.00/ >> >> Thanks, >> /Jesper From stuart.marks at oracle.com Mon Oct 17 22:01:02 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 17 Oct 2016 15:01:02 -0700 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors In-Reply-To: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> Message-ID: <96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com> Hi Paul, I took a look at the jdk changes. They look good to me. One section of code gave me pause, which is the throw of ClassCastException at 339 of CallSite.java, and the throw of the exception returned from wrongTargetType() at 344 of CallSite.java. This appears odd given the "rethrow any Error and wrap anything else in BSME" rule. But these throws are caught by the catch of Throwable below, which does the wrap and throw of BSME. This arrangement was already present in the code. Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted? I didn't look too closely at the hotspot changes; somebody else should review them. s'marks On 10/14/16 3:08 PM, Paul Sandoz wrote: > Hi, > > Please review: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/ > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/ > > The JMV spec was recently updated to state the following in the Linkage Exceptions section of the invokedynamic instruction: > > https://bugs.openjdk.java.net/browse/JDK-8164693 > > "If resolution of the symbolic reference to the call site specifier throws an exception E, the invokedynamic instruction > throws E if the type of E is Error or a subclass, else it > throws a BootstrapMethodError that wraps E." > > "Otherwise, during the continuing resolution of the call site specifier, if invocation of the bootstrap method completes abruptly (?2.6.5) because of a throw of exception E, the invokedynamic instruction > throws E if the type of E is Error or a subclass, else it > throws a BootstrapMethodError that wraps E.? > > Namely if linkage fails due to the throwing of an Error or subclass then that Error is re-thrown by the indy. All other types of exception that may be thrown are wrapped in BootstrapMethodError that is then thrown. This means that errors such as ThreadDeath or OutOfMemoryError will pass through unwrapped. > > This is a behavioural change but we anticipate one that should have minimal impact. > > > This change was motivated by updates to some classes using VarHandle for which VarHandle method linkage unduly wrapped some errors (like ThreadDeath) in LinkageError. (Note the same will apply for MethodHandle invoke linkage, or MethodHandle/MethodType resolution in the constant pool, or for string concatenation which now leverages indy). VarHandle/MethodHandle linkage error behaviour is tracked by another issue, https://bugs.openjdk.java.net/browse/JDK-8163553, which should not require any specification change. > > Thanks, > Paul. > From david.holmes at oracle.com Mon Oct 17 22:33:40 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Oct 2016 08:33:40 +1000 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors In-Reply-To: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> Message-ID: Hi Paul, Looking at hotspot changes only ... On 15/10/2016 8:08 AM, Paul Sandoz wrote: > Hi, > > Please review: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/ > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/ src/share/vm/interpreter/linkResolver.cpp Changes seem fine, but the existing tracing code - which was already somewhat incorrect - now appears even more incorrect. It either needs to be moved to after the return for the Error case - so that we really are going to throw BSME - or else two different trace statements are needed. -- test/runtime/invokedynamic/BootstrapMethodErrorTest.java 2 * Copyright (c) 2015, 2016 Oracle and/or its affiliates. All rights reserved. Need comma after 2016. Otherwise all good. Thanks, David > The JMV spec was recently updated to state the following in the Linkage Exceptions section of the invokedynamic instruction: > > https://bugs.openjdk.java.net/browse/JDK-8164693 > > "If resolution of the symbolic reference to the call site specifier throws an exception E, the invokedynamic instruction > throws E if the type of E is Error or a subclass, else it > throws a BootstrapMethodError that wraps E." > > "Otherwise, during the continuing resolution of the call site specifier, if invocation of the bootstrap method completes abruptly (?2.6.5) because of a throw of exception E, the invokedynamic instruction > throws E if the type of E is Error or a subclass, else it > throws a BootstrapMethodError that wraps E.? > > Namely if linkage fails due to the throwing of an Error or subclass then that Error is re-thrown by the indy. All other types of exception that may be thrown are wrapped in BootstrapMethodError that is then thrown. This means that errors such as ThreadDeath or OutOfMemoryError will pass through unwrapped. > > This is a behavioural change but we anticipate one that should have minimal impact. > > > This change was motivated by updates to some classes using VarHandle for which VarHandle method linkage unduly wrapped some errors (like ThreadDeath) in LinkageError. (Note the same will apply for MethodHandle invoke linkage, or MethodHandle/MethodType resolution in the constant pool, or for string concatenation which now leverages indy). VarHandle/MethodHandle linkage error behaviour is tracked by another issue, https://bugs.openjdk.java.net/browse/JDK-8163553, which should not require any specification change. > > Thanks, > Paul. > From paul.sandoz at oracle.com Mon Oct 17 22:38:17 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 17 Oct 2016 15:38:17 -0700 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors In-Reply-To: <96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com> References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> <96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com> Message-ID: <36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com> > On 17 Oct 2016, at 15:01, Stuart Marks wrote: > > Hi Paul, > > I took a look at the jdk changes. They look good to me. > > One section of code gave me pause, which is the throw of ClassCastException at 339 of CallSite.java, and the throw of the exception returned from wrongTargetType() at 344 of CallSite.java. This appears odd given the "rethrow any Error and wrap anything else in BSME" rule. But these throws are caught by the catch of Throwable below, which does the wrap and throw of BSME. > > This arrangement was already present in the code. > > Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted? In addition to the // See the "Linking Exceptions" section for the invokedynamic // instruction in JVMS 6.5. I can add something like: // Throws a runtime exception defining the cause that is then later wrapped in BootstrapMethodError ? Thanks, Paul. From jesper.wilhelmsson at oracle.com Mon Oct 17 22:47:36 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 18 Oct 2016 00:47:36 +0200 Subject: jdk9/hs is OPEN for pushes again Message-ID: <9fc09f34-a0f6-1fc2-550d-e1ca46272a6b@oracle.com> Hi, We are finally seeing proper test results and jdk9/hs is now open for normal business again. We are sorry for the inconvenience that this outage has caused everyone in the community. Thanks for your patience! /Jesper From jesper.wilhelmsson at oracle.com Mon Oct 17 22:53:01 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 18 Oct 2016 00:53:01 +0200 Subject: RFR(XS): 8168086: 8166869 broke jvmci build on aarch64 In-Reply-To: <048baf92-b5aa-eee8-5bdd-8148737e0200@oracle.com> References: <1d197712-ceaf-69ad-7bc9-d98acc0ea6c5@oracle.com> <0d39c485-c9bc-a350-ddf7-22df8063de19@oracle.com> <3fbef4a0-b06a-5f9b-2572-c22762be5a39@oracle.com> <048baf92-b5aa-eee8-5bdd-8148737e0200@oracle.com> Message-ID: Hi, I would have approved this change anyway, but I just sent out the notification that hs is open for pushes again so now you don't have to wait for me :) Thanks, /Jesper Den 17/10/16 kl. 19:27, skrev Vladimir Kozlov: > Okay, lets wait Jesper's approval. > > Vladimir > > On 10/17/16 10:06 AM, Daniel D. Daugherty wrote: >> So this is an AARCH64 specific change and I'm good with this change >> going in, but I'm not Jesper. >> >> I've re-read Jesper's internal email about allowing urgent changes >> to be pushed and it certainly sounds like he wants to approve the >> urgent changes on a case-by-case basis. >> >> I recommend waiting until you hear from Jesper, but that's just >> my opinion. >> >> Dan >> >> >> On 10/17/16 10:34 AM, Vladimir Kozlov wrote: >>> Changes does not affect code which we test. But RH can't build their >>> aarch64 port - they need to resolve it ASAP. Jesper said that we can >>> push urgent changes. >>> >>> Vladimir >>> >>> On 10/17/16 9:30 AM, Daniel D. Daugherty wrote: >>>> Jesper (as the JDK9-hs GK) needs to agree to this push since >>>> JDK9-hs is currently closed. >>>> >>>> Dan >>>> >>>> >>>> On 10/17/16 10:16 AM, Vladimir Kozlov wrote: >>>>> Looks good. >>>>> >>>>> Please, push it into jdk9/hs since hs-comp is closed and RO from now >>>>> on. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 10/17/16 8:58 AM, Roland Westrelin wrote: >>>>>> >>>>>> 8166869 was missing a similar aarch64 change which, I suppose, is >>>>>> something like this: >>>>>> >>>>>> http://cr.openjdk.java.net/~roland/8168086/webrev.00/ >>>>>> >>>>>> Roland. >>>>>> >>>> >> From john.r.rose at oracle.com Mon Oct 17 23:36:16 2016 From: john.r.rose at oracle.com (John Rose) Date: Mon, 17 Oct 2016 16:36:16 -0700 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors In-Reply-To: <36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com> References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> <96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com> <36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com> Message-ID: <8BBECF63-51A3-4508-9E19-27641BE8F951@oracle.com> On Oct 17, 2016, at 3:38 PM, Paul Sandoz wrote: > > >> On 17 Oct 2016, at 15:01, Stuart Marks wrote: >> >> Hi Paul, >> >> I took a look at the jdk changes. They look good to me. >> >> One section of code gave me pause, which is the throw of ClassCastException at 339 of CallSite.java, and the throw of the exception returned from wrongTargetType() at 344 of CallSite.java. This appears odd given the "rethrow any Error and wrap anything else in BSME" rule. But these throws are caught by the catch of Throwable below, which does the wrap and throw of BSME. >> >> This arrangement was already present in the code. >> >> Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted? > > In addition to the > > // See the "Linking Exceptions" section for the invokedynamic > // instruction in JVMS 6.5. > > I can add something like: > > // Throws a runtime exception defining the cause that is then later wrapped in BootstrapMethodError I agree. Maybe s/later/in the "catch (Throwable ex)" a few lines below"/, to be more specific. I think the throw-to-catch is good here, unusually, because it funnels all BSME-wrapped exceptions through one point. That may make somebody's day easier when placing breakpoints. The upgraded BootstrapMethodErrorTest.java is really, really good. Overall, I am *delighted* to see this little mess cleaned up. It makes indy linkage approximately as transparent to errors as the other invocation instructions, making it a better replacement for them. Thanks a million! Reviewed. ? John From paul.sandoz at oracle.com Tue Oct 18 00:09:13 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 17 Oct 2016 17:09:13 -0700 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors In-Reply-To: References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> Message-ID: <7344C790-3C2B-454B-9CCB-DB6D85567089@oracle.com> > On 17 Oct 2016, at 15:33, David Holmes wrote: > > Hi Paul, > > Looking at hotspot changes only ... > > On 15/10/2016 8:08 AM, Paul Sandoz wrote: >> Hi, >> >> Please review: >> >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/ >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/ > > src/share/vm/interpreter/linkResolver.cpp > > Changes seem fine, but the existing tracing code - which was already somewhat incorrect - now appears even more incorrect. It either needs to be moved to after the return for the Error case - so that we really are going to throw BSME - or else two different trace statements are needed. > Good catch. Updated in place. http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/src/share/vm/interpreter/linkResolver.cpp.sdiff.html Given how CallSite behaves the VM wrapping in BSME is likely to be rare, i am unsure how it could actually occur. > -- > > test/runtime/invokedynamic/BootstrapMethodErrorTest.java > > 2 * Copyright (c) 2015, 2016 Oracle and/or its affiliates. All rights reserved. > > Need comma after 2016. > Updated. Thanks, Paul. > Otherwise all good. > > Thanks, > David > From paul.sandoz at oracle.com Tue Oct 18 00:16:49 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 17 Oct 2016 17:16:49 -0700 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors In-Reply-To: <8BBECF63-51A3-4508-9E19-27641BE8F951@oracle.com> References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> <96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com> <36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com> <8BBECF63-51A3-4508-9E19-27641BE8F951@oracle.com> Message-ID: <4D6566EF-9D65-4729-87F5-E3F03189CDCD@oracle.com> > On 17 Oct 2016, at 16:36, John Rose wrote: > > On Oct 17, 2016, at 3:38 PM, Paul Sandoz wrote: >> >> >>> On 17 Oct 2016, at 15:01, Stuart Marks wrote: >>> >>> Hi Paul, >>> >>> I took a look at the jdk changes. They look good to me. >>> >>> One section of code gave me pause, which is the throw of ClassCastException at 339 of CallSite.java, and the throw of the exception returned from wrongTargetType() at 344 of CallSite.java. This appears odd given the "rethrow any Error and wrap anything else in BSME" rule. But these throws are caught by the catch of Throwable below, which does the wrap and throw of BSME. >>> >>> This arrangement was already present in the code. >>> >>> Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted? >> >> In addition to the >> >> // See the "Linking Exceptions" section for the invokedynamic >> // instruction in JVMS 6.5. >> >> I can add something like: >> >> // Throws a runtime exception defining the cause that is then later wrapped in BootstrapMethodError > > I agree. Maybe s/later/in the "catch (Throwable ex)" a few lines below"/, to be more specific. > Thanks, updated in place. > I think the throw-to-catch is good here, unusually, because it funnels all BSME-wrapped > exceptions through one point. That may make somebody's day easier when placing breakpoints. > > The upgraded BootstrapMethodErrorTest.java is really, really good. > > Overall, I am *delighted* to see this little mess cleaned up. It makes indy linkage approximately as > transparent to errors as the other invocation instructions, making it a better replacement for them. > Another patch will follow soon for the MH/VH linking (and general j.l.invoke code) unduly wrapping Errors. Thanks, Paul. > Thanks a million! Reviewed. > > ? John From david.holmes at oracle.com Tue Oct 18 00:27:15 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Oct 2016 10:27:15 +1000 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors In-Reply-To: <7344C790-3C2B-454B-9CCB-DB6D85567089@oracle.com> References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> <7344C790-3C2B-454B-9CCB-DB6D85567089@oracle.com> Message-ID: On 18/10/2016 10:09 AM, Paul Sandoz wrote: > >> On 17 Oct 2016, at 15:33, David Holmes > > wrote: >> >> Hi Paul, >> >> Looking at hotspot changes only ... >> >> On 15/10/2016 8:08 AM, Paul Sandoz wrote: >>> Hi, >>> >>> Please review: >>> >>> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-jdk/webrev/ >>> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/ >> >> src/share/vm/interpreter/linkResolver.cpp >> >> Changes seem fine, but the existing tracing code - which was already >> somewhat incorrect - now appears even more incorrect. It either needs >> to be moved to after the return for the Error case - so that we really >> are going to throw BSME - or else two different trace statements are >> needed. >> > > Good catch. Updated in place. > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8166974-indy-errors-not-wrapped-hotspot/webrev/src/share/vm/interpreter/linkResolver.cpp.sdiff.html Looks good. Thanks, David > Given how CallSite behaves the VM wrapping in BSME is likely to be rare, > i am unsure how it could actually occur. > > >> -- >> >> test/runtime/invokedynamic/BootstrapMethodErrorTest.java >> >> 2 * Copyright (c) 2015, 2016 Oracle and/or its affiliates. All rights >> reserved. >> >> Need comma after 2016. >> > > Updated. > > Thanks, > Paul. > >> Otherwise all good. >> >> Thanks, >> David >> From stuart.marks at oracle.com Tue Oct 18 00:52:10 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 17 Oct 2016 17:52:10 -0700 Subject: RFR 8166974: invokedynamic implementation should not wrap Errors In-Reply-To: <4D6566EF-9D65-4729-87F5-E3F03189CDCD@oracle.com> References: <2BFB1A17-DAD8-4555-AA56-E58DFB962BEE@oracle.com> <96353eae-f467-5a57-5f6f-b69bb0abdd2a@oracle.com> <36C92AA0-0DAE-44A8-9709-ABC0CD6C8AFC@oracle.com> <8BBECF63-51A3-4508-9E19-27641BE8F951@oracle.com> <4D6566EF-9D65-4729-87F5-E3F03189CDCD@oracle.com> Message-ID: <6212e871-fa7f-419e-383f-e008d1e2b214@oracle.com> On 10/17/16 5:16 PM, Paul Sandoz wrote: >> On 17 Oct 2016, at 16:36, John Rose wrote: >> On Oct 17, 2016, at 3:38 PM, Paul Sandoz wrote: >>>> On 17 Oct 2016, at 15:01, Stuart Marks wrote: >>>> >>>> Usually I wrinkle my nose at a throw that's caught by a catch clause later on, but in this case it's not obvious what would be better. Maybe a comment is warranted? >>> >>> In addition to the >>> >>> // See the "Linking Exceptions" section for the invokedynamic >>> // instruction in JVMS 6.5. >>> >>> I can add something like: >>> >>> // Throws a runtime exception defining the cause that is then later wrapped in BootstrapMethodError >> >> I agree. Maybe s/later/in the "catch (Throwable ex)" a few lines below"/, to be more specific. > > Thanks, updated in place. The revised comment is a bit more verbose than my taste, but it's fine as it is. The main point is to alert the reader that something unusual is going on. [from John] >> I think the throw-to-catch is good here, unusually, because it funnels all BSME-wrapped >> exceptions through one point. That may make somebody's day easier when placing breakpoints. Agreed. This approach is indeed unusual, but I think the alternatives are worse: either the BSME wrapping can be replicated at the point of the throw, or it could be refactored into a common method. Either seems like a step in the wrong direction. Once you figure out this code, it isn't so bad; it's just unusual. s'marks From mikael.vidstedt at oracle.com Tue Oct 18 01:54:43 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Mon, 17 Oct 2016 18:54:43 -0700 Subject: Merging jdk9/hs-comp with jdk9/hs In-Reply-To: <0C83967C-3900-4A2A-88F3-A4D956AB6049@oracle.com> References: <0C83967C-3900-4A2A-88F3-A4D956AB6049@oracle.com> Message-ID: <80587B9B-B7C8-48E2-B0E7-359128B558E1@oracle.com> A final integration of jdk9/hs-comp to jdk9/hs was made last Friday, and the test results look good (even though it took a while longer than we were hoping to get them). This means that jdk9/hs-comp has now been made READ-ONLY, and that all Hotspot changes should now be integrated through jdk9/hs. Please help keep eyes and ears open over the next few days/weeks as we evaluate the effects of the consolidation. Unless there are significant problems caused by the consolidation the new model will be the one used going forward. Thanks to everybody involved in this, as well as the earlier forest consolidation efforts! Cheers, Mikael > On Oct 7, 2016, at 10:21 AM, Mikael Vidstedt wrote: > > > Unfortunately the test results didn?t materialize in time, so we will have to postpone the consolidation. Until proven otherwise we?ll try again Friday next week - same bat-time, same bat-channel. > > Cheers, > Mikael > >> On Oct 5, 2016, at 2:37 PM, Mikael Vidstedt wrote: >> >> >> Having heard no objections, we?re looking to move forward with the consolidation. >> >> We of course want to make sure that what?s currently in hs-comp is ready for (the last) integration, and due to some technical difficulties we?re still waiting for the relevant test results. We?re still hoping to have everything ready to go on Friday, but if not we may have to postpone the merge. >> >> Cheers, >> Mikael >> >>> On Sep 29, 2016, at 11:23 AM, Mikael Vidstedt wrote: >>> >>> >>> All, >>> >>> Over the last two years we have worked towards simplifying the mercurial forest structure for Hotspot development by consolidating and removing forests. Last year we moved[1] the GC development work from jdk9/hs-gc[2] to jdk9/hs-rt[3], and earlier this year we further consolidated forests by moving[4] the jdk9/hs-rt work directly to the jdk9/hs[5] - aka. "hs main" - forest. The reduction in forests has led to less work spent on integration, faster quality feedback, more efficient hardware utilization, etc. All in all, the experience so far has been a good one. >>> >>> However, the Hotspot (JIT) compiler changes are still integrated through the jdk9/hs-comp[6] forest, and are tested separately before they are (bulk) integrated up to jdk9/hs. The separate forest of course comes with the expected integration/gatekeeper overhead and lead times. Since JDK 9 development is ramping down, we expect the number of changes to ramp down as well. This means that the benefit of having a separate jdk9/hs-comp forest is going to reduce over time. Now seems like a good time to look at doing a final hotspot forest consolidation. >>> >>> In line with this, we propose closing down jdk9/hs-comp and doing all remaining JDK 9 Hotspot development to jdk9/hs. We propose that this final forest consolidation to take place on Friday, October 7th. Much like the earlier forest consolidations the jdk9/hs-comp forest will be kept around, but will be locked down so that no accidental integrations are made to it. Any work in flight based on jdk9/hs-comp would have to be rebased on jdk9/hs. >>> >>> The earlier forest consolidations have gone very smoothly, and we expect this one to go smoothly as well, but if for some reason things don't work out we will of course have an option to go back to what we have today. That said, before using the escape hatch and reverting back, we will of course want to try to address any issues - thank you in advance for your patience while we work through any such issues. >>> >>> Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Wednesday, 5 October 2016, we will go ahead with the forest consolidation. >>> >>> Cheers, >>> Mikael >>> >>> [1]http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/thread.html >>> [2]http://hg.openjdk.java.net/jdk9/hs-gc >>> [3]http://hg.openjdk.java.net/jdk9/hs-rt >>> [4]http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-March/022218.html >>> [5]http://hg.openjdk.java.net/jdk9/hs >>> [6]http://hg.openjdk.java.net/jdk9/hs-comp >>> >> > From shafi.s.ahmad at oracle.com Tue Oct 18 07:33:35 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 18 Oct 2016 00:33:35 -0700 (PDT) Subject: [8u] RFR for JDK-8134918 - C2: Type speculation produces mismatched unsafe accesses In-Reply-To: References: Message-ID: <210c8120-d49b-441e-ae86-da2b53e4121c@default> May I get the review done for this backport. The jtreg test on linux [amd64 ] is fine. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Monday, October 17, 2016 3:43 PM > To: hotspot-dev at openjdk.java.net > Subject: [8u] RFR for JDK-8134918 - C2: Type speculation produces > mismatched unsafe accesses > > Hi All, > > Please review the backport of JDK-8134918 - C2: Type speculation produces > mismatched unsafe accesses to jdk8u-dev. > > Please note that backport is not clean and the conflict is due to: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/9108fab781a4#l5.165 > > Getting debug build failure because of: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/9108fab781a4#l5.155 > > The above changes are done under bug# 'JDK-8136473: failed: no > mismatched stores, except on raw memory: StoreB StoreI' which is not back > ported to jdk8u and the current backport is on top of above change. > > Please note that I am not sure if there is any dependency between these > two changesets. > > open webrev: http://cr.openjdk.java.net/~shshahma/8134918/webrev.00/ > jdk9 bug link: https://bugs.openjdk.java.net/browse/JDK-8134918 > jdk9 changeset: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/79dae2cd00ef > > testing: Passes JPRT, jtreg not completed > > Regards, > Shafi From rwestrel at redhat.com Tue Oct 18 08:41:08 2016 From: rwestrel at redhat.com (Roland Westrelin) Date: Tue, 18 Oct 2016 10:41:08 +0200 Subject: RFR(XS): 8168086: 8166869 broke jvmci build on aarch64 In-Reply-To: References: Message-ID: > Looks good. Thanks for the review, Vladimir. Roland. From erik.helin at oracle.com Tue Oct 18 08:53:51 2016 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 18 Oct 2016 10:53:51 +0200 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <328E1EF5-9DD6-41A9-900A-56E66DF245F3@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> <20161013101149.GL19291@ehelin.jrpg.bea.com> <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> <328E1EF5-9DD6-41A9-900A-56E66DF245F3@oracle.com> Message-ID: <20161018085351.GA19291@ehelin.jrpg.bea.com> On 2016-10-13, Kim Barrett wrote: > > On Oct 13, 2016, at 4:39 PM, Kim Barrett wrote: > >> On Oct 13, 2016, at 6:11 AM, Erik Helin wrote: > >> I agree that your change doesn't make it worse and I know that you will > >> work on JDK-8166711. The problem is that I get headache enough from > >> reasoning about these kind of issues, so I'm having a hard time > >> reviewing this patch and trying to figure if it is correct as it is. If > >> I also have to take into account that some known issues will be fixed, > >> then it becomes *really* hard for me to verify this patch :( > >> > >> If you plan on working on JDK-8166711, is there any chance you can send > >> out the patch for that issue first, and then this patch can be > >> rebased on top of that work? I will of course review the patch for > >> JDK-8166711 as well. > > > > I'm pretty sure I don't want to analyze the changes I'm thinking about > > making for JDK-8166711 against the old (pre-fix for JDK-8166607) code. > > So re-ordering the changes doesn't really work for me. > > > > That's why I suggested I could, in the webrev under review, add an > > OrderAccess::loadload() somewhere between the top() access and the > > is_young() check to ensure the ordering there, as a temporary > > workaround for JDK-8166711. > > Note that we?re using the wrong bug number for that other bug. It?s > JDK-8166811. > > That loadload addition wouldn?t actually work; I forgot that the needed store-side > ordering went missing at some point. > > > Or you could pretend it's there when > > reviewing (which is what I did when writing it). > > So what I?d like is that you just pretend for now that the comment about the > ordering in oops_on_card_xxx is actually correct. > > The change I?m working on for JDK-8166811 eliminates that ordering problem > completely, but depends heavily on the changes here. I've discussed this with Thomas, what do you think about pushing the fix for JDK-8166811 at the same time as this one? This patch looks good AFAICS, but I'm worried that your code will be exposed slightly different to existing bug than the current code. Given that crashes/issues resulting from these kinds of bugs often are nightmarish to track down, I'm a bit hesitant to push these patches one by one. What do you think? Thanks, Erik From tobias.hartmann at oracle.com Tue Oct 18 13:25:02 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 18 Oct 2016 15:25:02 +0200 Subject: [9] RFR(S): 8164612: NoSuchMethodException when method name contains NULL or Latin-1 supplement character Message-ID: <580622AE.9080802@oracle.com> Hi, please review the following patch: https://bugs.openjdk.java.net/browse/JDK-8164612 http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/ The test executes Java Script code that defines getter methods containing Latin-1 supplement characters (0x80 - 0xFF). Those methods are registered at runtime through anonymous classes via Unsafe_DefineAnonymousClass. When calling a method, the VM fails with a NoSuchMethodException in MethodHandles::resolve_MemberName(). The failure happens while looking up the method name symbol in java_lang_String::as_symbol_or_null() [1]: 544 jbyte* position = (length == 0) ? NULL : value->byte_at_addr(0); 545 const char* base = UNICODE::as_utf8(position, length); 546 return SymbolTable::probe(base, length); If Compact Strings is enabled, we pass the Latin-1 encoded method name to UNICODE::as_utf8() and probe for the UTF-8 String in the SymbolTable. Since the Latin-1 method name contains non-ASCII characters, the length of the resulting UTF-8 String is larger (characters >= 0x80 are encoded as two bytes in UTF-8). However, we pass the shorter Latin-1 length to SymbolTable::probe() resulting in a lookup failure. I fixed this by passing the String length by reference to UNICODE::as_utf8(). I also refactored the related code in utf8.cpp, added comments and updated the callers. Tested with regression test and hs-comp PIT RBT (running). Thanks, Tobias, [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/652537a80080/src/share/vm/classfile/javaClasses.cpp#l535 From kim.barrett at oracle.com Tue Oct 18 14:30:49 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 18 Oct 2016 10:30:49 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <20161018085351.GA19291@ehelin.jrpg.bea.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> <20161013101149.GL19291@ehelin.jrpg.bea.com> <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> <328E1EF5-9DD6-41A9-900A-56E66DF245F3@oracle.com> <20161018085351.GA19291@ehelin.jrpg.bea.com> Message-ID: <7A66BDA6-87B3-417A-8BF5-61E8EFFB8F29@oracle.com> > On Oct 18, 2016, at 4:53 AM, Erik Helin wrote: > > On 2016-10-13, Kim Barrett wrote: >> So what I?d like is that you just pretend for now that the comment about the >> ordering in oops_on_card_xxx is actually correct. >> >> The change I?m working on for JDK-8166811 eliminates that ordering problem >> completely, but depends heavily on the changes here. > > I've discussed this with Thomas, what do you think about pushing the > fix for JDK-8166811 at the same time as this one? This patch looks good > AFAICS, but I'm worried that your code will be exposed slightly > different to existing bug than the current code. Given that > crashes/issues resulting from these kinds of bugs often are nightmarish > to track down, I'm a bit hesitant to push these patches one by one. > > What do you think? Sure, I can do that. From erik.helin at oracle.com Tue Oct 18 15:19:28 2016 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 18 Oct 2016 17:19:28 +0200 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <7A66BDA6-87B3-417A-8BF5-61E8EFFB8F29@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> <20161013101149.GL19291@ehelin.jrpg.bea.com> <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> <328E1EF5-9DD6-41A9-900A-56E66DF245F3@oracle.com> <20161018085351.GA19291@ehelin.jrpg.bea.com> <7A66BDA6-87B3-417A-8BF5-61E8EFFB8F29@oracle.com> Message-ID: <299b4c9a-6090-cf9a-ad01-8a010146e647@oracle.com> On 10/18/2016 04:30 PM, Kim Barrett wrote: >> On Oct 18, 2016, at 4:53 AM, Erik Helin wrote: >> >> On 2016-10-13, Kim Barrett wrote: >>> So what I?d like is that you just pretend for now that the comment about the >>> ordering in oops_on_card_xxx is actually correct. >>> >>> The change I?m working on for JDK-8166811 eliminates that ordering problem >>> completely, but depends heavily on the changes here. >> >> I've discussed this with Thomas, what do you think about pushing the >> fix for JDK-8166811 at the same time as this one? This patch looks good >> AFAICS, but I'm worried that your code will be exposed slightly >> different to existing bug than the current code. Given that >> crashes/issues resulting from these kinds of bugs often are nightmarish >> to track down, I'm a bit hesitant to push these patches one by one. >> >> What do you think? > > Sure, I can do that. Ok, thanks! Then consider this reviewed by me, but please hold of pushing it until I (and most likely Thomas) have reviewed the patch for JDK-8166811 as well. Thanks, Erik From paul.sandoz at oracle.com Tue Oct 18 18:41:30 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 18 Oct 2016 11:41:30 -0700 Subject: RFR 8163553 java.lang.LinkageError from test java/lang/ThreadGroup/Stop.java Message-ID: <5EA4A44D-3E66-4B76-8160-163580606FF1@oracle.com> Hi, Please review: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8163553-vh-mh-link-errors-not-wrapped/webrev/ This is the issue that motivated a change in the behaviour of indy wrapping Errors in BootstrapMethodError, JDK-8166974. I plan to push this issue with JDK-8166974 to hs, since they are related in behaviour even though there is no direct dependency between the patches. When invoking signature-polymorphic methods a similar but hardcoded dance occurs, with an appeal to Java code, to link the call site. - MethodHandle.invoke/invokeExact (and the VH methods) would wrap all Errors in LinkageError. Now they are passed through, thus an Error like ThreadDeath is not wrapped. - MethodHandle.invoke/invokeExact/invokeBasic throw Throwable, and in certain cases the Throwable is wrapped in an InternalError. In many other cases Error and RuntimeException are propagated, which i think in general is the right pattern, so i consistently applied that. - I updated StringConcatFactory to also pass through Errors and avoid unduly wrapping StringConcatException in another instance of StringConcatException. (LambdaMetafactory and associated classes required no changes.) Thanks, Paul. From vladimir.kozlov at oracle.com Wed Oct 19 00:19:48 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 18 Oct 2016 17:19:48 -0700 Subject: [9] RFR(S): 8164612: NoSuchMethodException when method name contains NULL or Latin-1 supplement character In-Reply-To: <580622AE.9080802@oracle.com> References: <580622AE.9080802@oracle.com> Message-ID: Looks good to me. Thanks, Vladimir On 10/18/16 6:25 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8164612 > http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/ > > The test executes Java Script code that defines getter methods containing Latin-1 supplement characters (0x80 - 0xFF). Those methods are registered at runtime through anonymous classes via Unsafe_DefineAnonymousClass. When calling a method, the VM fails with a NoSuchMethodException in MethodHandles::resolve_MemberName(). > > The failure happens while looking up the method name symbol in java_lang_String::as_symbol_or_null() [1]: > 544 jbyte* position = (length == 0) ? NULL : value->byte_at_addr(0); > 545 const char* base = UNICODE::as_utf8(position, length); > 546 return SymbolTable::probe(base, length); > > If Compact Strings is enabled, we pass the Latin-1 encoded method name to UNICODE::as_utf8() and probe for the UTF-8 String in the SymbolTable. Since the Latin-1 method name contains non-ASCII characters, the length of the resulting UTF-8 String is larger (characters >= 0x80 are encoded as two bytes in UTF-8). However, we pass the shorter Latin-1 length to SymbolTable::probe() resulting in a lookup failure. > > I fixed this by passing the String length by reference to UNICODE::as_utf8(). I also refactored the related code in utf8.cpp, added comments and updated the callers. > > Tested with regression test and hs-comp PIT RBT (running). > > Thanks, > Tobias, > > [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/652537a80080/src/share/vm/classfile/javaClasses.cpp#l535 > From david.holmes at oracle.com Wed Oct 19 01:16:13 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Oct 2016 11:16:13 +1000 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> <34f41756-7ac7-684e-c896-9b8babf8f419@oracle.com> Message-ID: <6e196387-0692-0d09-cfc7-d76bb230be66@oracle.com> Changes pushed. David On 13/10/2016 10:47 PM, Alan Burlison wrote: > On 10/10/2016 11:05, Martin Walsh wrote: > >> Could I get a second reviewer for these changes please? > > LGTM. > From vladimir.kozlov at oracle.com Wed Oct 19 02:56:35 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 18 Oct 2016 19:56:35 -0700 Subject: [8u] RFR for JDK-8134918 - C2: Type speculation produces mismatched unsafe accesses In-Reply-To: References: Message-ID: <05a495b7-7865-ffbf-d182-c15aeadbd4b0@oracle.com> Hi Shafi, You should also consider backporting following related fixes: https://bugs.openjdk.java.net/browse/JDK-8155781 https://bugs.openjdk.java.net/browse/JDK-8162101 Otherwise you may hit asserts added by 8134918 changes. Thanks, Vladimir On 10/17/16 3:12 AM, Shafi Ahmad wrote: > Hi All, > > Please review the backport of JDK-8134918 - C2: Type speculation produces mismatched unsafe accesses to jdk8u-dev. > > Please note that backport is not clean and the conflict is due to: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/9108fab781a4#l5.165 > > Getting debug build failure because of: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/9108fab781a4#l5.155 > > The above changes are done under bug# 'JDK-8136473: failed: no mismatched stores, except on raw memory: StoreB StoreI' which is not back ported to jdk8u and the current backport is on top of above change. > > Please note that I am not sure if there is any dependency between these two changesets. > > open webrev: http://cr.openjdk.java.net/~shshahma/8134918/webrev.00/ > jdk9 bug link: https://bugs.openjdk.java.net/browse/JDK-8134918 > jdk9 changeset: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/79dae2cd00ef > > testing: Passes JPRT, jtreg not completed > > Regards, > Shafi > From tobias.hartmann at oracle.com Wed Oct 19 06:15:53 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 19 Oct 2016 08:15:53 +0200 Subject: [9] RFR(S): 8164612: NoSuchMethodException when method name contains NULL or Latin-1 supplement character In-Reply-To: References: <580622AE.9080802@oracle.com> Message-ID: <58070F99.3000801@oracle.com> Thanks for the review, Vladimir! Best regards, Tobias On 19.10.2016 02:19, Vladimir Kozlov wrote: > Looks good to me. > > Thanks, > Vladimir > > On 10/18/16 6:25 AM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8164612 >> http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/ >> >> The test executes Java Script code that defines getter methods containing Latin-1 supplement characters (0x80 - 0xFF). Those methods are registered at runtime through anonymous classes via Unsafe_DefineAnonymousClass. When calling a method, the VM fails with a NoSuchMethodException in MethodHandles::resolve_MemberName(). >> >> The failure happens while looking up the method name symbol in java_lang_String::as_symbol_or_null() [1]: >> 544 jbyte* position = (length == 0) ? NULL : value->byte_at_addr(0); >> 545 const char* base = UNICODE::as_utf8(position, length); >> 546 return SymbolTable::probe(base, length); >> >> If Compact Strings is enabled, we pass the Latin-1 encoded method name to UNICODE::as_utf8() and probe for the UTF-8 String in the SymbolTable. Since the Latin-1 method name contains non-ASCII characters, the length of the resulting UTF-8 String is larger (characters >= 0x80 are encoded as two bytes in UTF-8). However, we pass the shorter Latin-1 length to SymbolTable::probe() resulting in a lookup failure. >> >> I fixed this by passing the String length by reference to UNICODE::as_utf8(). I also refactored the related code in utf8.cpp, added comments and updated the callers. >> >> Tested with regression test and hs-comp PIT RBT (running). >> >> Thanks, >> Tobias, >> >> [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/652537a80080/src/share/vm/classfile/javaClasses.cpp#l535 >> From shafi.s.ahmad at oracle.com Wed Oct 19 07:53:03 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 19 Oct 2016 00:53:03 -0700 (PDT) Subject: [8u] RFR for JDK-8134389: Crash in HotSpot with jvm.dll+0x42b48 ciObjectFactory::create_new_metadata Message-ID: Hi All, Please review the backport of 'JDK-8134389: Crash in HotSpot with jvm.dll+0x42b48 ciObjectFactory::create_new_metadata' to jdk8u-dev. Please note that backport is not clean as I was getting build failure due to: Formal parameter 'ignore_return' in method GraphBuilder::method_return is added in the fix of https://bugs.openjdk.java.net/browse/JDK-8164122. The current code change is done on top of aforesaid bug fix and this formal parameter is referenced in this code change. * if (x != NULL && !ignore_return) { * Author of this code change suggested me, we can safely remove this addition conditional expression ' && !ignore_return'. open webrev: http://cr.openjdk.java.net/~shshahma/8134389/webrev.00/ jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8134389 jdk9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/4191b33b3629 testing: Passes JPRT, jtreg on Linux [amd64] and newly added test case Regards, Shafi From martin.walsh at oracle.com Wed Oct 19 08:30:09 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Wed, 19 Oct 2016 09:30:09 +0100 Subject: RFR: JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above processors Message-ID: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> Could I get a review for the following bug please: JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above processors https://bugs.openjdk.java.net/browse/JDK-8164002 Webrev is available here: http://cr.openjdk.java.net/~mwalsh/JDK-8164004/ The changes add support for the SPARC-S processors, meaning the correct performance optimizations are made. Thanks, Martin From jesper.wilhelmsson at oracle.com Wed Oct 19 10:30:41 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 19 Oct 2016 12:30:41 +0200 Subject: RFR: JDK-8156800 - Convert QuickSort_test to GTest In-Reply-To: References: Message-ID: <1fdc5bcd-77bc-08c9-9640-cf3a9b4c3e99@oracle.com> Hi, I managed to mix up the bugIDs and sent an update to this webrev to the TestOS GTest conversion in this thread: http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/024911.html Inlining the mail below for clarity: ----------------------------------------------------------- Kirill gave me some feedback offline and this is an updated version of the test. It's using EXPECT instead of ASSERT to allow all test cases to be executed even in the presence of failures and the test cases has been renamed. I also updated the copyright date since I learned that the copyright should follow the code, not the file, and we are moving older code into a new file. New webrev: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.01/ ----------------------------------------------------------- (I have moved the webrev to the right place and updated the link i the mail above.) Rachel replied and suggested to add parentheses in lines 57 and 58 to ease visual scanning. I have added that in my local change. 57 bool a_is_odd = ((a % 2) == 1); 58 bool b_is_odd = ((b % 2) == 1); I now have three reviewer approvals for this change but no Reviewer. Could someone please have a look? Thanks, /Jesper Den 25/8/16 kl. 18:55, skrev Jesper Wilhelmsson: > Hi, > > Please review this test conversion of the quicksort tests to GTest. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8156800 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.00/ > > Thanks, > /Jesper From jesper.wilhelmsson at oracle.com Wed Oct 19 10:32:09 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 19 Oct 2016 12:32:09 +0200 Subject: RFR: JDK-8157455 - Convert TestOS_test to GTest In-Reply-To: References: <851c8815-8a4d-df46-7426-3400c830f843@oracle.com> <0bf643bb-3c63-4fbc-900b-a69ae9705e0b@oracle.com> Message-ID: <6c03b013-a51f-6e3c-352b-d25935ad9690@oracle.com> Thanks Rachel! I've added the parentheses. I managed to mix up the bugIDs and sent the last mail in reply to the wrong GTest conversion thread. The webrev.01 linked to below was for the QuickSort test conversion covered in this thread: http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-August/024356.html I have moved the webrev to the right place: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.01/ Please reply to any QuickSort issues in the October part of the QuickSort thread: http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/024949.html The TestOS_test originally covered in this thread is already reviewed and ready to push. Sorry for the mess! /Jesper Den 17/10/16 kl. 19:57, skrev Rachel Protacio: > You might add parentheses in test_quicksort.cpp lines 57/58 to make visual > scanning slightly easier? Otherwise looks good to me. > > Rachel > > > On 10/14/2016 12:09 PM, Jesper Wilhelmsson wrote: >> Hi, >> >> Kirill gave me some feedback offline and this is an updated version of the test. >> >> It's using EXPECT instead of ASSERT to allow all test cases to be executed >> even in the presence of failures and the test cases has been renamed. >> >> I also updated the copyright date since I learned that the copyright should >> follow the code, not the file, and we are moving older code into a new file. >> >> New webrev: http://cr.openjdk.java.net/~jwilhelm/8157455/webrev.01/ >> >> Thanks, >> /Jesper >> >> >> Den 25/8/16 kl. 20:46, skrev Jesper Wilhelmsson: >>> Hi, >>> >>> Please review this test conversion of the large page tests to GTest. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8157455 >>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8157455/webrev.00/ >>> >>> Thanks, >>> /Jesper > From eric.caspole at oracle.com Wed Oct 19 13:27:35 2016 From: eric.caspole at oracle.com (Eric Caspole) Date: Wed, 19 Oct 2016 09:27:35 -0400 Subject: RFR: JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above processors In-Reply-To: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> References: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> Message-ID: Looks good to me. Eric On 10/19/2016 4:30 AM, Martin Walsh wrote: > Could I get a review for the following bug please: > > JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above > processors > > https://bugs.openjdk.java.net/browse/JDK-8164002 > > Webrev is available here: > > http://cr.openjdk.java.net/~mwalsh/JDK-8164004/ > > > The changes add support for the SPARC-S processors, meaning the > correct performance optimizations are made. > > > Thanks, > > Martin From david.holmes at oracle.com Wed Oct 19 13:34:32 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Oct 2016 23:34:32 +1000 Subject: RFR: JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above processors In-Reply-To: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> References: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> Message-ID: <50e2cc8d-68ac-6511-c45a-8d4f598cb6b6@oracle.com> On 19/10/2016 6:30 PM, Martin Walsh wrote: > Could I get a review for the following bug please: > > JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above > processors > > https://bugs.openjdk.java.net/browse/JDK-8164002 > > Webrev is available here: > > http://cr.openjdk.java.net/~mwalsh/JDK-8164004/ > > > The changes add support for the SPARC-S processors, meaning the correct > performance optimizations are made. Looks okay. Thanks, David > > Thanks, > > Martin From martin.walsh at oracle.com Wed Oct 19 13:43:16 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Wed, 19 Oct 2016 14:43:16 +0100 Subject: RFR: JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above processors In-Reply-To: <50e2cc8d-68ac-6511-c45a-8d4f598cb6b6@oracle.com> References: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> <50e2cc8d-68ac-6511-c45a-8d4f598cb6b6@oracle.com> Message-ID: On 19/10/2016 14:34, David Holmes wrote: > On 19/10/2016 6:30 PM, Martin Walsh wrote: >> Could I get a review for the following bug please: >> >> JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above >> processors >> >> https://bugs.openjdk.java.net/browse/JDK-8164002 >> >> Webrev is available here: >> >> http://cr.openjdk.java.net/~mwalsh/JDK-8164004/ >> >> >> The changes add support for the SPARC-S processors, meaning the correct >> performance optimizations are made. > > Looks okay. Thanks for the reviews. Would somebody be willing to sponsor these changes? These changes also need to be back-ported. Who do I need to speak to to make that happen? Thanks, Martin > > Thanks, > David > >> >> Thanks, >> >> Martin From marcus.larsson at oracle.com Wed Oct 19 14:17:13 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 19 Oct 2016 16:17:13 +0200 Subject: RFR: JDK-8156800 - Convert QuickSort_test to GTest In-Reply-To: <1fdc5bcd-77bc-08c9-9640-cf3a9b4c3e99@oracle.com> References: <1fdc5bcd-77bc-08c9-9640-cf3a9b4c3e99@oracle.com> Message-ID: <65a1a78d-778d-2c0d-a9a6-a5d88cb5c885@oracle.com> Hi Jesper, On 10/19/2016 12:30 PM, Jesper Wilhelmsson wrote: > Hi, > > I managed to mix up the bugIDs and sent an update to this webrev to > the TestOS GTest conversion in this thread: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/024911.html > > > Inlining the mail below for clarity: > > ----------------------------------------------------------- > Kirill gave me some feedback offline and this is an updated version of > the test. > > It's using EXPECT instead of ASSERT to allow all test cases to be > executed even > in the presence of failures and the test cases has been renamed. > > I also updated the copyright date since I learned that the copyright > should > follow the code, not the file, and we are moving older code into a new > file. > > New webrev: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.01/ The #includes in test_quicksort.cpp need to be sorted. Also, you should remove the include for allocation.hpp, since you include the .inline.hpp. Otherwise, this looks good to me! Thanks, Marcus > ----------------------------------------------------------- > > (I have moved the webrev to the right place and updated the link i the > mail above.) > > Rachel replied and suggested to add parentheses in lines 57 and 58 to > ease visual scanning. I have added that in my local change. > > 57 bool a_is_odd = ((a % 2) == 1); > 58 bool b_is_odd = ((b % 2) == 1); > > > I now have three reviewer approvals for this change but no Reviewer. > Could someone please have a look? > > Thanks, > /Jesper > > > Den 25/8/16 kl. 18:55, skrev Jesper Wilhelmsson: >> Hi, >> >> Please review this test conversion of the quicksort tests to GTest. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8156800 >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.00/ >> >> Thanks, >> /Jesper From jesper.wilhelmsson at oracle.com Wed Oct 19 14:21:05 2016 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 19 Oct 2016 16:21:05 +0200 Subject: RFR: JDK-8156800 - Convert QuickSort_test to GTest In-Reply-To: <65a1a78d-778d-2c0d-a9a6-a5d88cb5c885@oracle.com> References: <1fdc5bcd-77bc-08c9-9640-cf3a9b4c3e99@oracle.com> <65a1a78d-778d-2c0d-a9a6-a5d88cb5c885@oracle.com> Message-ID: <495e26b3-d392-9c17-09bd-e525d58917ea@oracle.com> Thanks Marcus! I'll sort and remove before pushing. /Jesper Den 19/10/16 kl. 16:17, skrev Marcus Larsson: > Hi Jesper, > > > On 10/19/2016 12:30 PM, Jesper Wilhelmsson wrote: >> Hi, >> >> I managed to mix up the bugIDs and sent an update to this webrev to the TestOS >> GTest conversion in this thread: >> >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/024911.html >> >> Inlining the mail below for clarity: >> >> ----------------------------------------------------------- >> Kirill gave me some feedback offline and this is an updated version of the test. >> >> It's using EXPECT instead of ASSERT to allow all test cases to be executed even >> in the presence of failures and the test cases has been renamed. >> >> I also updated the copyright date since I learned that the copyright should >> follow the code, not the file, and we are moving older code into a new file. >> >> New webrev: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.01/ > > The #includes in test_quicksort.cpp need to be sorted. Also, you should remove > the include for allocation.hpp, since you include the .inline.hpp. > > Otherwise, this looks good to me! > > Thanks, > Marcus > >> ----------------------------------------------------------- >> >> (I have moved the webrev to the right place and updated the link i the mail >> above.) >> >> Rachel replied and suggested to add parentheses in lines 57 and 58 to ease >> visual scanning. I have added that in my local change. >> >> 57 bool a_is_odd = ((a % 2) == 1); >> 58 bool b_is_odd = ((b % 2) == 1); >> >> >> I now have three reviewer approvals for this change but no Reviewer. Could >> someone please have a look? >> >> Thanks, >> /Jesper >> >> >> Den 25/8/16 kl. 18:55, skrev Jesper Wilhelmsson: >>> Hi, >>> >>> Please review this test conversion of the quicksort tests to GTest. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8156800 >>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.00/ >>> >>> Thanks, >>> /Jesper > From kirill.zhaldybin at oracle.com Wed Oct 19 16:11:12 2016 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Wed, 19 Oct 2016 19:11:12 +0300 Subject: RFR: JDK-8156800 - Convert QuickSort_test to GTest In-Reply-To: <1fdc5bcd-77bc-08c9-9640-cf3a9b4c3e99@oracle.com> References: <1fdc5bcd-77bc-08c9-9640-cf3a9b4c3e99@oracle.com> Message-ID: <58079B20.2070700@oracle.com> Jesper, Thank you for addressing my comments. Looks good to me. Regards, Kirill PS I am not a Reviewer On 19.10.2016 13:30, Jesper Wilhelmsson wrote: > Hi, > > I managed to mix up the bugIDs and sent an update to this webrev to the > TestOS GTest conversion in this thread: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/024911.html > > Inlining the mail below for clarity: > > ----------------------------------------------------------- > Kirill gave me some feedback offline and this is an updated version of > the test. > > It's using EXPECT instead of ASSERT to allow all test cases to be > executed even > in the presence of failures and the test cases has been renamed. > > I also updated the copyright date since I learned that the copyright should > follow the code, not the file, and we are moving older code into a new > file. > > New webrev: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.01/ > ----------------------------------------------------------- > > (I have moved the webrev to the right place and updated the link i the > mail above.) > > Rachel replied and suggested to add parentheses in lines 57 and 58 to > ease visual scanning. I have added that in my local change. > > 57 bool a_is_odd = ((a % 2) == 1); > 58 bool b_is_odd = ((b % 2) == 1); > > > I now have three reviewer approvals for this change but no Reviewer. > Could someone please have a look? > > Thanks, > /Jesper > > > Den 25/8/16 kl. 18:55, skrev Jesper Wilhelmsson: >> Hi, >> >> Please review this test conversion of the quicksort tests to GTest. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8156800 >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8156800/webrev.00/ >> >> Thanks, >> /Jesper From vladimir.kozlov at oracle.com Wed Oct 19 16:44:28 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 19 Oct 2016 09:44:28 -0700 Subject: [8u] RFR for JDK-8134389: Crash in HotSpot with jvm.dll+0x42b48 ciObjectFactory::create_new_metadata In-Reply-To: References: Message-ID: In ciMethod.hpp you duplicated comment line: + // Given a certain calling environment, find the monomorphic target // Given a certain calling environment, find the monomorphic target Otherwise looks good. Thanks, Vladimir K On 10/19/16 12:53 AM, Shafi Ahmad wrote: > Hi All, > > Please review the backport of 'JDK-8134389: Crash in HotSpot with jvm.dll+0x42b48 ciObjectFactory::create_new_metadata' to jdk8u-dev. > > Please note that backport is not clean as I was getting build failure due to: > Formal parameter 'ignore_return' in method GraphBuilder::method_return is added in the fix of https://bugs.openjdk.java.net/browse/JDK-8164122. > The current code change is done on top of aforesaid bug fix and this formal parameter is referenced in this code change. > * if (x != NULL && !ignore_return) { * > > Author of this code change suggested me, we can safely remove this addition conditional expression ' && !ignore_return'. > > open webrev: http://cr.openjdk.java.net/~shshahma/8134389/webrev.00/ > jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8134389 > jdk9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/4191b33b3629 > > testing: Passes JPRT, jtreg on Linux [amd64] and newly added test case > > Regards, > Shafi > From vladimir.kozlov at oracle.com Wed Oct 19 17:00:26 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 19 Oct 2016 10:00:26 -0700 Subject: RFR: JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above processors In-Reply-To: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> References: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> Message-ID: Hi Martin, Changes are incomplete. The code which set S_family flag is not there. It should be in vm_version_solaris_sparc.cpp. Thanks, Vladimir On 10/19/16 1:30 AM, Martin Walsh wrote: > Could I get a review for the following bug please: > > JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above > processors > > https://bugs.openjdk.java.net/browse/JDK-8164002 > > Webrev is available here: > > http://cr.openjdk.java.net/~mwalsh/JDK-8164004/ > > > The changes add support for the SPARC-S processors, meaning the correct > performance optimizations are made. > > > Thanks, > > Martin From 1072213404 at qq.com Wed Oct 19 12:35:32 2016 From: 1072213404 at qq.com (=?gb18030?B?tvHB6cbvyr8=?=) Date: Wed, 19 Oct 2016 20:35:32 +0800 Subject: please help me about understanding method "OrderAccess::acquire() and OrderAccess::acquire()" Message-ID: src/os_cpu/linux_x86/vm/orderAccess_linux_x86.inline.hpp inline void OrderAccess::acquire() { volatile intptr_t local_dummy; #ifdef AMD64 __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory"); #else __asm__ volatile ("movl 0(%%esp),%0" : "=r" (local_dummy) : : "memory"); #endif // AMD64 } inline void OrderAccess::acquire() { // Avoid hitting the same cache-line from // different threads. volatile jint local_dummy = 0; } I have a few questions: 1,does gcc support the c++ keyword 'volatile' aquire/release sematics? if question 1's answer is 'yes', then i can understand the implemetation of method 'OrderAccess::acquire()', 2, about the part 0f ' __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory");' the 'volatile' prevents compiler optimize, the 'memory' ensure compiler no-reordering, then what about ""movq 0(%%rsp), %0" : "=r" (local_dummy) "? what's this part effect? and the local_dummywas declared as 'volatile ',is it necessary? thank you so so much! From vladimir.kozlov at oracle.com Wed Oct 19 17:04:01 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 19 Oct 2016 10:04:01 -0700 Subject: RFR: JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above processors In-Reply-To: References: <40b1ff9f-6566-325c-e20d-15bf393344bd@oracle.com> Message-ID: <37a4d113-b309-5f87-d9d4-8319c76bd61c@oracle.com> Ignore my comment. The CPU name check now is done in vm_version_sparc.cpp and you correctly fixed it. Reviewed. I will sponsor it. Thanks, Vladimir On 10/19/16 10:00 AM, Vladimir Kozlov wrote: > Hi Martin, > > Changes are incomplete. The code which set S_family flag is not there. > It should be in vm_version_solaris_sparc.cpp. > > Thanks, > Vladimir > > On 10/19/16 1:30 AM, Martin Walsh wrote: >> Could I get a review for the following bug please: >> >> JDK-8164002: Add a new CPU family (S_family) for SPARC S7 and above >> processors >> >> https://bugs.openjdk.java.net/browse/JDK-8164002 >> >> Webrev is available here: >> >> http://cr.openjdk.java.net/~mwalsh/JDK-8164004/ >> >> >> The changes add support for the SPARC-S processors, meaning the correct >> performance optimizations are made. >> >> >> Thanks, >> >> Martin From kim.barrett at oracle.com Wed Oct 19 17:37:34 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 19 Oct 2016 13:37:34 -0400 Subject: please help me about understanding method "OrderAccess::acquire() and OrderAccess::acquire()" In-Reply-To: References: Message-ID: > On Oct 19, 2016, at 8:35 AM, ???? <1072213404 at qq.com> wrote: > > src/os_cpu/linux_x86/vm/orderAccess_linux_x86.inline.hpp > inline void OrderAccess::acquire() { > volatile intptr_t local_dummy; > #ifdef AMD64 > __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory"); > #else > __asm__ volatile ("movl 0(%%esp),%0" : "=r" (local_dummy) : : "memory"); > #endif // AMD64 > } FYI, this is an old implementation, which was replaced in JDK 9 more than a year and a half ago. > inline void OrderAccess::acquire() { > // Avoid hitting the same cache-line from > // different threads. > volatile jint local_dummy = 0; > } > > > I have a few questions: > 1,does gcc support the c++ keyword 'volatile' aquire/release sematics? > > > if question 1's answer is 'yes', then i can understand the implemetation of > method 'OrderAccess::acquire()', > > > 2, about the part 0f ' __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory");' > the 'volatile' prevents compiler optimize, > the 'memory' ensure compiler no-reordering, > > > then what about ""movq 0(%%rsp), %0" : "=r" (local_dummy) "? what's this part effect? and the local_dummywas declared as 'volatile ',is it necessary? > > > thank you so so much! From david.holmes at oracle.com Wed Oct 19 22:39:54 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Oct 2016 08:39:54 +1000 Subject: please help me about understanding method "OrderAccess::acquire() and OrderAccess::acquire()" In-Reply-To: References: Message-ID: On 19/10/2016 10:35 PM, ???? wrote: > src/os_cpu/linux_x86/vm/orderAccess_linux_x86.inline.hpp > inline void OrderAccess::acquire() { > volatile intptr_t local_dummy; > #ifdef AMD64 > __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory"); > #else > __asm__ volatile ("movl 0(%%esp),%0" : "=r" (local_dummy) : : "memory"); > #endif // AMD64 > } > > > inline void OrderAccess::acquire() { > // Avoid hitting the same cache-line from > // different threads. > volatile jint local_dummy = 0; > } As Kim stated these are old implementations. The intent was to produce some code to force a "compiler barrier" so that the acquire() semantics needed on x86 would exist - which is just a compiler barrier. The new code relies on a more direct gcc technique: // A compiler barrier, forcing the C++ compiler to invalidate all memory assumptions static inline void compiler_barrier() { __asm__ volatile ("" : : : "memory"); } inline void OrderAccess::acquire() { compiler_barrier(); } > > I have a few questions: > 1,does gcc support the c++ keyword 'volatile' aquire/release sematics? > > > if question 1's answer is 'yes', then i can understand the implemetation of > method 'OrderAccess::acquire()', Not sure exactly what you mean, but we do not rely on any C++ memory model operations in the hotspot code - acquire/release semanics - we just use volatile to flag variables that should not be optimized and use OrderAccess operations to explicitly enforce any memory ordering requirements. > > 2, about the part 0f ' __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory");' > the 'volatile' prevents compiler optimize, > the 'memory' ensure compiler no-reordering, Basically yes that was the intent. The implementation has changed over time. > > then what about ""movq 0(%%rsp), %0" : "=r" (local_dummy) "? what's this part effect? and the local_dummywas declared as 'volatile ',is it necessary? That was to do the actual assignment to which the volatile and memory would apply. This part is no longer necessary. David > > thank you so so much! > From david.holmes at oracle.com Thu Oct 20 03:20:51 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Oct 2016 13:20:51 +1000 Subject: =?UTF-8?B?UmU6IOWbnuWkje+8miBwbGVhc2UgaGVscCBtZSBhYm91dCB1bmRlcnN0?= =?UTF-8?Q?anding_method_=22OrderAccess::acquire=28=29and_OrderAccess::acqui?= =?UTF-8?B?cmUoKSI=?= In-Reply-To: References: Message-ID: <08f5dbf6-9f1f-9d13-68c1-082add8e2913@oracle.com> On 20/10/2016 12:33 PM, ???? wrote: > > thank you so much ,David ! > it's my fault not to clarify the problem, > -------------------------------------------------- > the openjdk8 code shuold be : > inline void OrderAccess::acquire() { > volatile intptr_t local_dummy; > #ifdef AMD64 > __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory"); > #else > __asm__ volatile ("movl 0(%%esp),%0" : "=r" (local_dummy) : : "memory"); > #endif // AMD64 > } > > inline void OrderAccess::release() { ----------- copied wrong name before Ah! I should have realized that. :) > // Avoid hitting the same cache-line from > // different threads. > volatile jint local_dummy = 0; > } > -------------------------------------------------- > 1, > just for openjdk version 8, > you said that "The intent was to produce some code to force a "compiler > barrier" so that the acquire() semantics needed on x86 would exist > force a "compiler barrier" ", > the keyword ' asm volatile' and 'memory' are not enough to force > a "compiler barrier"? > why is "movq 0(%%rsp), %0" : "=r" (local_dummy) " still needed? > and the local_dummy is a volatile filed, > if local_dummy without volatile qualifier , the code '__asm__ volatile > ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory");? still can > force a "compiler barrier"? > > and which part of '__asm__ volatile ("movq 0(%%rsp), %0" : "=r" > (local_dummy) : : "memory");? force a 'compiler barrier', > 'volatile ' or "movq 0(%%rsp), %0" : "=r" (local_dummy) or 'memory' > or the three together? You have to remember that this code has very old origins and that compilers have not been very clear when it comes to things like compiler barriers and memory ordering, and for a long time we were stuck using fairly old compilers. So what we have had in the past is a volatile load using asm "memory" to obtain the necessary "compiler barrier" to effect the acquire() semantics, and a volatile store to effect the "compiler barrier" for the release() semantics. For JDK 9, with our updated compilers, we have moved to the more direct compiler_barrier code for use with gcc: static inline void compiler_barrier() { __asm__ volatile ("" : : : "memory"); } and as you can see no actual store or load is needed any more. > ----- > 2, > on x86, for acquire() , foring a 'comliler barrier' is enough? > 'hardware barrier' not needed? > for the reason of x86 ensure loadload and loadstore? x86 has total-store-ordering so no hardware barriers are needed. As long as the compiler has not reordered the program statements all stores will happen in the written order, so nothing needs to happen to achieve "acquire" semantics. > --- > 3, > i am confused about OrderAccess::release() ; > in windows_x86 version the comment is "// A volatile store has release > semantics." > in linux_x86 version the comment is "// Avoid hitting the same > cache-line from > // different threads." > what's the difference about "volatile jint local_dummy = 0;" between > windows and linux? > for windows i can find something in the c++ doc > that from vc++ 2005 ,vc compiler supports volatile acquire/release > sematics, > but for linux ,i can not find anything about gcc describing volatile > acquire/release sematics, > > how does "volatile jint local_dummy = 0;" work on linux_x86? As you note VS 2005 explicitly provides acquire/release semantics for volatile read/write. So to tell the VS compiler to implement a release() operation we simply do: volatile int dummy = 0; and it will generate whatever code is needed to achieve release semantics (likely no code at all as it can elide the actual write to the dummy variable). For gcc there was no such explicit statement regarding release/acquire. So it was based on the observed behaviour and informal commentary on the gcc internals. It was found that a volatile write was sufficient to get the desired effects. The comment: // Avoid hitting the same cache-line from // different threads is unrelated to the acquire/release. In the past these operations wrote to a shared static variable. That turned out to be a performance issue because of cache contention. So the code was changed to write to a local variable instead - and the behaviour of the compiler was verified by inspection/observation. Hope that clarifies things. David > > thank you so so ... much ! > > > > ------------------ ???? ------------------ > *???:* "David Holmes";; > *????:* 2016?10?20?(???) ??6:39 > *???:* "????"<1072213404 at qq.com>; > "hotspot-dev"; > *??:* Re: please help me about understanding method > "OrderAccess::acquire()and OrderAccess::acquire()" > > On 19/10/2016 10:35 PM, ???? wrote: >> src/os_cpu/linux_x86/vm/orderAccess_linux_x86.inline.hpp >> inline void OrderAccess::acquire() { >> volatile intptr_t local_dummy; >> #ifdef AMD64 >> __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : > "memory"); >> #else >> __asm__ volatile ("movl 0(%%esp),%0" : "=r" (local_dummy) : : "memory"); >> #endif // AMD64 >> } >> >> >> inline void OrderAccess::acquire() { ---------- should be release() >> // Avoid hitting the same cache-line from >> // different threads. >> volatile jint local_dummy = 0; >> } > > As Kim stated these are old implementations. The intent was to produce > some code to force a "compiler barrier" so that the acquire() semantics > needed on x86 would exist - which is just a compiler barrier. The new > code relies on a more direct gcc technique: > > // A compiler barrier, forcing the C++ compiler to invalidate all memory > assumptions > static inline void compiler_barrier() { > __asm__ volatile ("" : : : "memory"); > } > inline void OrderAccess::acquire() { compiler_barrier(); } > >> >> I have a few questions: >> 1,does gcc support the c++ keyword 'volatile' aquire/release sematics? >> >> >> if question 1's answer is 'yes', then i can understand the > implemetation of >> method 'OrderAccess::acquire()', > > Not sure exactly what you mean, but we do not rely on any C++ memory > model operations in the hotspot code - acquire/release semanics - we > just use volatile to flag variables that should not be optimized and use > OrderAccess operations to explicitly enforce any memory ordering > requirements. > >> >> 2, about the part 0f ' __asm__ volatile ("movq 0(%%rsp), %0" : "=r" > (local_dummy) : : "memory");' >> the 'volatile' prevents compiler optimize, >> the 'memory' ensure compiler no-reordering, > > Basically yes that was the intent. The implementation has changed over time. > >> >> then what about ""movq 0(%%rsp), %0" : "=r" (local_dummy) "? what's > this part effect? and the local_dummywas declared as 'volatile ',is it > necessary? > > That was to do the actual assignment to which the volatile and memory > would apply. This part is no longer necessary. > > David > >> >> thank you so so much! >> From david.holmes at oracle.com Thu Oct 20 04:21:52 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Oct 2016 14:21:52 +1000 Subject: =?UTF-8?B?UmU6IOWbnuWkje+8miDlm57lpI3vvJogcGxlYXNlIGhlbHAgbWUgYWJv?= =?UTF-8?Q?ut_understanding_method_=22OrderAccess::acquire=28=29and_OrderAcc?= =?UTF-8?B?ZXNzOjphY3F1aXJlKCki?= In-Reply-To: References: Message-ID: <28db3564-69f8-caa7-c708-5bbf38963380@oracle.com> Adding back hotspot-dev mailing list ... On 20/10/2016 1:52 PM, ???? wrote: > > Thank you very much as i could, David ! > > two more question , > 1, simple one, > is there a download link of openjdk9 source code like openjdk8 ? > i just find version8 http://download.java.net/openjdk/jdk8/ I wasn't aware of the jdk8 download link. The main location for the sources are the Mercurial repositories: http://hg.openjdk.java.net/ > 2, about volatile implemetation, > src/share/vm/c1/c1_LIRGenerator.cpp > void LIRGenerator::do_StoreField(StoreField* x) > void LIRGenerator::do_LoadField(LoadField* x) > > i add "cout << " in below methods > volatile_field_store > volatile_field_load > > this is my test class > class Test extends Thread { > > volatile int keepRunning = 1; > > public void run() { > while (keepRunning == 10000000) { > } > System.out.println(Thread.currentThread().getName()+ " Thread > terminated."); > } > > public static void main(String[] args) { > try{ > Test t = new Test(); > t.start(); > > for(int i=0; i<10000000; i++){ > t.keepRunning += 1; > } > > }catch(Exception e){ > } > System.out.println(Thread.currentThread().getName() +" > keepRunning set to false."); > } > } > result: volatile_field_store print does not occur every time, why ? You added the cout to the C1 compiler code, so it will only show up when running compiled code - initially your program will execute in the interpreter then eventually the loop will be compiled via on-stack-replacement and you will start to see the cout results. You can try running with -Xcomp to force everything to be compiled on startup. > if i change for(int i=0; i<10000000; i++){ > t.keepRunning += 1; > } > the 10000000 to 20, > then volatile_field_store never occured, You never compile the loop with a count of only 20. Cheers, David > > i learn the implemetation place of volatile from google, > and add cout to verify if it's true, > but i am confused about the testing result, > if i am wrong at some place? > > thank you ...........!!!!!!!!!! > > > > ------------------ ???? ------------------ > *???:* "David Holmes";; > *????:* 2016?10?20?(???) ??11:20 > *???:* "????"<1072213404 at qq.com>; "hotspot-dev > developers"; > *??:* Re: ??? please help me about understanding method > "OrderAccess::acquire()and OrderAccess::acquire()" > > On 20/10/2016 12:33 PM, ???? wrote: >> >> thank you so much ,David ! >> it's my fault not to clarify the problem, >> -------------------------------------------------- >> the openjdk8 code shuold be : >> inline void OrderAccess::acquire() { >> volatile intptr_t local_dummy; >> #ifdef AMD64 >> __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : > "memory"); >> #else >> __asm__ volatile ("movl 0(%%esp),%0" : "=r" (local_dummy) : : "memory"); >> #endif // AMD64 >> } >> >> inline void OrderAccess::release() { ----------- copied wrong name > before > > Ah! I should have realized that. :) > >> // Avoid hitting the same cache-line from >> // different threads. >> volatile jint local_dummy = 0; >> } >> -------------------------------------------------- >> 1, >> just for openjdk version 8, >> you said that "The intent was to produce some code to force a "compiler >> barrier" so that the acquire() semantics needed on x86 would exist >> force a "compiler barrier" ", >> the keyword ' asm volatile' and 'memory' are not enough to force >> a "compiler barrier"? >> why is "movq 0(%%rsp), %0" : "=r" (local_dummy) " still needed? >> and the local_dummy is a volatile filed, >> if local_dummy without volatile qualifier , the code '__asm__ volatile >> ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : "memory");? still can >> force a "compiler barrier"? >> >> and which part of '__asm__ volatile ("movq 0(%%rsp), %0" : "=r" >> (local_dummy) : : "memory");? force a 'compiler barrier', >> 'volatile ' or "movq 0(%%rsp), %0" : "=r" (local_dummy) or 'memory' >> or the three together? > > You have to remember that this code has very old origins and that > compilers have not been very clear when it comes to things like compiler > barriers and memory ordering, and for a long time we were stuck using > fairly old compilers. So what we have had in the past is a volatile load > using asm "memory" to obtain the necessary "compiler barrier" to effect > the acquire() semantics, and a volatile store to effect the "compiler > barrier" for the release() semantics. > > For JDK 9, with our updated compilers, we have moved to the more direct > compiler_barrier code for use with gcc: > > static inline void compiler_barrier() { > __asm__ volatile ("" : : : "memory"); > } > > and as you can see no actual store or load is needed any more. > >> ----- >> 2, >> on x86, for acquire() , foring a 'comliler barrier' is enough? >> 'hardware barrier' not needed? >> for the reason of x86 ensure loadload and loadstore? > > x86 has total-store-ordering so no hardware barriers are needed. As long > as the compiler has not reordered the program statements all stores will > happen in the written order, so nothing needs to happen to achieve > "acquire" semantics. > >> --- >> 3, >> i am confused about OrderAccess::release() ; >> in windows_x86 version the comment is "// A volatile store has release >> semantics." >> in linux_x86 version the comment is "// Avoid hitting the same >> cache-line from >> // different threads." >> what's the difference about "volatile jint local_dummy = 0;" between >> windows and linux? >> for windows i can find something in the c++ doc >> that from vc++ 2005 ,vc compiler supports volatile acquire/release >> sematics, >> but for linux ,i can not find anything about gcc describing volatile >> acquire/release sematics, >> >> how does "volatile jint local_dummy = 0;" work on linux_x86? > > As you note VS 2005 explicitly provides acquire/release semantics for > volatile read/write. So to tell the VS compiler to implement a release() > operation we simply do: > > volatile int dummy = 0; > > and it will generate whatever code is needed to achieve release > semantics (likely no code at all as it can elide the actual write to the > dummy variable). > > For gcc there was no such explicit statement regarding release/acquire. > So it was based on the observed behaviour and informal commentary on the > gcc internals. It was found that a volatile write was sufficient to get > the desired effects. > > The comment: > > // Avoid hitting the same cache-line from > // different threads > > is unrelated to the acquire/release. In the past these operations wrote > to a shared static variable. That turned out to be a performance issue > because of cache contention. So the code was changed to write to a local > variable instead - and the behaviour of the compiler was verified by > inspection/observation. > > Hope that clarifies things. > > David > >> >> thank you so so ... much ! >> >> >> >> ------------------ ???? ------------------ >> *???:* "David Holmes";; >> *????:* 2016?10?20?(???) ??6:39 >> *???:* "????"<1072213404 at qq.com>; >> "hotspot-dev"; >> *??:* Re: please help me about understanding method >> "OrderAccess::acquire()and OrderAccess::acquire()" >> >> On 19/10/2016 10:35 PM, ???? wrote: >>> src/os_cpu/linux_x86/vm/orderAccess_linux_x86.inline.hpp >>> inline void OrderAccess::acquire() { >>> volatile intptr_t local_dummy; >>> #ifdef AMD64 >>> __asm__ volatile ("movq 0(%%rsp), %0" : "=r" (local_dummy) : : >> "memory"); >>> #else >>> __asm__ volatile ("movl 0(%%esp),%0" : "=r" (local_dummy) : : > "memory"); >>> #endif // AMD64 >>> } >>> >>> >>> inline void OrderAccess::acquire() { ---------- should be release() >>> // Avoid hitting the same cache-line from >>> // different threads. >>> volatile jint local_dummy = 0; >>> } >> >> As Kim stated these are old implementations. The intent was to produce >> some code to force a "compiler barrier" so that the acquire() semantics >> needed on x86 would exist - which is just a compiler barrier. The new >> code relies on a more direct gcc technique: >> >> // A compiler barrier, forcing the C++ compiler to invalidate all memory >> assumptions >> static inline void compiler_barrier() { >> __asm__ volatile ("" : : : "memory"); >> } >> inline void OrderAccess::acquire() { compiler_barrier(); } >> >>> >>> I have a few questions: >>> 1,does gcc support the c++ keyword 'volatile' aquire/release sematics? >>> >>> >>> if question 1's answer is 'yes', then i can understand the >> implemetation of >>> method 'OrderAccess::acquire()', >> >> Not sure exactly what you mean, but we do not rely on any C++ memory >> model operations in the hotspot code - acquire/release semanics - we >> just use volatile to flag variables that should not be optimized and use >> OrderAccess operations to explicitly enforce any memory ordering >> requirements. >> >>> >>> 2, about the part 0f ' __asm__ volatile ("movq 0(%%rsp), %0" : "=r" >> (local_dummy) : : "memory");' >>> the 'volatile' prevents compiler optimize, >>> the 'memory' ensure compiler no-reordering, >> >> Basically yes that was the intent. The implementation has changed over > time. >> >>> >>> then what about ""movq 0(%%rsp), %0" : "=r" (local_dummy) "? what's >> this part effect? and the local_dummywas declared as 'volatile ',is it >> necessary? >> >> That was to do the actual assignment to which the volatile and memory >> would apply. This part is no longer necessary. >> >> David >> >>> >>> thank you so so much! >>> From shafi.s.ahmad at oracle.com Thu Oct 20 04:24:44 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 19 Oct 2016 21:24:44 -0700 (PDT) Subject: [8u] RFR for JDK-8134389: Crash in HotSpot with jvm.dll+0x42b48 ciObjectFactory::create_new_metadata In-Reply-To: References: Message-ID: <2e1de7f0-cc65-47f7-9f97-cb0e56dacfe1@default> Thank you Vladimir for the review. Please find the updated webrev link. http://cr.openjdk.java.net/~shshahma/8134389/webrev.01/ All, May I get 2nd review for this. Regards, Shafi > -----Original Message----- > From: Vladimir Kozlov > Sent: Wednesday, October 19, 2016 10:14 PM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Cc: Vladimir Ivanov; Jamsheed C M > Subject: Re: [8u] RFR for JDK-8134389: Crash in HotSpot with jvm.dll+0x42b48 > ciObjectFactory::create_new_metadata > > In ciMethod.hpp you duplicated comment line: > > + // Given a certain calling environment, find the monomorphic target > // Given a certain calling environment, find the monomorphic target > > Otherwise looks good. > > Thanks, > Vladimir K > > On 10/19/16 12:53 AM, Shafi Ahmad wrote: > > Hi All, > > > > Please review the backport of 'JDK-8134389: Crash in HotSpot with > jvm.dll+0x42b48 ciObjectFactory::create_new_metadata' to jdk8u-dev. > > > > Please note that backport is not clean as I was getting build failure due to: > > Formal parameter 'ignore_return' in method GraphBuilder::method_return > is added in the fix of https://bugs.openjdk.java.net/browse/JDK-8164122. > > The current code change is done on top of aforesaid bug fix and this formal > parameter is referenced in this code change. > > * if (x != NULL && !ignore_return) { * > > > > Author of this code change suggested me, we can safely remove this > addition conditional expression ' && !ignore_return'. > > > > open webrev: http://cr.openjdk.java.net/~shshahma/8134389/webrev.00/ > > jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8134389 > > jdk9 changeset: http://hg.openjdk.java.net/jdk9/hs- > comp/hotspot/rev/4191b33b3629 > > > > testing: Passes JPRT, jtreg on Linux [amd64] and newly added test case > > > > Regards, > > Shafi > > From shafi.s.ahmad at oracle.com Thu Oct 20 04:38:08 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 19 Oct 2016 21:38:08 -0700 (PDT) Subject: [8u] RFR for JDK-8134918 - C2: Type speculation produces mismatched unsafe accesses In-Reply-To: <05a495b7-7865-ffbf-d182-c15aeadbd4b0@oracle.com> References: <05a495b7-7865-ffbf-d182-c15aeadbd4b0@oracle.com> Message-ID: Thanks Vladimir. I will create dependent backport of 1. https://bugs.openjdk.java.net/browse/JDK-8136473 2. https://bugs.openjdk.java.net/browse/JDK-8155781 3. https://bugs.openjdk.java.net/browse/JDK-8162101 Regards, Shafi > -----Original Message----- > From: Vladimir Kozlov > Sent: Wednesday, October 19, 2016 8:27 AM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8134918 - C2: Type speculation produces > mismatched unsafe accesses > > Hi Shafi, > > You should also consider backporting following related fixes: > > https://bugs.openjdk.java.net/browse/JDK-8155781 > https://bugs.openjdk.java.net/browse/JDK-8162101 > > Otherwise you may hit asserts added by 8134918 changes. > > Thanks, > Vladimir > > On 10/17/16 3:12 AM, Shafi Ahmad wrote: > > Hi All, > > > > Please review the backport of JDK-8134918 - C2: Type speculation produces > mismatched unsafe accesses to jdk8u-dev. > > > > Please note that backport is not clean and the conflict is due to: > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/9108fab781a4#l5.165 > > > > Getting debug build failure because of: > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/9108fab781a4#l5.155 > > > > The above changes are done under bug# 'JDK-8136473: failed: no > mismatched stores, except on raw memory: StoreB StoreI' which is not back > ported to jdk8u and the current backport is on top of above change. > > > > Please note that I am not sure if there is any dependency between these > two changesets. > > > > open webrev: http://cr.openjdk.java.net/~shshahma/8134918/webrev.00/ > > jdk9 bug link: https://bugs.openjdk.java.net/browse/JDK-8134918 > > jdk9 changeset: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/79dae2cd00ef > > > > testing: Passes JPRT, jtreg not completed > > > > Regards, > > Shafi > > From 1072213404 at qq.com Thu Oct 20 04:43:08 2016 From: 1072213404 at qq.com (=?gb18030?B?tvHB6cbvyr8=?=) Date: Thu, 20 Oct 2016 12:43:08 +0800 Subject: about openjdk9 source code tar file download link Message-ID: is there a download link of openjdk9 source code like openjdk8 ? i just find version8 http://download.java.net/openjdk/jdk8/ From patrick at reini.net Thu Oct 20 06:14:05 2016 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 20 Oct 2016 08:14:05 +0200 Subject: about openjdk9 source code tar file download link In-Reply-To: References: Message-ID: You can find the latest early build here: https://jdk9.java.net/download/ The source itself can be cloned using mercurial form the OpenJDK website see http://openjdk.java.net/faq/ On 2016-10-20 06:43, ???? wrote: > is there a download link of openjdk9 source code like openjdk8 ? > i just find version8 http://download.java.net/openjdk/jdk8/ -Patrick From tobias.hartmann at oracle.com Thu Oct 20 06:50:22 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 20 Oct 2016 08:50:22 +0200 Subject: [9] RFR(S): 8164612: NoSuchMethodException when method name contains NULL or Latin-1 supplement character In-Reply-To: <580622AE.9080802@oracle.com> References: <580622AE.9080802@oracle.com> Message-ID: <5808692E.9090905@oracle.com> Hi, since this is affecting runtime code, could someone from the runtime team please have a look as well? Thanks, Tobias On 18.10.2016 15:25, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8164612 > http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/ > > The test executes Java Script code that defines getter methods containing Latin-1 supplement characters (0x80 - 0xFF). Those methods are registered at runtime through anonymous classes via Unsafe_DefineAnonymousClass. When calling a method, the VM fails with a NoSuchMethodException in MethodHandles::resolve_MemberName(). > > The failure happens while looking up the method name symbol in java_lang_String::as_symbol_or_null() [1]: > 544 jbyte* position = (length == 0) ? NULL : value->byte_at_addr(0); > 545 const char* base = UNICODE::as_utf8(position, length); > 546 return SymbolTable::probe(base, length); > > If Compact Strings is enabled, we pass the Latin-1 encoded method name to UNICODE::as_utf8() and probe for the UTF-8 String in the SymbolTable. Since the Latin-1 method name contains non-ASCII characters, the length of the resulting UTF-8 String is larger (characters >= 0x80 are encoded as two bytes in UTF-8). However, we pass the shorter Latin-1 length to SymbolTable::probe() resulting in a lookup failure. > > I fixed this by passing the String length by reference to UNICODE::as_utf8(). I also refactored the related code in utf8.cpp, added comments and updated the callers. > > Tested with regression test and hs-comp PIT RBT (running). > > Thanks, > Tobias, > > [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/652537a80080/src/share/vm/classfile/javaClasses.cpp#l535 > From staffan.larsen at oracle.com Thu Oct 20 11:42:05 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 20 Oct 2016 13:42:05 +0200 Subject: RFR(s): 8168409 Update list of tools run by the jtreg timeouthandler Message-ID: <1BBA33BA-DD17-4B46-9FD1-3EC4AF1EBD33@oracle.com> Please review this small update to the list of tools that the jtreg timeouthandler runs when a timeout is hit. This change removes all calls to the old SA-tools and instead uses Diagnostic Commands to gather the same information. The possible drawback is that Diagnostic Commands do not work on processes where the JVM is completely hung, but in that case we can fallback to running the SA-tools on the core file that is also generated. webrev: http://cr.openjdk.java.net/~sla/8168409/webrev.00 bug: https://bugs.openjdk.java.net/browse/JDK-8168409 Thanks, /Staffan From staffan.larsen at oracle.com Thu Oct 20 12:03:15 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 20 Oct 2016 14:03:15 +0200 Subject: RFR(XS): 8168412 Reduce buffering in jtreg timeouthandler Message-ID: <9CCA6BAF-F694-4FF2-888B-F4DE5334B276@oracle.com> Hi, Please review this small patch to the jtreg timeouthandler to remove some buffering. We create PrintWriters in a couple of places and this patch adds the second parameter to the constructor to enable automatic flushing of the PrintWriter. webrev: http://cr.openjdk.java.net/~sla/8168412/webrev.00 bug: https://bugs.openjdk.java.net/browse/JDK-8168412 Thanks, /Staffan From staffan.larsen at oracle.com Thu Oct 20 13:27:41 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 20 Oct 2016 15:27:41 +0200 Subject: RFR: 8168414 Various timeouthandler fixes Message-ID: When looking for some timeout handler problems I found a few things that should be fixed: * Strange use of Thread.currentThread().interrupt() * Add logging for timeouts * Milliseconds sometimes printed as microseconds or nanoseconds * Should use destroyForcibly() to terminate processes * No need to sleep in the last iteration when running a command multiple times * Disable timeout handling timeouts since we do that ourselves I didn?t want to file individual bugs for all of these, so I have lumped them together in one bug. bug: https://bugs.openjdk.java.net/browse/JDK-8168414 webrev: http://cr.openjdk.java.net/~sla/8168414/webrev.00 Thanks, /Staffan From david.holmes at oracle.com Thu Oct 20 13:41:34 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Oct 2016 23:41:34 +1000 Subject: RFR: 8168414 Various timeouthandler fixes In-Reply-To: References: Message-ID: <878580a7-7248-2e75-69ab-31ff735c0de0@oracle.com> On 20/10/2016 11:27 PM, Staffan Larsen wrote: > When looking for some timeout handler problems I found a few things that should be fixed: > > * Strange use of Thread.currentThread().interrupt() That isn't "strange" it is an idiomatic usage - if you can't propagate the InterruptedException to show your caller you have been interrupted then you re-assert the interrupt state. Cheers, David > * Add logging for timeouts > * Milliseconds sometimes printed as microseconds or nanoseconds > * Should use destroyForcibly() to terminate processes > * No need to sleep in the last iteration when running a command multiple times > * Disable timeout handling timeouts since we do that ourselves > > I didn?t want to file individual bugs for all of these, so I have lumped them together in one bug. > > bug: https://bugs.openjdk.java.net/browse/JDK-8168414 webrev: http://cr.openjdk.java.net/~sla/8168414/webrev.00 > > Thanks, > /Staffan > From staffan.larsen at oracle.com Thu Oct 20 13:44:11 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 20 Oct 2016 15:44:11 +0200 Subject: RFR: 8168414 Various timeouthandler fixes In-Reply-To: <878580a7-7248-2e75-69ab-31ff735c0de0@oracle.com> References: <878580a7-7248-2e75-69ab-31ff735c0de0@oracle.com> Message-ID: > On 20 Oct 2016, at 15:41, David Holmes wrote: > > On 20/10/2016 11:27 PM, Staffan Larsen wrote: >> When looking for some timeout handler problems I found a few things that should be fixed: >> >> * Strange use of Thread.currentThread().interrupt() > > That isn't "strange" it is an idiomatic usage - if you can't propagate the InterruptedException to show your caller you have been interrupted then you re-assert the interrupt state. OK, then. In this case it is less ?strange? than ?wrong?. The cases where this is done should not propagate the interrupt state - it only serves to throw unexpected exceptions higher up in the call-chain. > > Cheers, > David > >> * Add logging for timeouts >> * Milliseconds sometimes printed as microseconds or nanoseconds >> * Should use destroyForcibly() to terminate processes >> * No need to sleep in the last iteration when running a command multiple times >> * Disable timeout handling timeouts since we do that ourselves >> >> I didn?t want to file individual bugs for all of these, so I have lumped them together in one bug. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8168414 webrev: http://cr.openjdk.java.net/~sla/8168414/webrev.00 >> >> Thanks, >> /Staffan >> From marcus.larsson at oracle.com Thu Oct 20 13:50:44 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Thu, 20 Oct 2016 15:50:44 +0200 Subject: RFR(s): 8168409 Update list of tools run by the jtreg timeouthandler In-Reply-To: <1BBA33BA-DD17-4B46-9FD1-3EC4AF1EBD33@oracle.com> References: <1BBA33BA-DD17-4B46-9FD1-3EC4AF1EBD33@oracle.com> Message-ID: Hi, On 10/20/2016 01:42 PM, Staffan Larsen wrote: > Please review this small update to the list of tools that the jtreg timeouthandler runs when a timeout is hit. This change removes all calls to the old SA-tools and instead uses Diagnostic Commands to gather the same information. The possible drawback is that Diagnostic Commands do not work on processes where the JVM is completely hung, but in that case we can fallback to running the SA-tools on the core file that is also generated. > > webrev: http://cr.openjdk.java.net/~sla/8168409/webrev.00 Looks good to me. Thanks, Marcus > bug: https://bugs.openjdk.java.net/browse/JDK-8168409 > > Thanks, > /Staffan From marcus.larsson at oracle.com Thu Oct 20 13:58:01 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Thu, 20 Oct 2016 15:58:01 +0200 Subject: RFR(XS): 8168412 Reduce buffering in jtreg timeouthandler In-Reply-To: <9CCA6BAF-F694-4FF2-888B-F4DE5334B276@oracle.com> References: <9CCA6BAF-F694-4FF2-888B-F4DE5334B276@oracle.com> Message-ID: <6c23a9d5-4de3-44c8-3122-4d21c3eebca4@oracle.com> Hi, On 10/20/2016 02:03 PM, Staffan Larsen wrote: > Hi, > > Please review this small patch to the jtreg timeouthandler to remove some buffering. We create PrintWriters in a couple of places and this patch adds the second parameter to the constructor to enable automatic flushing of the PrintWriter. > > webrev: http://cr.openjdk.java.net/~sla/8168412/webrev.00 Looks good! Thanks, Marcus > bug: https://bugs.openjdk.java.net/browse/JDK-8168412 > > Thanks, > /Staffan > From marcus.larsson at oracle.com Thu Oct 20 14:17:27 2016 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Thu, 20 Oct 2016 16:17:27 +0200 Subject: RFR: 8168414 Various timeouthandler fixes In-Reply-To: References: Message-ID: <465d0b32-c657-1a51-6df8-62627dbf1ee4@oracle.com> Hi, On 10/20/2016 03:27 PM, Staffan Larsen wrote: > When looking for some timeout handler problems I found a few things that should be fixed: > > * Strange use of Thread.currentThread().interrupt() > * Add logging for timeouts > * Milliseconds sometimes printed as microseconds or nanoseconds > * Should use destroyForcibly() to terminate processes > * No need to sleep in the last iteration when running a command multiple times > * Disable timeout handling timeouts since we do that ourselves > > I didn?t want to file individual bugs for all of these, so I have lumped them together in one bug. > > bug: https://bugs.openjdk.java.net/browse/JDK-8168414 > webrev: http://cr.openjdk.java.net/~sla/8168414/webrev.00 Looks good to me! Thanks, Marcus > > Thanks, > /Staffan From staffan.larsen at oracle.com Thu Oct 20 19:36:31 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 20 Oct 2016 21:36:31 +0200 Subject: RFR(s): 8168409 Update list of tools run by the jtreg timeouthandler In-Reply-To: References: <1BBA33BA-DD17-4B46-9FD1-3EC4AF1EBD33@oracle.com> Message-ID: <48E4F267-DB83-4F3D-88A6-7A843808D960@oracle.com> Thanks Marcus for all three reviews! > On 20 Oct 2016, at 15:50, Marcus Larsson wrote: > > Hi, > > >> On 10/20/2016 01:42 PM, Staffan Larsen wrote: >> Please review this small update to the list of tools that the jtreg timeouthandler runs when a timeout is hit. This change removes all calls to the old SA-tools and instead uses Diagnostic Commands to gather the same information. The possible drawback is that Diagnostic Commands do not work on processes where the JVM is completely hung, but in that case we can fallback to running the SA-tools on the core file that is also generated. >> >> webrev: http://cr.openjdk.java.net/~sla/8168409/webrev.00 > > Looks good to me. > > Thanks, > Marcus > >> bug: https://bugs.openjdk.java.net/browse/JDK-8168409 >> >> Thanks, >> /Staffan > From dmitry.samersoff at oracle.com Thu Oct 20 20:12:15 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 20 Oct 2016 23:12:15 +0300 Subject: RFR(s): 8168409 Update list of tools run by the jtreg timeouthandler In-Reply-To: <1BBA33BA-DD17-4B46-9FD1-3EC4AF1EBD33@oracle.com> References: <1BBA33BA-DD17-4B46-9FD1-3EC4AF1EBD33@oracle.com> Message-ID: <4673b551-eb61-2f7f-a0bb-39964e5e7016@oracle.com> Staffan, Could you add jhsdb jstack --mixed --pid to get mixed stack trace when it possible? -Dmitry On 2016-10-20 14:42, Staffan Larsen wrote: > Please review this small update to the list of tools that the jtreg timeouthandler runs when a timeout is hit. This change removes all calls to the old SA-tools and instead uses Diagnostic Commands to gather the same information. The possible drawback is that Diagnostic Commands do not work on processes where the JVM is completely hung, but in that case we can fallback to running the SA-tools on the core file that is also generated. > > webrev: http://cr.openjdk.java.net/~sla/8168409/webrev.00 > bug: https://bugs.openjdk.java.net/browse/JDK-8168409 > > Thanks, > /Staffan > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From dean.long at oracle.com Thu Oct 20 22:47:39 2016 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 20 Oct 2016 15:47:39 -0700 Subject: RFR(XS) 8160411: SIGSEGV in frame::safe_for_sender on incomplete DeoptimizationBlob frame Message-ID: https://bugs.openjdk.java.net/browse/JDK-8160411 http://cr.openjdk.java.net/~dlong/8160411/webrev/ I have only observed this crash on solaris x64 so far, but I have included aarch64 in the fix since it uses similar code to x64. This crash happens when we call pd_get_top_frame_for_profiling() and the top frame is executing in the deopt stub, but hasn't finished pushing all the registers yet. If we add cb->frame_size() at this point, we can end up with a sender_sp that is past the stack base. The fix is to add a range check for sender_sp before using it. Tested by hand on linux x64 by forcing a SEGV in the deopt stub, then calling pd_get_top_frame_for_profiling() and safe_for_sender() in the signal handler. dl From david.holmes at oracle.com Fri Oct 21 04:10:02 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Oct 2016 14:10:02 +1000 Subject: RFR: 8168414 Various timeouthandler fixes In-Reply-To: References: <878580a7-7248-2e75-69ab-31ff735c0de0@oracle.com> Message-ID: On 20/10/2016 11:44 PM, Staffan Larsen wrote: > >> On 20 Oct 2016, at 15:41, David Holmes wrote: >> >> On 20/10/2016 11:27 PM, Staffan Larsen wrote: >>> When looking for some timeout handler problems I found a few things that should be fixed: >>> >>> * Strange use of Thread.currentThread().interrupt() >> >> That isn't "strange" it is an idiomatic usage - if you can't propagate the InterruptedException to show your caller you have been interrupted then you re-assert the interrupt state. > > OK, then. In this case it is less ?strange? than ?wrong?. The cases where this is done should not propagate the interrupt state - it only serves to throw unexpected exceptions higher up in the call-chain. If code higher up the call chain can not handle being interrupted and interrupt is being used within the program to signal cancellation, then it is the higher up code that has a problem. As I said the existing code is an idiomatic approach to dealing with cancellation requests. Cheers, David >> >> Cheers, >> David >> >>> * Add logging for timeouts >>> * Milliseconds sometimes printed as microseconds or nanoseconds >>> * Should use destroyForcibly() to terminate processes >>> * No need to sleep in the last iteration when running a command multiple times >>> * Disable timeout handling timeouts since we do that ourselves >>> >>> I didn?t want to file individual bugs for all of these, so I have lumped them together in one bug. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8168414 webrev: http://cr.openjdk.java.net/~sla/8168414/webrev.00 >>> >>> Thanks, >>> /Staffan >>> > From staffan.larsen at oracle.com Fri Oct 21 07:49:56 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 21 Oct 2016 09:49:56 +0200 Subject: RFR(s): 8168409 Update list of tools run by the jtreg timeouthandler In-Reply-To: <4673b551-eb61-2f7f-a0bb-39964e5e7016@oracle.com> References: <1BBA33BA-DD17-4B46-9FD1-3EC4AF1EBD33@oracle.com> <4673b551-eb61-2f7f-a0bb-39964e5e7016@oracle.com> Message-ID: Dmitry, I would like to avoid running SA-based tools in the timeout handler since on macOS they pop up dialogs asking for permission to attach to the process. This is problematic in our testing infrastructure. Instead, I hope we can run jhsdb on the core file that is generated. I tried doing this automatically in the timeout handler, but could not find a simple way to do it. jhsdb requires the path to the executable and the timeout handler does not know this. Perhaps a future enhancement to the timeout handler could fix this. /Staffan > On 20 Oct 2016, at 22:12, Dmitry Samersoff wrote: > > Staffan, > > Could you add > jhsdb jstack --mixed --pid > to get mixed stack trace when it possible? > > -Dmitry > > On 2016-10-20 14:42, Staffan Larsen wrote: >> Please review this small update to the list of tools that the jtreg timeouthandler runs when a timeout is hit. This change removes all calls to the old SA-tools and instead uses Diagnostic Commands to gather the same information. The possible drawback is that Diagnostic Commands do not work on processes where the JVM is completely hung, but in that case we can fallback to running the SA-tools on the core file that is also generated. >> >> webrev: http://cr.openjdk.java.net/~sla/8168409/webrev.00 >> bug: https://bugs.openjdk.java.net/browse/JDK-8168409 >> >> Thanks, >> /Staffan >> > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From staffan.larsen at oracle.com Fri Oct 21 08:00:08 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 21 Oct 2016 10:00:08 +0200 Subject: RFR: 8168414 Various timeouthandler fixes In-Reply-To: References: <878580a7-7248-2e75-69ab-31ff735c0de0@oracle.com> Message-ID: <8F0C1E8B-8538-4E10-981C-3901B5AE248B@oracle.com> > On 21 Oct 2016, at 06:10, David Holmes wrote: > > On 20/10/2016 11:44 PM, Staffan Larsen wrote: >> >>> On 20 Oct 2016, at 15:41, David Holmes wrote: >>> >>> On 20/10/2016 11:27 PM, Staffan Larsen wrote: >>>> When looking for some timeout handler problems I found a few things that should be fixed: >>>> >>>> * Strange use of Thread.currentThread().interrupt() >>> >>> That isn't "strange" it is an idiomatic usage - if you can't propagate the InterruptedException to show your caller you have been interrupted then you re-assert the interrupt state. >> >> OK, then. In this case it is less ?strange? than ?wrong?. The cases where this is done should not propagate the interrupt state - it only serves to throw unexpected exceptions higher up in the call-chain. > > If code higher up the call chain can not handle being interrupted and interrupt is being used within the program to signal cancellation, then it is the higher up code that has a problem. As I said the existing code is an idiomatic approach to dealing with cancellation requests. jtreg uses interrupt() to try to signal to the timeout handler that it has exceeded its timeout (-timeoutHandlerTimeout). This is the default way for jtreg to cancel timeout handlers. There are a couple of issues with this. First, interrupt() does not work well as a way to cancel I/O work (which is what the timeout handler is doing most of the time). Second, this particular timeout handler does a very good job of handling timeouts on its own. I will soon send out a patch to remove jtreg?s timeout when running with our timeout handler, but even with that patch there are cases when jtreg calls interrupt() on the timeout handler. In those cases we just want to ignore that and keep on doing our work, taking the liberty of saying that we know better than jtreg. /Staffan > > Cheers, > David > >>> >>> Cheers, >>> David >>> >>>> * Add logging for timeouts >>>> * Milliseconds sometimes printed as microseconds or nanoseconds >>>> * Should use destroyForcibly() to terminate processes >>>> * No need to sleep in the last iteration when running a command multiple times >>>> * Disable timeout handling timeouts since we do that ourselves >>>> >>>> I didn?t want to file individual bugs for all of these, so I have lumped them together in one bug. >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8168414 webrev: http://cr.openjdk.java.net/~sla/8168414/webrev.00 >>>> >>>> Thanks, >>>> /Staffan From aph at redhat.com Fri Oct 21 08:20:22 2016 From: aph at redhat.com (Andrew Haley) Date: Fri, 21 Oct 2016 09:20:22 +0100 Subject: RFR(XS) 8160411: SIGSEGV in frame::safe_for_sender on incomplete DeoptimizationBlob frame In-Reply-To: References: Message-ID: On 20/10/16 23:47, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8160411 > > http://cr.openjdk.java.net/~dlong/8160411/webrev/ > > I have only observed this crash on solaris x64 so far, but I have > included aarch64 in the fix since it uses similar code to x64. This > crash happens when we call pd_get_top_frame_for_profiling() and the top > frame is executing in the deopt stub, but hasn't finished pushing all > the registers yet. If we add cb->frame_size() at this point, we can end > up with a sender_sp that is past the stack base. The fix is to add a > range check for sender_sp before using it. > > Tested by hand on linux x64 by forcing a SEGV in the deopt stub, then > calling pd_get_top_frame_for_profiling() and safe_for_sender() in the > signal handler. That looks good. Thanks, Andrew. From dean.long at oracle.com Fri Oct 21 08:28:02 2016 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 21 Oct 2016 01:28:02 -0700 Subject: RFR(XS) 8160411: SIGSEGV in frame::safe_for_sender on incomplete DeoptimizationBlob frame In-Reply-To: References: Message-ID: Thanks Andrew. dl On 10/21/16 1:20 AM, Andrew Haley wrote: > On 20/10/16 23:47, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8160411 >> >> http://cr.openjdk.java.net/~dlong/8160411/webrev/ >> >> I have only observed this crash on solaris x64 so far, but I have >> included aarch64 in the fix since it uses similar code to x64. This >> crash happens when we call pd_get_top_frame_for_profiling() and the top >> frame is executing in the deopt stub, but hasn't finished pushing all >> the registers yet. If we add cb->frame_size() at this point, we can end >> up with a sender_sp that is past the stack base. The fix is to add a >> range check for sender_sp before using it. >> >> Tested by hand on linux x64 by forcing a SEGV in the deopt stub, then >> calling pd_get_top_frame_for_profiling() and safe_for_sender() in the >> signal handler. > That looks good. > > Thanks, > > Andrew. > > From staffan.larsen at oracle.com Fri Oct 21 13:24:05 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 21 Oct 2016 15:24:05 +0200 Subject: RFR(S): 8168483 Remove jtreg timeout handler timeout Message-ID: When we run jtreg in the hotspot, jdk and nashorn folders we run it with the timeout handler found under test/failure_handler. This handler is sometimes very slow (since it creates core files of hung processes). The handler is also very good at handling its own timeouts. The default implementation in jtreg tries to interrupt() a timeout handler after 300 seconds. This is not enough time for test/failure_handler. Rather than increasing the timeout, we should disable it when running with test/failure_handler and let it handle any timeouts itself. webrev: http://cr.openjdk.java.net/~sla/8168483/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8168483 /Staffan From daniel.daugherty at oracle.com Fri Oct 21 15:26:44 2016 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 21 Oct 2016 09:26:44 -0600 Subject: RFR(XS) 8160411: SIGSEGV in frame::safe_for_sender on incomplete DeoptimizationBlob frame In-Reply-To: References: Message-ID: <271322e8-47ae-0dc8-af78-d56ad81263e7@oracle.com> On 10/20/16 4:47 PM, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8160411 > > http://cr.openjdk.java.net/~dlong/8160411/webrev/ src/cpu/aarch64/vm/frame_aarch64.cpp No comments. src/cpu/x86/vm/frame_x86.cpp No comments. Thumbs up on both new sanity checks! Both you and Chris P are fixing bugs in the same functions. You guys need to coordinate your merges. Dan > > I have only observed this crash on solaris x64 so far, but I have > included aarch64 in the fix since it uses similar code to x64. This > crash happens when we call pd_get_top_frame_for_profiling() and the > top frame is executing in the deopt stub, but hasn't finished pushing > all the registers yet. If we add cb->frame_size() at this point, we > can end up with a sender_sp that is past the stack base. The fix is > to add a range check for sender_sp before using it. > > Tested by hand on linux x64 by forcing a SEGV in the deopt stub, then > calling pd_get_top_frame_for_profiling() and safe_for_sender() in the > signal handler. > > dl > > From volker.simonis at gmail.com Fri Oct 21 15:35:07 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 21 Oct 2016 17:35:07 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows Message-ID: Hi, can I please have a review and sponsor for the following trivial fix: http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ https://bugs.openjdk.java.net/browse/JDK-8168490 Currently we use the i64/ui64 suffixes to declare 64-bit integer literals on Windows (see src/share/vm/utilities/globalDefinitions_visCPP.hpp). Unfortunately this suffix is not known to Eclipse so most of a hotspot files have errors in an Eclipse project (e.g. NULL is defined as '0i64' which gives a syntax error in Eclipse, but NULL is used in most hotspot of the source files). Fortunately VS supports the more standard conforming LL/ULL suffixes (see http://en.cppreference.com/w/cpp/language/integer_literal) since at least VS2010 (see https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I therefore propose to change the suffix for integer literals from i64/ui64 to LL/ULL on Windows. Thank you and best regards, Volker From dean.long at oracle.com Fri Oct 21 18:50:00 2016 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 21 Oct 2016 11:50:00 -0700 Subject: RFR(XS) 8160411: SIGSEGV in frame::safe_for_sender on incomplete DeoptimizationBlob frame In-Reply-To: <271322e8-47ae-0dc8-af78-d56ad81263e7@oracle.com> References: <271322e8-47ae-0dc8-af78-d56ad81263e7@oracle.com> Message-ID: On 10/21/16 8:26 AM, Daniel D. Daugherty wrote: > On 10/20/16 4:47 PM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8160411 >> >> http://cr.openjdk.java.net/~dlong/8160411/webrev/ > > src/cpu/aarch64/vm/frame_aarch64.cpp > No comments. > > src/cpu/x86/vm/frame_x86.cpp > No comments. > > Thumbs up on both new sanity checks! > > Both you and Chris P are fixing bugs in the same functions. > You guys need to coordinate your merges. > OK, thanks Dan. dl > Dan > > >> >> I have only observed this crash on solaris x64 so far, but I have >> included aarch64 in the fix since it uses similar code to x64. This >> crash happens when we call pd_get_top_frame_for_profiling() and the >> top frame is executing in the deopt stub, but hasn't finished pushing >> all the registers yet. If we add cb->frame_size() at this point, we >> can end up with a sender_sp that is past the stack base. The fix is >> to add a range check for sender_sp before using it. >> >> Tested by hand on linux x64 by forcing a SEGV in the deopt stub, then >> calling pd_get_top_frame_for_profiling() and safe_for_sender() in the >> signal handler. >> >> dl >> >> > From chris.plummer at oracle.com Fri Oct 21 19:19:18 2016 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 21 Oct 2016 12:19:18 -0700 Subject: RFR(XS) 8160411: SIGSEGV in frame::safe_for_sender on incomplete DeoptimizationBlob frame In-Reply-To: <271322e8-47ae-0dc8-af78-d56ad81263e7@oracle.com> References: <271322e8-47ae-0dc8-af78-d56ad81263e7@oracle.com> Message-ID: On 10/21/16 8:26 AM, Daniel D. Daugherty wrote: > On 10/20/16 4:47 PM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8160411 >> >> http://cr.openjdk.java.net/~dlong/8160411/webrev/ > > src/cpu/aarch64/vm/frame_aarch64.cpp > No comments. > > src/cpu/x86/vm/frame_x86.cpp > No comments. > > Thumbs up on both new sanity checks! > > Both you and Chris P are fixing bugs in the same functions. > You guys need to coordinate your merges. Doesn't look like it will be a problem. Dean is pushing first. I'll merge. Chris > > Dan > > >> >> I have only observed this crash on solaris x64 so far, but I have >> included aarch64 in the fix since it uses similar code to x64. This >> crash happens when we call pd_get_top_frame_for_profiling() and the >> top frame is executing in the deopt stub, but hasn't finished pushing >> all the registers yet. If we add cb->frame_size() at this point, we >> can end up with a sender_sp that is past the stack base. The fix is >> to add a range check for sender_sp before using it. >> >> Tested by hand on linux x64 by forcing a SEGV in the deopt stub, then >> calling pd_get_top_frame_for_profiling() and safe_for_sender() in the >> signal handler. >> >> dl >> >> > From david.holmes at oracle.com Sat Oct 22 00:24:40 2016 From: david.holmes at oracle.com (David Holmes) Date: Sat, 22 Oct 2016 10:24:40 +1000 Subject: RFR(S): 8168483 Remove jtreg timeout handler timeout In-Reply-To: References: Message-ID: <41fd7610-f933-0282-4481-8155423114b4@oracle.com> Looks fine. Thanks, David On 21/10/2016 11:24 PM, Staffan Larsen wrote: > When we run jtreg in the hotspot, jdk and nashorn folders we run it with the timeout handler found under test/failure_handler. This handler is sometimes very slow (since it creates core files of hung processes). The handler is also very good at handling its own timeouts. > > The default implementation in jtreg tries to interrupt() a timeout handler after 300 seconds. This is not enough time for test/failure_handler. Rather than increasing the timeout, we should disable it when running with test/failure_handler and let it handle any timeouts itself. > > webrev: http://cr.openjdk.java.net/~sla/8168483/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8168483 > > /Staffan > From kim.barrett at oracle.com Sat Oct 22 00:46:08 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Oct 2016 20:46:08 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <299b4c9a-6090-cf9a-ad01-8a010146e647@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> <20161013101149.GL19291@ehelin.jrpg.bea.com> <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> <328E1EF5-9DD6-41A9-900A-56E66DF245F3@oracle.com> <20161018085351.GA19291@ehelin.jrpg.bea.com> <7A66BDA6-87B3-417A-8BF5-61E8EFFB8F29@oracle.com> <299b4c9a-6090-cf9a-ad01-8a010146e647@oracle.com> Message-ID: <36E3C19E-F2B7-4EA3-BBF5-2A09540EDE16@oracle.com> > On Oct 18, 2016, at 11:19 AM, Erik Helin wrote: > > On 10/18/2016 04:30 PM, Kim Barrett wrote: >>> On Oct 18, 2016, at 4:53 AM, Erik Helin wrote: >>> >>> On 2016-10-13, Kim Barrett wrote: >>>> So what I?d like is that you just pretend for now that the comment about the >>>> ordering in oops_on_card_xxx is actually correct. >>>> >>>> The change I?m working on for JDK-8166811 eliminates that ordering problem >>>> completely, but depends heavily on the changes here. >>> >>> I've discussed this with Thomas, what do you think about pushing the >>> fix for JDK-8166811 at the same time as this one? This patch looks good >>> AFAICS, but I'm worried that your code will be exposed slightly >>> different to existing bug than the current code. Given that >>> crashes/issues resulting from these kinds of bugs often are nightmarish >>> to track down, I'm a bit hesitant to push these patches one by one. >>> >>> What do you think? >> >> Sure, I can do that. > > Ok, thanks! Then consider this reviewed by me, but please hold of pushing it until I (and most likely Thomas) have reviewed the patch for JDK-8166811 as well. Good call! While working on JDK-8166811 I discovered a bug in the proposed fix for JDK-8166607. In the humongous case, if it bails because klass_or_null == NULL, we must re-enqueue the card, since *card_ptr has been cleared by the time of that check. The recorded write we?re dealing with was stale, since klass_or_null == NULL, but there could be an in-flight real recorded write to the same card from the allocating thread. Our setting the card to clean could cancel processing of that in-flight record. From kim.barrett at oracle.com Sat Oct 22 01:54:06 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Oct 2016 21:54:06 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <36E3C19E-F2B7-4EA3-BBF5-2A09540EDE16@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> <20161013101149.GL19291@ehelin.jrpg.bea.com> <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> <328E1EF5-9DD6-41A9-900A-56E66DF245F3@oracle.com> <20161018085351.GA19291@ehelin.jrpg.bea.com> <7A66BDA6-87B3-417A-8BF5-61E8EFFB8F29@oracle.com> <299b4c9a-6090-cf9a-ad01-8a010146e647@oracle.com> <36E3C19E-F2B7-4EA3-BBF5-2A09540EDE16@oracle.com> Message-ID: <788C53B9-1CB1-45C9-BFA9-58D4DAE94C3E@oracle.com> > On Oct 21, 2016, at 8:46 PM, Kim Barrett wrote: > In the humongous case, if it bails because klass_or_null == NULL, we must re-enqueue > the card, since *card_ptr has been cleared by the time of that check. The recorded write we?re > dealing with was stale, since klass_or_null == NULL, but there could be an in-flight real recorded > write to the same card from the allocating thread. Our setting the card to clean could cancel > processing of that in-flight record. That isn?t the right failure description; it?s that the allocating thread might write to the card and decide not to record because card is marked dirty, then we process stale card and mark it clean but don?t process it. I got the description mixed up somewhere between whiteboard and email. From tim.bell at oracle.com Sat Oct 22 03:21:04 2016 From: tim.bell at oracle.com (Tim Bell) Date: Fri, 21 Oct 2016 20:21:04 -0700 Subject: RFR(S): 8168483 Remove jtreg timeout handler timeout In-Reply-To: <41fd7610-f933-0282-4481-8155423114b4@oracle.com> References: <41fd7610-f933-0282-4481-8155423114b4@oracle.com> Message-ID: <50ec4d36-ba94-2794-5116-4273f3db4748@oracle.com> Looks good to me as well. Tim On 10/21/16 17:24, David Holmes wrote: > Looks fine. > > Thanks, > David > > On 21/10/2016 11:24 PM, Staffan Larsen wrote: >> When we run jtreg in the hotspot, jdk and nashorn folders we run it >> with the timeout handler found under test/failure_handler. This >> handler is sometimes very slow (since it creates core files of hung >> processes). The handler is also very good at handling its own timeouts. >> >> The default implementation in jtreg tries to interrupt() a timeout >> handler after 300 seconds. This is not enough time for >> test/failure_handler. Rather than increasing the timeout, we should >> disable it when running with test/failure_handler and let it handle >> any timeouts itself. >> >> webrev: http://cr.openjdk.java.net/~sla/8168483/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8168483 >> >> >> /Staffan >> From dmitry.samersoff at oracle.com Sat Oct 22 07:28:50 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Sat, 22 Oct 2016 10:28:50 +0300 Subject: RFR(s): 8168409 Update list of tools run by the jtreg timeouthandler In-Reply-To: References: <1BBA33BA-DD17-4B46-9FD1-3EC4AF1EBD33@oracle.com> <4673b551-eb61-2f7f-a0bb-39964e5e7016@oracle.com> Message-ID: Staffan, Thank you for the explanation. The fix looks good for me. -Dmitry On 2016-10-21 10:49, Staffan Larsen wrote: > Dmitry, > > I would like to avoid running SA-based tools in the timeout handler since on macOS they pop up dialogs asking for permission to attach to the process. This is problematic in our testing infrastructure. Instead, I hope we can run jhsdb on the core file that is generated. > > I tried doing this automatically in the timeout handler, but could not find a simple way to do it. jhsdb requires the path to the executable and the timeout handler does not know this. Perhaps a future enhancement to the timeout handler could fix this. > > /Staffan > >> On 20 Oct 2016, at 22:12, Dmitry Samersoff wrote: >> >> Staffan, >> >> Could you add >> jhsdb jstack --mixed --pid >> to get mixed stack trace when it possible? >> >> -Dmitry >> >> On 2016-10-20 14:42, Staffan Larsen wrote: >>> Please review this small update to the list of tools that the jtreg timeouthandler runs when a timeout is hit. This change removes all calls to the old SA-tools and instead uses Diagnostic Commands to gather the same information. The possible drawback is that Diagnostic Commands do not work on processes where the JVM is completely hung, but in that case we can fallback to running the SA-tools on the core file that is also generated. >>> >>> webrev: http://cr.openjdk.java.net/~sla/8168409/webrev.00 >>> bug: https://bugs.openjdk.java.net/browse/JDK-8168409 >>> >>> Thanks, >>> /Staffan >>> >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From david.holmes at oracle.com Mon Oct 24 00:43:19 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 24 Oct 2016 10:43:19 +1000 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: Message-ID: Hi Volker, On 22/10/2016 1:35 AM, Volker Simonis wrote: > Hi, > > can I please have a review and sponsor for the following trivial fix: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ > https://bugs.openjdk.java.net/browse/JDK-8168490 > > Currently we use the i64/ui64 suffixes to declare 64-bit integer > literals on Windows (see > src/share/vm/utilities/globalDefinitions_visCPP.hpp). > > Unfortunately this suffix is not known to Eclipse so most of a hotspot > files have errors in an Eclipse project (e.g. NULL is defined as > '0i64' which gives a syntax error in Eclipse, but NULL is used in most > hotspot of the source files). > > Fortunately VS supports the more standard conforming LL/ULL suffixes > (see http://en.cppreference.com/w/cpp/language/integer_literal) since > at least VS2010 (see > https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I > therefore propose to change the suffix for integer literals from > i64/ui64 to LL/ULL on Windows. In the sense that this changes the VIS specific code to use a more stand conforming form this seems fine. But as I wrote in the bug report, should we even be using this file with non Visual Studio compilers? Thanks, David > Thank you and best regards, > Volker > From staffan.larsen at oracle.com Mon Oct 24 07:06:05 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 24 Oct 2016 09:06:05 +0200 Subject: RFR(S): 8168483 Remove jtreg timeout handler timeout In-Reply-To: <41fd7610-f933-0282-4481-8155423114b4@oracle.com> References: <41fd7610-f933-0282-4481-8155423114b4@oracle.com> Message-ID: <97C86FBD-FEB0-48A4-B655-545BE5EE82FF@oracle.com> Thank David! > On 22 Oct 2016, at 02:24, David Holmes wrote: > > Looks fine. > > Thanks, > David > > On 21/10/2016 11:24 PM, Staffan Larsen wrote: >> When we run jtreg in the hotspot, jdk and nashorn folders we run it with the timeout handler found under test/failure_handler. This handler is sometimes very slow (since it creates core files of hung processes). The handler is also very good at handling its own timeouts. >> >> The default implementation in jtreg tries to interrupt() a timeout handler after 300 seconds. This is not enough time for test/failure_handler. Rather than increasing the timeout, we should disable it when running with test/failure_handler and let it handle any timeouts itself. >> >> webrev: http://cr.openjdk.java.net/~sla/8168483/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8168483 >> >> /Staffan >> From staffan.larsen at oracle.com Mon Oct 24 07:05:59 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 24 Oct 2016 09:05:59 +0200 Subject: RFR(S): 8168483 Remove jtreg timeout handler timeout In-Reply-To: <50ec4d36-ba94-2794-5116-4273f3db4748@oracle.com> References: <41fd7610-f933-0282-4481-8155423114b4@oracle.com> <50ec4d36-ba94-2794-5116-4273f3db4748@oracle.com> Message-ID: Thanks Tim! > On 22 Oct 2016, at 05:21, Tim Bell wrote: > > Looks good to me as well. > > Tim > > On 10/21/16 17:24, David Holmes wrote: >> Looks fine. >> >> Thanks, >> David >> >> On 21/10/2016 11:24 PM, Staffan Larsen wrote: >>> When we run jtreg in the hotspot, jdk and nashorn folders we run it >>> with the timeout handler found under test/failure_handler. This >>> handler is sometimes very slow (since it creates core files of hung >>> processes). The handler is also very good at handling its own timeouts. >>> >>> The default implementation in jtreg tries to interrupt() a timeout >>> handler after 300 seconds. This is not enough time for >>> test/failure_handler. Rather than increasing the timeout, we should >>> disable it when running with test/failure_handler and let it handle >>> any timeouts itself. >>> >>> webrev: http://cr.openjdk.java.net/~sla/8168483/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8168483 >>> >>> >>> /Staffan >>> > From volker.simonis at gmail.com Mon Oct 24 07:12:40 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 24 Oct 2016 09:12:40 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: Message-ID: On Mon, Oct 24, 2016 at 2:43 AM, David Holmes wrote: > Hi Volker, > > On 22/10/2016 1:35 AM, Volker Simonis wrote: >> >> Hi, >> >> can I please have a review and sponsor for the following trivial fix: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >> https://bugs.openjdk.java.net/browse/JDK-8168490 >> >> Currently we use the i64/ui64 suffixes to declare 64-bit integer >> literals on Windows (see >> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >> >> Unfortunately this suffix is not known to Eclipse so most of a hotspot >> files have errors in an Eclipse project (e.g. NULL is defined as >> '0i64' which gives a syntax error in Eclipse, but NULL is used in most >> hotspot of the source files). >> >> Fortunately VS supports the more standard conforming LL/ULL suffixes >> (see http://en.cppreference.com/w/cpp/language/integer_literal) since >> at least VS2010 (see >> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >> therefore propose to change the suffix for integer literals from >> i64/ui64 to LL/ULL on Windows. > > > In the sense that this changes the VIS specific code to use a more stand > conforming form this seems fine. > > But as I wrote in the bug report, should we even be using this file with non > Visual Studio compilers? > What I meant with "Eclipse doesn't understand" the suffix is that the Eclipse C++ parser doesn't understand it. Eclipse just parses all the C++ files which belong to a project in order to build up its code database for code navigation. As such it reads globalDefinitions_visCPP.hpp on Windows (if we configure a corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... This does not mean that we are using globalDefinitions_visCPP.hpp for building with another compiler. And we have no IDE-specific headers (this actually makes no sense since the IDEs should use the same settings which are used for the actual builds). Not sure about other IDEs, but Eclipse's C++ parser doesn't understand the i64/ui64 suffix and as there is a better, more standard-conforming way of declaring 64-bit integer literals on Windows I've just proposed the cleanup. Does this answer your question? Thank you and best egards, Volker > Thanks, > David > > >> Thank you and best regards, >> Volker >> > From david.holmes at oracle.com Mon Oct 24 07:14:26 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 24 Oct 2016 17:14:26 +1000 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: Message-ID: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> On 24/10/2016 5:12 PM, Volker Simonis wrote: > On Mon, Oct 24, 2016 at 2:43 AM, David Holmes wrote: >> Hi Volker, >> >> On 22/10/2016 1:35 AM, Volker Simonis wrote: >>> >>> Hi, >>> >>> can I please have a review and sponsor for the following trivial fix: >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >>> https://bugs.openjdk.java.net/browse/JDK-8168490 >>> >>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >>> literals on Windows (see >>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >>> >>> Unfortunately this suffix is not known to Eclipse so most of a hotspot >>> files have errors in an Eclipse project (e.g. NULL is defined as >>> '0i64' which gives a syntax error in Eclipse, but NULL is used in most >>> hotspot of the source files). >>> >>> Fortunately VS supports the more standard conforming LL/ULL suffixes >>> (see http://en.cppreference.com/w/cpp/language/integer_literal) since >>> at least VS2010 (see >>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >>> therefore propose to change the suffix for integer literals from >>> i64/ui64 to LL/ULL on Windows. >> >> >> In the sense that this changes the VIS specific code to use a more stand >> conforming form this seems fine. >> >> But as I wrote in the bug report, should we even be using this file with non >> Visual Studio compilers? >> > > What I meant with "Eclipse doesn't understand" the suffix is that the > Eclipse C++ parser doesn't understand it. Eclipse just parses all the > C++ files which belong to a project in order to build up its code > database for code navigation. As such it reads > globalDefinitions_visCPP.hpp on Windows (if we configure a > corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... > > This does not mean that we are using globalDefinitions_visCPP.hpp for > building with another compiler. And we have no IDE-specific headers > (this actually makes no sense since the IDEs should use the same > settings which are used for the actual builds). > > Not sure about other IDEs, but Eclipse's C++ parser doesn't understand > the i64/ui64 suffix and as there is a better, more standard-conforming > way of declaring 64-bit integer literals on Windows I've just proposed > the cleanup. > > Does this answer your question? Yes - thanks for clarifying. David > Thank you and best egards, > Volker > > >> Thanks, >> David >> >> >>> Thank you and best regards, >>> Volker >>> >> From volker.simonis at gmail.com Mon Oct 24 08:53:26 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 24 Oct 2016 10:53:26 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> Message-ID: On Mon, Oct 24, 2016 at 9:14 AM, David Holmes wrote: > On 24/10/2016 5:12 PM, Volker Simonis wrote: >> >> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes >> wrote: >>> >>> Hi Volker, >>> >>> On 22/10/2016 1:35 AM, Volker Simonis wrote: >>>> >>>> >>>> Hi, >>>> >>>> can I please have a review and sponsor for the following trivial fix: >>>> >>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >>>> https://bugs.openjdk.java.net/browse/JDK-8168490 >>>> >>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >>>> literals on Windows (see >>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >>>> >>>> Unfortunately this suffix is not known to Eclipse so most of a hotspot >>>> files have errors in an Eclipse project (e.g. NULL is defined as >>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in most >>>> hotspot of the source files). >>>> >>>> Fortunately VS supports the more standard conforming LL/ULL suffixes >>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) since >>>> at least VS2010 (see >>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >>>> therefore propose to change the suffix for integer literals from >>>> i64/ui64 to LL/ULL on Windows. >>> >>> >>> >>> In the sense that this changes the VIS specific code to use a more stand >>> conforming form this seems fine. >>> >>> But as I wrote in the bug report, should we even be using this file with >>> non >>> Visual Studio compilers? >>> >> >> What I meant with "Eclipse doesn't understand" the suffix is that the >> Eclipse C++ parser doesn't understand it. Eclipse just parses all the >> C++ files which belong to a project in order to build up its code >> database for code navigation. As such it reads >> globalDefinitions_visCPP.hpp on Windows (if we configure a >> corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... >> >> This does not mean that we are using globalDefinitions_visCPP.hpp for >> building with another compiler. And we have no IDE-specific headers >> (this actually makes no sense since the IDEs should use the same >> settings which are used for the actual builds). >> >> Not sure about other IDEs, but Eclipse's C++ parser doesn't understand >> the i64/ui64 suffix and as there is a better, more standard-conforming >> way of declaring 64-bit integer literals on Windows I've just proposed >> the cleanup. >> >> Does this answer your question? > > > Yes - thanks for clarifying. > Your welcome! Could you then please sponsor this change :) > > David > >> Thank you and best egards, >> Volker >> >> >>> Thanks, >>> David >>> >>> >>>> Thank you and best regards, >>>> Volker >>>> >>> > From mikael.gerdin at oracle.com Mon Oct 24 09:21:33 2016 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 24 Oct 2016 11:21:33 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> Message-ID: <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> Hi Volker, On 2016-10-24 10:53, Volker Simonis wrote: > On Mon, Oct 24, 2016 at 9:14 AM, David Holmes wrote: >> On 24/10/2016 5:12 PM, Volker Simonis wrote: >>> >>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes >>> wrote: >>>> >>>> Hi Volker, >>>> >>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: >>>>> >>>>> >>>>> Hi, >>>>> >>>>> can I please have a review and sponsor for the following trivial fix: >>>>> >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 >>>>> >>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >>>>> literals on Windows (see >>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >>>>> >>>>> Unfortunately this suffix is not known to Eclipse so most of a hotspot >>>>> files have errors in an Eclipse project (e.g. NULL is defined as >>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in most >>>>> hotspot of the source files). >>>>> >>>>> Fortunately VS supports the more standard conforming LL/ULL suffixes >>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) since >>>>> at least VS2010 (see >>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >>>>> therefore propose to change the suffix for integer literals from >>>>> i64/ui64 to LL/ULL on Windows. >>>> >>>> >>>> >>>> In the sense that this changes the VIS specific code to use a more stand >>>> conforming form this seems fine. >>>> >>>> But as I wrote in the bug report, should we even be using this file with >>>> non >>>> Visual Studio compilers? >>>> >>> >>> What I meant with "Eclipse doesn't understand" the suffix is that the >>> Eclipse C++ parser doesn't understand it. Eclipse just parses all the >>> C++ files which belong to a project in order to build up its code >>> database for code navigation. As such it reads >>> globalDefinitions_visCPP.hpp on Windows (if we configure a >>> corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... >>> >>> This does not mean that we are using globalDefinitions_visCPP.hpp for >>> building with another compiler. And we have no IDE-specific headers >>> (this actually makes no sense since the IDEs should use the same >>> settings which are used for the actual builds). >>> >>> Not sure about other IDEs, but Eclipse's C++ parser doesn't understand >>> the i64/ui64 suffix and as there is a better, more standard-conforming >>> way of declaring 64-bit integer literals on Windows I've just proposed >>> the cleanup. >>> >>> Does this answer your question? >> >> >> Yes - thanks for clarifying. >> > > Your welcome! Could you then please sponsor this change :) Does this change make the definition of CONST64 / UCONST64 the same on all globalDefinitions_x variants? If so can we move the definition to the common file instead of having it in the per-compiler files? Thanks /Mikael > >> >> David >> >>> Thank you and best egards, >>> Volker >>> >>> >>>> Thanks, >>>> David >>>> >>>> >>>>> Thank you and best regards, >>>>> Volker >>>>> >>>> >> From yasuenag at gmail.com Mon Oct 24 10:24:44 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 24 Oct 2016 19:24:44 +0900 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: Hi Dmitry, David, I want to resume to work for this issue because this issue has been marked P3. Dmitry, do you have any idea for it? IMHO, we can fix as below: http://cr.openjdk.java.net/~ysuenaga/JDK-8165869/webrev.00/ According to David, JvmtiExport::load_agent_library() should return zero when the operation which is kicked by AttachListener is succeeded. AttachListener will write response code from JvmtiExport::load_agent_library() at first, and the client (e.g. HotSpotVirtualMachine.java) checks it. Thus I keep current implementation about return code, and I added the code to send error message to the client. It works fine with jdk/test/com/sun/tools/attach/BasicTests.java . What do you think about it? Thanks, Yasumasa On 2016/09/15 20:08, Dmitry Samersoff wrote: > Yasumasa, David, > > I'm looking into this issue and come back as soon as I have a clear > picture. > > It takes some time. Sorry! > > -Dmitry > > On 2016-09-15 14:01, Yasumasa Suenaga wrote: >> Hi David, >> >> I agree the call sequence which you write in the case of "jcmd" operation. >> However, we should think about "load" operation in AttachListener. >> >> We can access JvmtiExport::load_agent_library() through two routes: >> >> Route "load": AttachListener -> JvmtiExport::load_agent_library() >> Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> >> DCmd::parse_and_execute() >> >> "load" sets error code (it means first data in the stream) when >> JvmtiExport::load_agent_library() returns JNI_ERR. >> OTOH "jcmd" sets error code if exception is occurred ONLY. >> >> If we try to load invalid JVMTI agent, "load" writes JNI_ERR to stream, >> however "jcmd" writes JNI_OK because pending exception is not available >> at this point. >> Currently, JCmd.java shows "Command executed successfully" when it >> cannot read the message from the socket. In case of this issue, >> JVMTIAgentLoadDCmd does not write any message because it cannot attach >> the agent. >> If return code which is in the top of stream should mean whether >> communication with AttachListener is succeeded, I think HotSpot should >> write error message or error code to the stream for JCmd.java . >> >> I'm not sure this email is the answer for you. >> Sorry for my English. >> >> >> Yasumasa >> >> >> On 2016/09/15 18:43, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>>> Hi, >>>> >>>>> If this is done through jcmd then jcmd needs to know how to "unwrap" >>>>> the information it gets back from the Dcmd. >>>> >>>> In case of JVMTI.agent_load dcmd, I think we should be able to get the >>>> response as below: >>>> >>>> 1. Invalid JVMTI agent >>>> 2. Agent_OnAttach() did not return JNI_OK >>>> 3. Succeeded >>>> >>>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, >>>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>>> occurred (in Agent_OnAttach()). >>> >>> Can you respond to my comment in the bug report about the chain of >>> calls and explain exactly where you think things are going wrong in >>> this scenario. I'm having a lot of trouble trying to see how the >>> information flows back through the layers. >>> >>>> IMHO, HotSpot should return error code (in top of the stream: written by >>>> AttachListener at first). So we should be able to get some error code in >>>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>>> parse text information in the stream. >>> >>> It returns the high-level response code first "did communication with >>> the target VM succeed", then the actual action response code. That >>> seems the right thing to do to me. It is up to the callers reading the >>> stream to understand how to read it. To me the issue lies somewhere on >>> the jcmd/dcmd side. >>> >>> Thanks, >>> David >>> >>>> >>>> Sorry for my English. >>>> >>>> Yasumasa >>>> >>>> >>>> On 2016/09/14 7:03, David Holmes wrote: >>>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>>> Thanks David! >>>>>> If we should not change the meaning of return code from >>>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>>> below: >>>>>> ------------------- >>>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>>>>> +0000 >>>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>>>>> +0900 >>>>>> @@ -2412,6 +2412,10 @@ >>>>>> result = JNI_OK; >>>>>> } >>>>>> } >>>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>>> + if (result != JNI_OK) { >>>>>> + st->print_cr("%d", result); >>>>>> + } >>>>>> return result; >>>>>> } >>>>>> ------------------- >>>>> >>>>> Not sure I see the point. "return result" will put the error code into >>>>> the socket stream and that error will be seen and responded to. I >>>>> don't expect anything to then read further into the stream to see the >>>>> "result" repeated a second time. In other words if execute() doesn't >>>>> see a zero result then you go no further; if execute sees a zero >>>>> result then the actual action may have had an issue so you proceed to >>>>> read the "action's result". >>>>> >>>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get -1 if >>>>>> we try to attach invalid agent. >>>>> >>>>> Can you please outline your test case for this again. If this is done >>>>> through jcmd then jcmd needs to know how to "unwrap" the information >>>>> it gets back from the Dcmd. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>>> David, >>>>>>> >>>>>>> Thank you for the evaluation. >>>>>>> >>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>> is to >>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>> Though I'm >>>>>>>> not sure how exactly. >>>>>>> >>>>>>> This logic seems completely broken for me. But I don't see >>>>>>> anything we >>>>>>> can do right now - for jdk 9. >>>>>>> >>>>>>> It requires careful changes of both - code and tests. >>>>>>> >>>>>>> I can help with this task but not today. >>>>>>> >>>>>>> -Dmitry >>>>>>> >>>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>>> >>>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>>> difference between what happens with your fix and what happens >>>>>>>>> without. >>>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>>> how we >>>>>>>>> get >>>>>>>>> AgentInitializationException instead of AgentLoadException. I need >>>>>>>>> to be >>>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>>> very >>>>>>>>> difficult to arrange. :( >>>>>>>> >>>>>>>> I finally managed to connect all the pieces on this. >>>>>>>> >>>>>>>> The basic problem is that with the proposed change >>>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which >>>>>>>> then >>>>>>>> leads to the InternalError. Without the change we reach this >>>>>>>> block in >>>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>>> >>>>>>>> int result = readInt(in); >>>>>>>> if (result != 0) { >>>>>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>>>>> result); >>>>>>>> } >>>>>>>> >>>>>>>> and so get the expected AgentInitializationException. >>>>>>>> >>>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>>> original: >>>>>>>> >>>>>>>> st->print_cr("%d", result); >>>>>>>> result = JNI_OK; >>>>>>>> >>>>>>>> then execute() manages to read a zero completion status from the >>>>>>>> stream, >>>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>>> whether >>>>>>>> zero or not - from the stream. This is because the AttachListener >>>>>>>> code >>>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>>> wrote >>>>>>>> the result value above in print_cr. Then if we look at, for example, >>>>>>>> LinuxAttachOperation::complete, we will write "result" to the socket >>>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>>> operation >>>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>>> read 0 >>>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>>> proposed >>>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>>> throws >>>>>>>> AgentLoadException. >>>>>>>> >>>>>>>> So in summary there are two different error indicators written into >>>>>>>> the >>>>>>>> stream, and we need the first to be zero unless some major error >>>>>>>> with >>>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>>> "load" >>>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>>> >>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>> is to >>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>> Though I'm >>>>>>>> not sure how exactly. >>>>>>>> >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> David >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>>> someone >>>>>>>>>> help me? >>>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>>> I think I see the problem. The attach framework tries to load the >>>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>>> actually >>>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>>> through. >>>>>>>>>>> I'm guessing, at the moment, that the failure is due to a missing >>>>>>>>>>> module related option. >>>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>> >>>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>>> >>>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>>> >>>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>>> seems it >>>>>>>>>>>> may >>>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>>> ended up >>>>>>>>>>>> not >>>>>>>>>>>> touching!). >>>>>>>>>>>> >>>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and >>>>>>>>>>>> see >>>>>>>>>>>> what >>>>>>>>>>>> happens? We see AgentLoadException from the code that internally >>>>>>>>>>>> tries >>>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>>> >>>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception during >>>>>>>>>>>> test >>>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>>> missing in >>>>>>>>>>>> target VM >>>>>>>>>>>> ... >>>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>>> missing in >>>>>>>>>>>> target VM >>>>>>>>>>>> ... >>>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to >>>>>>>>>>>> load >>>>>>>>>>>> agent >>>>>>>>>>>> library >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>>>>> when >>>>>>>>>>>>> the >>>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>>> fixed >>>>>>>>>>>>> in 9 >>>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>>> >>>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>>> appropriate >>>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>>> problem - >>>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>>> unrelated >>>>>>>>>>>> issue in our testing environment. >>>>>>>>>>>> >>>>>>>>>>>> Unfortunately I have some high priority tasks to handle right >>>>>>>>>>>> now, >>>>>>>>>>>> so I >>>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>>> >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>>> For the record it looks like the tests involved are broken, in >>>>>>>>>>>>>> that >>>>>>>>>>>>>> because the VM used to lie about the success of attaching the >>>>>>>>>>>>>> agent, the >>>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>>> >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Harold >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>> >>>>>>> > > From volker.simonis at gmail.com Mon Oct 24 12:51:58 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 24 Oct 2016 14:51:58 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> Message-ID: Hi Mikael, you're right - the OpenJDK now has the same definitions for CONST64/UCONST64 on all platforms. If you don't have other settings for you're closed ports I can certainly generalize the definition into globalDefinitions.hpp (it also works for HP-UX which is our last closed platform :) Notice that this would also require to move the definition of min_jlong/max_jlong from the compiler-specific files to globalDefinitions.hpp. I can do a new webrev with these additional changes. Regards, Volker On Mon, Oct 24, 2016 at 11:21 AM, Mikael Gerdin wrote: > Hi Volker, > > > On 2016-10-24 10:53, Volker Simonis wrote: >> >> On Mon, Oct 24, 2016 at 9:14 AM, David Holmes >> wrote: >>> >>> On 24/10/2016 5:12 PM, Volker Simonis wrote: >>>> >>>> >>>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes >>>> wrote: >>>>> >>>>> >>>>> Hi Volker, >>>>> >>>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: >>>>>> >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> can I please have a review and sponsor for the following trivial fix: >>>>>> >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 >>>>>> >>>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >>>>>> literals on Windows (see >>>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >>>>>> >>>>>> Unfortunately this suffix is not known to Eclipse so most of a hotspot >>>>>> files have errors in an Eclipse project (e.g. NULL is defined as >>>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in most >>>>>> hotspot of the source files). >>>>>> >>>>>> Fortunately VS supports the more standard conforming LL/ULL suffixes >>>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) since >>>>>> at least VS2010 (see >>>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >>>>>> therefore propose to change the suffix for integer literals from >>>>>> i64/ui64 to LL/ULL on Windows. >>>>> >>>>> >>>>> >>>>> >>>>> In the sense that this changes the VIS specific code to use a more >>>>> stand >>>>> conforming form this seems fine. >>>>> >>>>> But as I wrote in the bug report, should we even be using this file >>>>> with >>>>> non >>>>> Visual Studio compilers? >>>>> >>>> >>>> What I meant with "Eclipse doesn't understand" the suffix is that the >>>> Eclipse C++ parser doesn't understand it. Eclipse just parses all the >>>> C++ files which belong to a project in order to build up its code >>>> database for code navigation. As such it reads >>>> globalDefinitions_visCPP.hpp on Windows (if we configure a >>>> corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... >>>> >>>> This does not mean that we are using globalDefinitions_visCPP.hpp for >>>> building with another compiler. And we have no IDE-specific headers >>>> (this actually makes no sense since the IDEs should use the same >>>> settings which are used for the actual builds). >>>> >>>> Not sure about other IDEs, but Eclipse's C++ parser doesn't understand >>>> the i64/ui64 suffix and as there is a better, more standard-conforming >>>> way of declaring 64-bit integer literals on Windows I've just proposed >>>> the cleanup. >>>> >>>> Does this answer your question? >>> >>> >>> >>> Yes - thanks for clarifying. >>> >> >> Your welcome! Could you then please sponsor this change :) > > > Does this change make the definition of CONST64 / UCONST64 the same on all > globalDefinitions_x variants? > If so can we move the definition to the common file instead of having it in > the per-compiler files? > > Thanks > /Mikael > > >> >>> >>> David >>> >>>> Thank you and best egards, >>>> Volker >>>> >>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>>>> Thank you and best regards, >>>>>> Volker >>>>>> >>>>> >>> > From erik.helin at oracle.com Mon Oct 24 14:01:05 2016 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 24 Oct 2016 16:01:05 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: References: <20161013121510.GM19291@ehelin.jrpg.bea.com> Message-ID: <20161024140105.GE19291@ehelin.jrpg.bea.com> On 2016-10-13, Thomas St?fe wrote: > Hi Erik, > > On Thu, Oct 13, 2016 at 2:15 PM, Erik Helin wrote: > > > Hi Thomas, > > > > thanks for submitting the JEP and proposing this feature! > > > > On 2016-10-10, Thomas St?fe wrote: > > > Hi all, > > > > > > May I have please some feedback for this enhancement proposal? > > > > > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > > > > > > In one very short sentence it proposes a better allocation scheme for > > > Metaspace Chunks in order to reduce fragmentation and metaspace waste. > > > > > > I also added a short presentation which describes the problem and how we > > > solved it in our VM port. > > > > > > https://bugs.openjdk.java.net/secure/attachment/63894/ > > Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf > > > > Do we really need the flag -XX:+CoalesceMetaspace? Having two differnent > > ways to handle the chunk free lists in Metaspace is unfortunate, it > > might introduce hard to detect bugs and will also require much more > > testing (runnings lots of tests with the flag both on and off). > > > > You are right. If the new allocator works well, there is no reason to keep > the old allocator around. > > We wanted for a temporary time to be able to switch between both old and > new allocator. Just to have a fallback if problems occur. But if it works, > it makes sense to only have one allocator, and the "CoalesceMetaspace" flag > can be removed, and also the code can be made a lot simpler because we do > not need both code paths. Yeah, I would strongly prefer to not introduce a new flag for this. Have you thought about testing? Do you intend to write new tests to stress the coalescing? > > > > Do you think your proposed solution has low enough overhead (in terms > > of CPU and memory) to be on "by default"? > > > > We decided to switch it on by default in our VM. > > Memory overhead can be almost exactly calculated. Bitmasks take 2 bits per > specialized-chunk-sized-area. That means, for specialized-chunk-size = 1k > (128 meta words): metaspace size / 8192. So, for 1G of metaspace we pay > 132KB overhead for the bitmasks, or roughly 0.1%. > > There is some CPU overhead, but in my tests I could not measure anything > above noise level. Those numbers seems low enough to me in order to not warrant a new flag. > > Thanks, > > Erik > > > > > Btw, I understand that it is difficult to estimate this proposal without a > prototype to play around. As I already mentioned, the patch right now only > exists in our code base and not yet in the OpenJDK. If you guys are > seriously interested in this JEP, I will invest the time to port the patch > to the OpenJDK, so that you can check it out for yourself. Yes, we are seriously interested :) I think the proposal sounds good. I guess the devil will be in the details, so I (we) would really appreciate if you want to port your internal patch to OpenJDK. Thanks, Erik > Kind Regards, Thomas > > > > > > > Thank you very much! > > > > > > Kind Regards, Thomas > > > > > > > > > On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe > > > wrote: > > > > > > > Dear all, > > > > > > > > please take a look at this Enhancement Proposal for the metaspace > > > > allocator. I hope these are the right groups for this discussion. > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > > > > > Background: > > > > > > > > We at SAP see at times at customer installations OOMs in Metaspace > > > > (usually, with compressed class pointers enabled, in Compressed Class > > > > Space). The VM attempts to allocate metaspace and fails, hitting the > > > > CompressedClassSpaceSize limit. Note that we usually set the limit > > lower > > > > than the default, typically at 256M. > > > > > > > > When analyzing, we observed that a large part of the metaspace is > > indeed > > > > free but "locked in" into metaspace chunks of the wrong size: often we > > > > would find a lot of free small chunks, but the allocation request was > > for > > > > medium chunks, and failed. > > > > > > > > The reason was that if at some point in time a lot of class loaders > > were > > > > alive, each with only a few small classes loaded. This would lead to > > the > > > > metaspace being swamped with lots of small chunks. This is because each > > > > SpaceManager first allocates small chunks, only after a certain amount > > of > > > > allocation requests switches to larger chunks. > > > > > > > > These small chunks are free and wait in the freelist, but cannot be > > reused > > > > for allocation requests which require larger chunks, even if they are > > > > physically adjacent in the virtual space. > > > > > > > > We (at SAP) added a patch which allows on-the-fly metaspace chunk > > merging > > > > - to merge multiple adjacent smaller chunk to form a larger chunk. > > This, in > > > > combination with the reverse direction - splitting a large chunk to get > > > > smaller chunks - partly negates the "chunks-are-locked-in-into- > > their-size" > > > > limitation and provides for better reuse of metaspace chunks. It also > > > > provides better defragmentation as well. > > > > > > > > I discussed this fix off-list with Coleen Phillimore and Jon Masamitsu, > > > > and instead of just offering this as a fix, both recommended to open a > > JEP > > > > for this, because its scope would be beyond that of a simple fix. > > > > > > > > So here is my first JEP :) I hope it follows the right form. Please, if > > > > you have time, take a look and tell us what you think. > > > > > > > > Thank you, and Kind Regards, > > > > > > > > Thomas St?fe > > > > > > > > > > > > > > > > > > From volker.simonis at gmail.com Mon Oct 24 15:36:08 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 24 Oct 2016 17:36:08 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> Message-ID: So here comes the new webrev: http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v2/ It moves the definition of CONST64/UCONST64 and min_jlong/max_jlong from the compiler-specific files to globalDefinitions.hpp. I also had to remove some AIX-specific constant definitions and replace them by the more standard '*[K,M,G]' notation. I did build and smoke tested this on Linux/amd64, MaxOS X, AIX, Solaris and Windows. Thanks, Volker On Mon, Oct 24, 2016 at 2:51 PM, Volker Simonis wrote: > Hi Mikael, > > you're right - the OpenJDK now has the same definitions for > CONST64/UCONST64 on all platforms. If you don't have other settings > for you're closed ports I can certainly generalize the definition into > globalDefinitions.hpp (it also works for HP-UX which is our last > closed platform :) > > Notice that this would also require to move the definition of > min_jlong/max_jlong from the compiler-specific files to > globalDefinitions.hpp. > > I can do a new webrev with these additional changes. > > Regards, > Volker > > > On Mon, Oct 24, 2016 at 11:21 AM, Mikael Gerdin > wrote: >> Hi Volker, >> >> >> On 2016-10-24 10:53, Volker Simonis wrote: >>> >>> On Mon, Oct 24, 2016 at 9:14 AM, David Holmes >>> wrote: >>>> >>>> On 24/10/2016 5:12 PM, Volker Simonis wrote: >>>>> >>>>> >>>>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes >>>>> wrote: >>>>>> >>>>>> >>>>>> Hi Volker, >>>>>> >>>>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> can I please have a review and sponsor for the following trivial fix: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 >>>>>>> >>>>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >>>>>>> literals on Windows (see >>>>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >>>>>>> >>>>>>> Unfortunately this suffix is not known to Eclipse so most of a hotspot >>>>>>> files have errors in an Eclipse project (e.g. NULL is defined as >>>>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in most >>>>>>> hotspot of the source files). >>>>>>> >>>>>>> Fortunately VS supports the more standard conforming LL/ULL suffixes >>>>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) since >>>>>>> at least VS2010 (see >>>>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >>>>>>> therefore propose to change the suffix for integer literals from >>>>>>> i64/ui64 to LL/ULL on Windows. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> In the sense that this changes the VIS specific code to use a more >>>>>> stand >>>>>> conforming form this seems fine. >>>>>> >>>>>> But as I wrote in the bug report, should we even be using this file >>>>>> with >>>>>> non >>>>>> Visual Studio compilers? >>>>>> >>>>> >>>>> What I meant with "Eclipse doesn't understand" the suffix is that the >>>>> Eclipse C++ parser doesn't understand it. Eclipse just parses all the >>>>> C++ files which belong to a project in order to build up its code >>>>> database for code navigation. As such it reads >>>>> globalDefinitions_visCPP.hpp on Windows (if we configure a >>>>> corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... >>>>> >>>>> This does not mean that we are using globalDefinitions_visCPP.hpp for >>>>> building with another compiler. And we have no IDE-specific headers >>>>> (this actually makes no sense since the IDEs should use the same >>>>> settings which are used for the actual builds). >>>>> >>>>> Not sure about other IDEs, but Eclipse's C++ parser doesn't understand >>>>> the i64/ui64 suffix and as there is a better, more standard-conforming >>>>> way of declaring 64-bit integer literals on Windows I've just proposed >>>>> the cleanup. >>>>> >>>>> Does this answer your question? >>>> >>>> >>>> >>>> Yes - thanks for clarifying. >>>> >>> >>> Your welcome! Could you then please sponsor this change :) >> >> >> Does this change make the definition of CONST64 / UCONST64 the same on all >> globalDefinitions_x variants? >> If so can we move the definition to the common file instead of having it in >> the per-compiler files? >> >> Thanks >> /Mikael >> >> >>> >>>> >>>> David >>>> >>>>> Thank you and best egards, >>>>> Volker >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> >>>>>>> Thank you and best regards, >>>>>>> Volker >>>>>>> >>>>>> >>>> >> From coleen.phillimore at oracle.com Mon Oct 24 16:32:08 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 24 Oct 2016 12:32:08 -0400 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: <20161024140105.GE19291@ehelin.jrpg.bea.com> References: <20161013121510.GM19291@ehelin.jrpg.bea.com> <20161024140105.GE19291@ehelin.jrpg.bea.com> Message-ID: <6482d642-428a-0e53-c8da-d61de968defe@oracle.com> Hi Thomas, I agree with Erik. If this works well for you, then it should just be implemented without an option. If done early in JDK10, it'll get a lot of good testing. This looks like a very good improvement. We had discussed coalescing blocks and other improvements like this early on, but wanted to wait to see problems in the field to motivate the changes. We've seen these sorts of problems now too. One of the things we've considered is that we wanted to use the operating system's version of malloc for the chunks, so that it can split and coalesce chunks for us. The Solaris malloc is improved recently. But I don't think it's time to make change to use malloc'ed chunks yet because we have to consider all of the operating systems that we and the OpenJDK community supports. So, yes, I think the JEP looks good and your slides are absolutely beautiful. Everyone should see these slides. Thanks, Coleen On 10/24/16 10:01 AM, Erik Helin wrote: > On 2016-10-13, Thomas St?fe wrote: >> Hi Erik, >> >> On Thu, Oct 13, 2016 at 2:15 PM, Erik Helin wrote: >> >>> Hi Thomas, >>> >>> thanks for submitting the JEP and proposing this feature! >>> >>> On 2016-10-10, Thomas St?fe wrote: >>>> Hi all, >>>> >>>> May I have please some feedback for this enhancement proposal? >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8166690 >>>> >>>> >>>> In one very short sentence it proposes a better allocation scheme for >>>> Metaspace Chunks in order to reduce fragmentation and metaspace waste. >>>> >>>> I also added a short presentation which describes the problem and how we >>>> solved it in our VM port. >>>> >>>> https://bugs.openjdk.java.net/secure/attachment/63894/ >>> Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf >>> >>> Do we really need the flag -XX:+CoalesceMetaspace? Having two differnent >>> ways to handle the chunk free lists in Metaspace is unfortunate, it >>> might introduce hard to detect bugs and will also require much more >>> testing (runnings lots of tests with the flag both on and off). >>> >> You are right. If the new allocator works well, there is no reason to keep >> the old allocator around. >> >> We wanted for a temporary time to be able to switch between both old and >> new allocator. Just to have a fallback if problems occur. But if it works, >> it makes sense to only have one allocator, and the "CoalesceMetaspace" flag >> can be removed, and also the code can be made a lot simpler because we do >> not need both code paths. > Yeah, I would strongly prefer to not introduce a new flag for this. Have > you thought about testing? Do you intend to write new tests to stress > the coalescing? > >>> Do you think your proposed solution has low enough overhead (in terms >>> of CPU and memory) to be on "by default"? >>> >> We decided to switch it on by default in our VM. >> >> Memory overhead can be almost exactly calculated. Bitmasks take 2 bits per >> specialized-chunk-sized-area. That means, for specialized-chunk-size = 1k >> (128 meta words): metaspace size / 8192. So, for 1G of metaspace we pay >> 132KB overhead for the bitmasks, or roughly 0.1%. >> >> There is some CPU overhead, but in my tests I could not measure anything >> above noise level. > Those numbers seems low enough to me in order to not warrant a new flag. > >>> Thanks, >>> Erik >>> >>> >> Btw, I understand that it is difficult to estimate this proposal without a >> prototype to play around. As I already mentioned, the patch right now only >> exists in our code base and not yet in the OpenJDK. If you guys are >> seriously interested in this JEP, I will invest the time to port the patch >> to the OpenJDK, so that you can check it out for yourself. > Yes, we are seriously interested :) I think the proposal sounds good. I guess > the devil will be in the details, so I (we) would really appreciate if > you want to port your internal patch to OpenJDK. > > Thanks, > Erik > >> Kind Regards, Thomas >> >> >> >> >>>> Thank you very much! >>>> >>>> Kind Regards, Thomas >>>> >>>> >>>> On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe >>>> wrote: >>>> >>>>> Dear all, >>>>> >>>>> please take a look at this Enhancement Proposal for the metaspace >>>>> allocator. I hope these are the right groups for this discussion. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8166690 >>>>> >>>>> Background: >>>>> >>>>> We at SAP see at times at customer installations OOMs in Metaspace >>>>> (usually, with compressed class pointers enabled, in Compressed Class >>>>> Space). The VM attempts to allocate metaspace and fails, hitting the >>>>> CompressedClassSpaceSize limit. Note that we usually set the limit >>> lower >>>>> than the default, typically at 256M. >>>>> >>>>> When analyzing, we observed that a large part of the metaspace is >>> indeed >>>>> free but "locked in" into metaspace chunks of the wrong size: often we >>>>> would find a lot of free small chunks, but the allocation request was >>> for >>>>> medium chunks, and failed. >>>>> >>>>> The reason was that if at some point in time a lot of class loaders >>> were >>>>> alive, each with only a few small classes loaded. This would lead to >>> the >>>>> metaspace being swamped with lots of small chunks. This is because each >>>>> SpaceManager first allocates small chunks, only after a certain amount >>> of >>>>> allocation requests switches to larger chunks. >>>>> >>>>> These small chunks are free and wait in the freelist, but cannot be >>> reused >>>>> for allocation requests which require larger chunks, even if they are >>>>> physically adjacent in the virtual space. >>>>> >>>>> We (at SAP) added a patch which allows on-the-fly metaspace chunk >>> merging >>>>> - to merge multiple adjacent smaller chunk to form a larger chunk. >>> This, in >>>>> combination with the reverse direction - splitting a large chunk to get >>>>> smaller chunks - partly negates the "chunks-are-locked-in-into- >>> their-size" >>>>> limitation and provides for better reuse of metaspace chunks. It also >>>>> provides better defragmentation as well. >>>>> >>>>> I discussed this fix off-list with Coleen Phillimore and Jon Masamitsu, >>>>> and instead of just offering this as a fix, both recommended to open a >>> JEP >>>>> for this, because its scope would be beyond that of a simple fix. >>>>> >>>>> So here is my first JEP :) I hope it follows the right form. Please, if >>>>> you have time, take a look and tell us what you think. >>>>> >>>>> Thank you, and Kind Regards, >>>>> >>>>> Thomas St?fe >>>>> >>>>> >>>>> >>>>> From vladimir.kozlov at oracle.com Mon Oct 24 20:19:38 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 24 Oct 2016 13:19:38 -0700 Subject: [aarch64-port-dev ] JEP to add Oracle arm32/arm64 port to JDK 9 In-Reply-To: <1477261372.5559.15.camel@gmail.com> References: <1477261372.5559.15.camel@gmail.com> Message-ID: <580E6CDA.700@oracle.com> CCing to hotspot-dev and porters-dev Vladimir On 10/23/16 3:22 PM, Edward Nevill wrote: > Hi, > > I have submitted the following JEP to add the Oracle contributed arm32/arm64 port to JDK 9. > > https://bugs.openjdk.java.net/browse/JDK-8168503 > > I have filed this with the jdk9-fc-request label and sent a copy to jep-submit at openjdk.java.net. > > Please feel free to comment either on the list, or on the JBS issue above, > > All the best, > Ed. > From shafi.s.ahmad at oracle.com Tue Oct 25 03:39:03 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Mon, 24 Oct 2016 20:39:03 -0700 (PDT) Subject: [8u] RFR for JDK-8134389: Crash in HotSpot with jvm.dll+0x42b48 ciObjectFactory::create_new_metadata In-Reply-To: <2e1de7f0-cc65-47f7-9f97-cb0e56dacfe1@default> References: <2e1de7f0-cc65-47f7-9f97-cb0e56dacfe1@default> Message-ID: May I get the second review for this backport. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Thursday, October 20, 2016 9:55 AM > To: Vladimir Kozlov; hotspot-dev at openjdk.java.net > Subject: RE: [8u] RFR for JDK-8134389: Crash in HotSpot with jvm.dll+0x42b48 > ciObjectFactory::create_new_metadata > > Thank you Vladimir for the review. > > Please find the updated webrev link. > http://cr.openjdk.java.net/~shshahma/8134389/webrev.01/ > > All, > > May I get 2nd review for this. > > Regards, > Shafi > > > -----Original Message----- > > From: Vladimir Kozlov > > Sent: Wednesday, October 19, 2016 10:14 PM > > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > > Cc: Vladimir Ivanov; Jamsheed C M > > Subject: Re: [8u] RFR for JDK-8134389: Crash in HotSpot with > > jvm.dll+0x42b48 ciObjectFactory::create_new_metadata > > > > In ciMethod.hpp you duplicated comment line: > > > > + // Given a certain calling environment, find the monomorphic > > + target > > // Given a certain calling environment, find the monomorphic > > target > > > > Otherwise looks good. > > > > Thanks, > > Vladimir K > > > > On 10/19/16 12:53 AM, Shafi Ahmad wrote: > > > Hi All, > > > > > > Please review the backport of 'JDK-8134389: Crash in HotSpot with > > jvm.dll+0x42b48 ciObjectFactory::create_new_metadata' to jdk8u-dev. > > > > > > Please note that backport is not clean as I was getting build failure due to: > > > Formal parameter 'ignore_return' in method > > > GraphBuilder::method_return > > is added in the fix of https://bugs.openjdk.java.net/browse/JDK-8164122. > > > The current code change is done on top of aforesaid bug fix and this > > > formal > > parameter is referenced in this code change. > > > * if (x != NULL && !ignore_return) { * > > > > > > Author of this code change suggested me, we can safely remove this > > addition conditional expression ' && !ignore_return'. > > > > > > open webrev: > http://cr.openjdk.java.net/~shshahma/8134389/webrev.00/ > > > jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-8134389 > > > jdk9 changeset: http://hg.openjdk.java.net/jdk9/hs- > > comp/hotspot/rev/4191b33b3629 > > > > > > testing: Passes JPRT, jtreg on Linux [amd64] and newly added test > > > case > > > > > > Regards, > > > Shafi > > > From david.holmes at oracle.com Tue Oct 25 03:41:19 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 25 Oct 2016 13:41:19 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: On 24/10/2016 8:24 PM, Yasumasa Suenaga wrote: > Hi Dmitry, David, > > I want to resume to work for this issue because this issue has been > marked P3. > Dmitry, do you have any idea for it? > > IMHO, we can fix as below: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8165869/webrev.00/ > > According to David, JvmtiExport::load_agent_library() should return zero > when the operation which is kicked by AttachListener is succeeded. > AttachListener will write response code from > JvmtiExport::load_agent_library() at first, and the client (e.g. > HotSpotVirtualMachine.java) checks it. > > Thus I keep current implementation about return code, and I added the > code to send error message to the client. > It works fine with jdk/test/com/sun/tools/attach/BasicTests.java . > > What do you think about it? Sorry but I still see the problem as being in the JCmd code, as per my previous email on the subject. [1] David ----- [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-September/024546.html > > Thanks, > > Yasumasa > > > On 2016/09/15 20:08, Dmitry Samersoff wrote: >> Yasumasa, David, >> >> I'm looking into this issue and come back as soon as I have a clear >> picture. >> >> It takes some time. Sorry! >> >> -Dmitry >> >> On 2016-09-15 14:01, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> I agree the call sequence which you write in the case of "jcmd" >>> operation. >>> However, we should think about "load" operation in AttachListener. >>> >>> We can access JvmtiExport::load_agent_library() through two routes: >>> >>> Route "load": AttachListener -> JvmtiExport::load_agent_library() >>> Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> >>> DCmd::parse_and_execute() >>> >>> "load" sets error code (it means first data in the stream) when >>> JvmtiExport::load_agent_library() returns JNI_ERR. >>> OTOH "jcmd" sets error code if exception is occurred ONLY. >>> >>> If we try to load invalid JVMTI agent, "load" writes JNI_ERR to stream, >>> however "jcmd" writes JNI_OK because pending exception is not available >>> at this point. >>> Currently, JCmd.java shows "Command executed successfully" when it >>> cannot read the message from the socket. In case of this issue, >>> JVMTIAgentLoadDCmd does not write any message because it cannot attach >>> the agent. >>> If return code which is in the top of stream should mean whether >>> communication with AttachListener is succeeded, I think HotSpot should >>> write error message or error code to the stream for JCmd.java . >>> >>> I'm not sure this email is the answer for you. >>> Sorry for my English. >>> >>> >>> Yasumasa >>> >>> >>> On 2016/09/15 18:43, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>>>> Hi, >>>>> >>>>>> If this is done through jcmd then jcmd needs to know how to "unwrap" >>>>>> the information it gets back from the Dcmd. >>>>> >>>>> In case of JVMTI.agent_load dcmd, I think we should be able to get the >>>>> response as below: >>>>> >>>>> 1. Invalid JVMTI agent >>>>> 2. Agent_OnAttach() did not return JNI_OK >>>>> 3. Succeeded >>>>> >>>>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, >>>>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>>>> occurred (in Agent_OnAttach()). >>>> >>>> Can you respond to my comment in the bug report about the chain of >>>> calls and explain exactly where you think things are going wrong in >>>> this scenario. I'm having a lot of trouble trying to see how the >>>> information flows back through the layers. >>>> >>>>> IMHO, HotSpot should return error code (in top of the stream: >>>>> written by >>>>> AttachListener at first). So we should be able to get some error >>>>> code in >>>>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>>>> parse text information in the stream. >>>> >>>> It returns the high-level response code first "did communication with >>>> the target VM succeed", then the actual action response code. That >>>> seems the right thing to do to me. It is up to the callers reading the >>>> stream to understand how to read it. To me the issue lies somewhere on >>>> the jcmd/dcmd side. >>>> >>>> Thanks, >>>> David >>>> >>>>> >>>>> Sorry for my English. >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2016/09/14 7:03, David Holmes wrote: >>>>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>>>> Thanks David! >>>>>>> If we should not change the meaning of return code from >>>>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>>>> below: >>>>>>> ------------------- >>>>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>>>>>> +0000 >>>>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>>>>>> +0900 >>>>>>> @@ -2412,6 +2412,10 @@ >>>>>>> result = JNI_OK; >>>>>>> } >>>>>>> } >>>>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>>>> + if (result != JNI_OK) { >>>>>>> + st->print_cr("%d", result); >>>>>>> + } >>>>>>> return result; >>>>>>> } >>>>>>> ------------------- >>>>>> >>>>>> Not sure I see the point. "return result" will put the error code >>>>>> into >>>>>> the socket stream and that error will be seen and responded to. I >>>>>> don't expect anything to then read further into the stream to see the >>>>>> "result" repeated a second time. In other words if execute() doesn't >>>>>> see a zero result then you go no further; if execute sees a zero >>>>>> result then the actual action may have had an issue so you proceed to >>>>>> read the "action's result". >>>>>> >>>>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get >>>>>>> -1 if >>>>>>> we try to attach invalid agent. >>>>>> >>>>>> Can you please outline your test case for this again. If this is done >>>>>> through jcmd then jcmd needs to know how to "unwrap" the information >>>>>> it gets back from the Dcmd. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>>>> David, >>>>>>>> >>>>>>>> Thank you for the evaluation. >>>>>>>> >>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>> is to >>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>> Though I'm >>>>>>>>> not sure how exactly. >>>>>>>> >>>>>>>> This logic seems completely broken for me. But I don't see >>>>>>>> anything we >>>>>>>> can do right now - for jdk 9. >>>>>>>> >>>>>>>> It requires careful changes of both - code and tests. >>>>>>>> >>>>>>>> I can help with this task but not today. >>>>>>>> >>>>>>>> -Dmitry >>>>>>>> >>>>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>>>> >>>>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>>>> difference between what happens with your fix and what happens >>>>>>>>>> without. >>>>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>>>> how we >>>>>>>>>> get >>>>>>>>>> AgentInitializationException instead of AgentLoadException. I >>>>>>>>>> need >>>>>>>>>> to be >>>>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>>>> very >>>>>>>>>> difficult to arrange. :( >>>>>>>>> >>>>>>>>> I finally managed to connect all the pieces on this. >>>>>>>>> >>>>>>>>> The basic problem is that with the proposed change >>>>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which >>>>>>>>> then >>>>>>>>> leads to the InternalError. Without the change we reach this >>>>>>>>> block in >>>>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>>>> >>>>>>>>> int result = readInt(in); >>>>>>>>> if (result != 0) { >>>>>>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>>>>>> result); >>>>>>>>> } >>>>>>>>> >>>>>>>>> and so get the expected AgentInitializationException. >>>>>>>>> >>>>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>>>> original: >>>>>>>>> >>>>>>>>> st->print_cr("%d", result); >>>>>>>>> result = JNI_OK; >>>>>>>>> >>>>>>>>> then execute() manages to read a zero completion status from the >>>>>>>>> stream, >>>>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>>>> whether >>>>>>>>> zero or not - from the stream. This is because the AttachListener >>>>>>>>> code >>>>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>>>> wrote >>>>>>>>> the result value above in print_cr. Then if we look at, for >>>>>>>>> example, >>>>>>>>> LinuxAttachOperation::complete, we will write "result" to the >>>>>>>>> socket >>>>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>>>> operation >>>>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>>>> read 0 >>>>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>>>> proposed >>>>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>>>> throws >>>>>>>>> AgentLoadException. >>>>>>>>> >>>>>>>>> So in summary there are two different error indicators written >>>>>>>>> into >>>>>>>>> the >>>>>>>>> stream, and we need the first to be zero unless some major error >>>>>>>>> with >>>>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>>>> "load" >>>>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>>>> >>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>> is to >>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>> Though I'm >>>>>>>>> not sure how exactly. >>>>>>>>> >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>>>> someone >>>>>>>>>>> help me? >>>>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>>>> I think I see the problem. The attach framework tries to >>>>>>>>>>>> load the >>>>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>>>> actually >>>>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>>>> through. >>>>>>>>>>>> I'm guessing, at the moment, that the failure is due to a >>>>>>>>>>>> missing >>>>>>>>>>>> module related option. >>>>>>>>>>>> >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>>>> >>>>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>>>> >>>>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>>>> seems it >>>>>>>>>>>>> may >>>>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>>>> ended up >>>>>>>>>>>>> not >>>>>>>>>>>>> touching!). >>>>>>>>>>>>> >>>>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) >>>>>>>>>>>>> and >>>>>>>>>>>>> see >>>>>>>>>>>>> what >>>>>>>>>>>>> happens? We see AgentLoadException from the code that >>>>>>>>>>>>> internally >>>>>>>>>>>>> tries >>>>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>>>> >>>>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception >>>>>>>>>>>>> during >>>>>>>>>>>>> test >>>>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>>>> missing in >>>>>>>>>>>>> target VM >>>>>>>>>>>>> ... >>>>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>>>> missing in >>>>>>>>>>>>> target VM >>>>>>>>>>>>> ... >>>>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to >>>>>>>>>>>>> load >>>>>>>>>>>>> agent >>>>>>>>>>>>> library >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>>>>>> when >>>>>>>>>>>>>> the >>>>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>>>> fixed >>>>>>>>>>>>>> in 9 >>>>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>>>> >>>>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>>>> appropriate >>>>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>>>> problem - >>>>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>>>> unrelated >>>>>>>>>>>>> issue in our testing environment. >>>>>>>>>>>>> >>>>>>>>>>>>> Unfortunately I have some high priority tasks to handle right >>>>>>>>>>>>> now, >>>>>>>>>>>>> so I >>>>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>>>> >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>>>> For the record it looks like the tests involved are >>>>>>>>>>>>>>> broken, in >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>> because the VM used to lie about the success of attaching >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> agent, the >>>>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Harold >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>> >>>>>>>> >> >> From david.holmes at oracle.com Tue Oct 25 03:58:37 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 25 Oct 2016 13:58:37 +1000 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> Message-ID: <9371661e-99d1-6035-b9c4-d93d65fe3c72@oracle.com> On 25/10/2016 1:36 AM, Volker Simonis wrote: > So here comes the new webrev: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v2/ > > It moves the definition of CONST64/UCONST64 and min_jlong/max_jlong > from the compiler-specific files to globalDefinitions.hpp. Seems okay. > I also had to remove some AIX-specific constant definitions and > replace them by the more standard '*[K,M,G]' notation. Assuming AIX is 64-bit only this seems okay - the numeric values have changed from uint64_t to size_t, which should still be a 64-bit unsigned value. BTW I'm traveling tomorrow so would not be able to sponsor this till late Wednesday at the earliest. Thanks, David > I did build and smoke tested this on Linux/amd64, MaxOS X, AIX, > Solaris and Windows. > > Thanks, > Volker > > > On Mon, Oct 24, 2016 at 2:51 PM, Volker Simonis > wrote: >> Hi Mikael, >> >> you're right - the OpenJDK now has the same definitions for >> CONST64/UCONST64 on all platforms. If you don't have other settings >> for you're closed ports I can certainly generalize the definition into >> globalDefinitions.hpp (it also works for HP-UX which is our last >> closed platform :) >> >> Notice that this would also require to move the definition of >> min_jlong/max_jlong from the compiler-specific files to >> globalDefinitions.hpp. >> >> I can do a new webrev with these additional changes. >> >> Regards, >> Volker >> >> >> On Mon, Oct 24, 2016 at 11:21 AM, Mikael Gerdin >> wrote: >>> Hi Volker, >>> >>> >>> On 2016-10-24 10:53, Volker Simonis wrote: >>>> >>>> On Mon, Oct 24, 2016 at 9:14 AM, David Holmes >>>> wrote: >>>>> >>>>> On 24/10/2016 5:12 PM, Volker Simonis wrote: >>>>>> >>>>>> >>>>>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> Hi Volker, >>>>>>> >>>>>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> can I please have a review and sponsor for the following trivial fix: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 >>>>>>>> >>>>>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >>>>>>>> literals on Windows (see >>>>>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >>>>>>>> >>>>>>>> Unfortunately this suffix is not known to Eclipse so most of a hotspot >>>>>>>> files have errors in an Eclipse project (e.g. NULL is defined as >>>>>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in most >>>>>>>> hotspot of the source files). >>>>>>>> >>>>>>>> Fortunately VS supports the more standard conforming LL/ULL suffixes >>>>>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) since >>>>>>>> at least VS2010 (see >>>>>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >>>>>>>> therefore propose to change the suffix for integer literals from >>>>>>>> i64/ui64 to LL/ULL on Windows. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> In the sense that this changes the VIS specific code to use a more >>>>>>> stand >>>>>>> conforming form this seems fine. >>>>>>> >>>>>>> But as I wrote in the bug report, should we even be using this file >>>>>>> with >>>>>>> non >>>>>>> Visual Studio compilers? >>>>>>> >>>>>> >>>>>> What I meant with "Eclipse doesn't understand" the suffix is that the >>>>>> Eclipse C++ parser doesn't understand it. Eclipse just parses all the >>>>>> C++ files which belong to a project in order to build up its code >>>>>> database for code navigation. As such it reads >>>>>> globalDefinitions_visCPP.hpp on Windows (if we configure a >>>>>> corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... >>>>>> >>>>>> This does not mean that we are using globalDefinitions_visCPP.hpp for >>>>>> building with another compiler. And we have no IDE-specific headers >>>>>> (this actually makes no sense since the IDEs should use the same >>>>>> settings which are used for the actual builds). >>>>>> >>>>>> Not sure about other IDEs, but Eclipse's C++ parser doesn't understand >>>>>> the i64/ui64 suffix and as there is a better, more standard-conforming >>>>>> way of declaring 64-bit integer literals on Windows I've just proposed >>>>>> the cleanup. >>>>>> >>>>>> Does this answer your question? >>>>> >>>>> >>>>> >>>>> Yes - thanks for clarifying. >>>>> >>>> >>>> Your welcome! Could you then please sponsor this change :) >>> >>> >>> Does this change make the definition of CONST64 / UCONST64 the same on all >>> globalDefinitions_x variants? >>> If so can we move the definition to the common file instead of having it in >>> the per-compiler files? >>> >>> Thanks >>> /Mikael >>> >>> >>>> >>>>> >>>>> David >>>>> >>>>>> Thank you and best egards, >>>>>> Volker >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> >>>>>>>> Thank you and best regards, >>>>>>>> Volker >>>>>>>> >>>>>>> >>>>> >>> From thomas.stuefe at gmail.com Tue Oct 25 06:09:56 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Oct 2016 08:09:56 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> Message-ID: Hi Volker, so you didn't like my old SIZE_.. macros huh :) The changes are fine, and strictly speaking make the AIX port more correct, because before it was assumed all over the place that size_t is 64bit, and now we use the correct size_t typed constants. So should we ever do an AIX 32bit port this will help :) Nitpicks: - globals_aix.hpp: you can remove the UCONST64 macro on MaxExpectedDataSegmentSize. - virtualspace.cpp: can you remove pls also remove the SIZE_xx definitions and usage around get_attach_addresses_for_disjoint_mode()? Thanks, Thomas On Mon, Oct 24, 2016 at 5:36 PM, Volker Simonis wrote: > So here comes the new webrev: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v2/ > > It moves the definition of CONST64/UCONST64 and min_jlong/max_jlong > from the compiler-specific files to globalDefinitions.hpp. > > I also had to remove some AIX-specific constant definitions and > replace them by the more standard '*[K,M,G]' notation. > > I did build and smoke tested this on Linux/amd64, MaxOS X, AIX, > Solaris and Windows. > > Thanks, > Volker > > > On Mon, Oct 24, 2016 at 2:51 PM, Volker Simonis > wrote: > > Hi Mikael, > > > > you're right - the OpenJDK now has the same definitions for > > CONST64/UCONST64 on all platforms. If you don't have other settings > > for you're closed ports I can certainly generalize the definition into > > globalDefinitions.hpp (it also works for HP-UX which is our last > > closed platform :) > > > > Notice that this would also require to move the definition of > > min_jlong/max_jlong from the compiler-specific files to > > globalDefinitions.hpp. > > > > I can do a new webrev with these additional changes. > > > > Regards, > > Volker > > > > > > On Mon, Oct 24, 2016 at 11:21 AM, Mikael Gerdin > > wrote: > >> Hi Volker, > >> > >> > >> On 2016-10-24 10:53, Volker Simonis wrote: > >>> > >>> On Mon, Oct 24, 2016 at 9:14 AM, David Holmes > > >>> wrote: > >>>> > >>>> On 24/10/2016 5:12 PM, Volker Simonis wrote: > >>>>> > >>>>> > >>>>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes < > david.holmes at oracle.com> > >>>>> wrote: > >>>>>> > >>>>>> > >>>>>> Hi Volker, > >>>>>> > >>>>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> can I please have a review and sponsor for the following trivial > fix: > >>>>>>> > >>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ > >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 > >>>>>>> > >>>>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer > >>>>>>> literals on Windows (see > >>>>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). > >>>>>>> > >>>>>>> Unfortunately this suffix is not known to Eclipse so most of a > hotspot > >>>>>>> files have errors in an Eclipse project (e.g. NULL is defined as > >>>>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in > most > >>>>>>> hotspot of the source files). > >>>>>>> > >>>>>>> Fortunately VS supports the more standard conforming LL/ULL > suffixes > >>>>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) > since > >>>>>>> at least VS2010 (see > >>>>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100). > aspx).I > >>>>>>> therefore propose to change the suffix for integer literals from > >>>>>>> i64/ui64 to LL/ULL on Windows. > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> In the sense that this changes the VIS specific code to use a more > >>>>>> stand > >>>>>> conforming form this seems fine. > >>>>>> > >>>>>> But as I wrote in the bug report, should we even be using this file > >>>>>> with > >>>>>> non > >>>>>> Visual Studio compilers? > >>>>>> > >>>>> > >>>>> What I meant with "Eclipse doesn't understand" the suffix is that the > >>>>> Eclipse C++ parser doesn't understand it. Eclipse just parses all the > >>>>> C++ files which belong to a project in order to build up its code > >>>>> database for code navigation. As such it reads > >>>>> globalDefinitions_visCPP.hpp on Windows (if we configure a > >>>>> corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... > >>>>> > >>>>> This does not mean that we are using globalDefinitions_visCPP.hpp for > >>>>> building with another compiler. And we have no IDE-specific headers > >>>>> (this actually makes no sense since the IDEs should use the same > >>>>> settings which are used for the actual builds). > >>>>> > >>>>> Not sure about other IDEs, but Eclipse's C++ parser doesn't > understand > >>>>> the i64/ui64 suffix and as there is a better, more > standard-conforming > >>>>> way of declaring 64-bit integer literals on Windows I've just > proposed > >>>>> the cleanup. > >>>>> > >>>>> Does this answer your question? > >>>> > >>>> > >>>> > >>>> Yes - thanks for clarifying. > >>>> > >>> > >>> Your welcome! Could you then please sponsor this change :) > >> > >> > >> Does this change make the definition of CONST64 / UCONST64 the same on > all > >> globalDefinitions_x variants? > >> If so can we move the definition to the common file instead of having > it in > >> the per-compiler files? > >> > >> Thanks > >> /Mikael > >> > >> > >>> > >>>> > >>>> David > >>>> > >>>>> Thank you and best egards, > >>>>> Volker > >>>>> > >>>>> > >>>>>> Thanks, > >>>>>> David > >>>>>> > >>>>>> > >>>>>>> Thank you and best regards, > >>>>>>> Volker > >>>>>>> > >>>>>> > >>>> > >> > From dmitry.samersoff at oracle.com Tue Oct 25 06:09:02 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 25 Oct 2016 09:09:02 +0300 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: <1af531b8-fc35-bc25-d1c8-1688517399ac@oracle.com> Yasumasa, It's hard to fix this issue without access to Oracle testing infrastructure. I'm working on the issue but unfortunately I was caught by some internal problems so it progresses slowly than I expected. I'm sorry about it. -Dmitry On 2016-10-24 13:24, Yasumasa Suenaga wrote: > Hi Dmitry, David, > > I want to resume to work for this issue because this issue has been > marked P3. > Dmitry, do you have any idea for it? > > IMHO, we can fix as below: > > http://cr.openjdk.java.net/~ysuenaga/JDK-8165869/webrev.00/ > > According to David, JvmtiExport::load_agent_library() should return zero > when the operation which is kicked by AttachListener is succeeded. > AttachListener will write response code from > JvmtiExport::load_agent_library() at first, and the client (e.g. > HotSpotVirtualMachine.java) checks it. > > Thus I keep current implementation about return code, and I added the > code to send error message to the client. > It works fine with jdk/test/com/sun/tools/attach/BasicTests.java . > > What do you think about it? > > > Thanks, > > Yasumasa > > > On 2016/09/15 20:08, Dmitry Samersoff wrote: >> Yasumasa, David, >> >> I'm looking into this issue and come back as soon as I have a clear >> picture. >> >> It takes some time. Sorry! >> >> -Dmitry >> >> On 2016-09-15 14:01, Yasumasa Suenaga wrote: >>> Hi David, >>> >>> I agree the call sequence which you write in the case of "jcmd" >>> operation. >>> However, we should think about "load" operation in AttachListener. >>> >>> We can access JvmtiExport::load_agent_library() through two routes: >>> >>> Route "load": AttachListener -> JvmtiExport::load_agent_library() >>> Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> >>> DCmd::parse_and_execute() >>> >>> "load" sets error code (it means first data in the stream) when >>> JvmtiExport::load_agent_library() returns JNI_ERR. >>> OTOH "jcmd" sets error code if exception is occurred ONLY. >>> >>> If we try to load invalid JVMTI agent, "load" writes JNI_ERR to stream, >>> however "jcmd" writes JNI_OK because pending exception is not available >>> at this point. >>> Currently, JCmd.java shows "Command executed successfully" when it >>> cannot read the message from the socket. In case of this issue, >>> JVMTIAgentLoadDCmd does not write any message because it cannot attach >>> the agent. >>> If return code which is in the top of stream should mean whether >>> communication with AttachListener is succeeded, I think HotSpot should >>> write error message or error code to the stream for JCmd.java . >>> >>> I'm not sure this email is the answer for you. >>> Sorry for my English. >>> >>> >>> Yasumasa >>> >>> >>> On 2016/09/15 18:43, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>>>> Hi, >>>>> >>>>>> If this is done through jcmd then jcmd needs to know how to "unwrap" >>>>>> the information it gets back from the Dcmd. >>>>> >>>>> In case of JVMTI.agent_load dcmd, I think we should be able to get the >>>>> response as below: >>>>> >>>>> 1. Invalid JVMTI agent >>>>> 2. Agent_OnAttach() did not return JNI_OK >>>>> 3. Succeeded >>>>> >>>>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, >>>>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>>>> occurred (in Agent_OnAttach()). >>>> >>>> Can you respond to my comment in the bug report about the chain of >>>> calls and explain exactly where you think things are going wrong in >>>> this scenario. I'm having a lot of trouble trying to see how the >>>> information flows back through the layers. >>>> >>>>> IMHO, HotSpot should return error code (in top of the stream: >>>>> written by >>>>> AttachListener at first). So we should be able to get some error >>>>> code in >>>>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>>>> parse text information in the stream. >>>> >>>> It returns the high-level response code first "did communication with >>>> the target VM succeed", then the actual action response code. That >>>> seems the right thing to do to me. It is up to the callers reading the >>>> stream to understand how to read it. To me the issue lies somewhere on >>>> the jcmd/dcmd side. >>>> >>>> Thanks, >>>> David >>>> >>>>> >>>>> Sorry for my English. >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2016/09/14 7:03, David Holmes wrote: >>>>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>>>> Thanks David! >>>>>>> If we should not change the meaning of return code from >>>>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>>>> below: >>>>>>> ------------------- >>>>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>>>>>> +0000 >>>>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>>>>>> +0900 >>>>>>> @@ -2412,6 +2412,10 @@ >>>>>>> result = JNI_OK; >>>>>>> } >>>>>>> } >>>>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>>>> + if (result != JNI_OK) { >>>>>>> + st->print_cr("%d", result); >>>>>>> + } >>>>>>> return result; >>>>>>> } >>>>>>> ------------------- >>>>>> >>>>>> Not sure I see the point. "return result" will put the error code >>>>>> into >>>>>> the socket stream and that error will be seen and responded to. I >>>>>> don't expect anything to then read further into the stream to see the >>>>>> "result" repeated a second time. In other words if execute() doesn't >>>>>> see a zero result then you go no further; if execute sees a zero >>>>>> result then the actual action may have had an issue so you proceed to >>>>>> read the "action's result". >>>>>> >>>>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get >>>>>>> -1 if >>>>>>> we try to attach invalid agent. >>>>>> >>>>>> Can you please outline your test case for this again. If this is done >>>>>> through jcmd then jcmd needs to know how to "unwrap" the information >>>>>> it gets back from the Dcmd. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>>>> David, >>>>>>>> >>>>>>>> Thank you for the evaluation. >>>>>>>> >>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>> is to >>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>> Though I'm >>>>>>>>> not sure how exactly. >>>>>>>> >>>>>>>> This logic seems completely broken for me. But I don't see >>>>>>>> anything we >>>>>>>> can do right now - for jdk 9. >>>>>>>> >>>>>>>> It requires careful changes of both - code and tests. >>>>>>>> >>>>>>>> I can help with this task but not today. >>>>>>>> >>>>>>>> -Dmitry >>>>>>>> >>>>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>>>> >>>>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>>>> difference between what happens with your fix and what happens >>>>>>>>>> without. >>>>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>>>> how we >>>>>>>>>> get >>>>>>>>>> AgentInitializationException instead of AgentLoadException. I >>>>>>>>>> need >>>>>>>>>> to be >>>>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>>>> very >>>>>>>>>> difficult to arrange. :( >>>>>>>>> >>>>>>>>> I finally managed to connect all the pieces on this. >>>>>>>>> >>>>>>>>> The basic problem is that with the proposed change >>>>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which >>>>>>>>> then >>>>>>>>> leads to the InternalError. Without the change we reach this >>>>>>>>> block in >>>>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>>>> >>>>>>>>> int result = readInt(in); >>>>>>>>> if (result != 0) { >>>>>>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>>>>>> result); >>>>>>>>> } >>>>>>>>> >>>>>>>>> and so get the expected AgentInitializationException. >>>>>>>>> >>>>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>>>> original: >>>>>>>>> >>>>>>>>> st->print_cr("%d", result); >>>>>>>>> result = JNI_OK; >>>>>>>>> >>>>>>>>> then execute() manages to read a zero completion status from the >>>>>>>>> stream, >>>>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>>>> whether >>>>>>>>> zero or not - from the stream. This is because the AttachListener >>>>>>>>> code >>>>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>>>> wrote >>>>>>>>> the result value above in print_cr. Then if we look at, for >>>>>>>>> example, >>>>>>>>> LinuxAttachOperation::complete, we will write "result" to the >>>>>>>>> socket >>>>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>>>> operation >>>>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>>>> read 0 >>>>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>>>> proposed >>>>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>>>> throws >>>>>>>>> AgentLoadException. >>>>>>>>> >>>>>>>>> So in summary there are two different error indicators written >>>>>>>>> into >>>>>>>>> the >>>>>>>>> stream, and we need the first to be zero unless some major error >>>>>>>>> with >>>>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>>>> "load" >>>>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>>>> >>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>> is to >>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>> Though I'm >>>>>>>>> not sure how exactly. >>>>>>>>> >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>>>> someone >>>>>>>>>>> help me? >>>>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>>>> I think I see the problem. The attach framework tries to >>>>>>>>>>>> load the >>>>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>>>> actually >>>>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>>>> through. >>>>>>>>>>>> I'm guessing, at the moment, that the failure is due to a >>>>>>>>>>>> missing >>>>>>>>>>>> module related option. >>>>>>>>>>>> >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>> >>>>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>>>> >>>>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>>>> >>>>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>>>> seems it >>>>>>>>>>>>> may >>>>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>>>> ended up >>>>>>>>>>>>> not >>>>>>>>>>>>> touching!). >>>>>>>>>>>>> >>>>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) >>>>>>>>>>>>> and >>>>>>>>>>>>> see >>>>>>>>>>>>> what >>>>>>>>>>>>> happens? We see AgentLoadException from the code that >>>>>>>>>>>>> internally >>>>>>>>>>>>> tries >>>>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>>>> >>>>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception >>>>>>>>>>>>> during >>>>>>>>>>>>> test >>>>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>>>> missing in >>>>>>>>>>>>> target VM >>>>>>>>>>>>> ... >>>>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>>>> missing in >>>>>>>>>>>>> target VM >>>>>>>>>>>>> ... >>>>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to >>>>>>>>>>>>> load >>>>>>>>>>>>> agent >>>>>>>>>>>>> library >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>>>>>> when >>>>>>>>>>>>>> the >>>>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>>>> fixed >>>>>>>>>>>>>> in 9 >>>>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>>>> >>>>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>>>> appropriate >>>>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>>>> problem - >>>>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>>>> unrelated >>>>>>>>>>>>> issue in our testing environment. >>>>>>>>>>>>> >>>>>>>>>>>>> Unfortunately I have some high priority tasks to handle right >>>>>>>>>>>>> now, >>>>>>>>>>>>> so I >>>>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>>>> >>>>>>>>>>>>> David >>>>>>>>>>>>> ----- >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>>>> For the record it looks like the tests involved are >>>>>>>>>>>>>>> broken, in >>>>>>>>>>>>>>> that >>>>>>>>>>>>>>> because the VM used to lie about the success of attaching >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> agent, the >>>>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Harold >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>> >>>>>>>> >> >> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From thomas.stuefe at gmail.com Tue Oct 25 07:33:46 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Oct 2016 09:33:46 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: <20161024140105.GE19291@ehelin.jrpg.bea.com> References: <20161013121510.GM19291@ehelin.jrpg.bea.com> <20161024140105.GE19291@ehelin.jrpg.bea.com> Message-ID: Hi Erik, On Mon, Oct 24, 2016 at 4:01 PM, Erik Helin wrote: > On 2016-10-13, Thomas St?fe wrote: > > Hi Erik, > > > > On Thu, Oct 13, 2016 at 2:15 PM, Erik Helin > wrote: > > > > > Hi Thomas, > > > > > > thanks for submitting the JEP and proposing this feature! > > > > > > On 2016-10-10, Thomas St?fe wrote: > > > > Hi all, > > > > > > > > May I have please some feedback for this enhancement proposal? > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > > > > > > > > > In one very short sentence it proposes a better allocation scheme for > > > > Metaspace Chunks in order to reduce fragmentation and metaspace > waste. > > > > > > > > I also added a short presentation which describes the problem and > how we > > > > solved it in our VM port. > > > > > > > > https://bugs.openjdk.java.net/secure/attachment/63894/ > > > Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf > > > > > > Do we really need the flag -XX:+CoalesceMetaspace? Having two > differnent > > > ways to handle the chunk free lists in Metaspace is unfortunate, it > > > might introduce hard to detect bugs and will also require much more > > > testing (runnings lots of tests with the flag both on and off). > > > > > > > You are right. If the new allocator works well, there is no reason to > keep > > the old allocator around. > > > > We wanted for a temporary time to be able to switch between both old and > > new allocator. Just to have a fallback if problems occur. But if it > works, > > it makes sense to only have one allocator, and the "CoalesceMetaspace" > flag > > can be removed, and also the code can be made a lot simpler because we do > > not need both code paths. > > Yeah, I would strongly prefer to not introduce a new flag for this. Have > you thought about testing? Do you intend to write new tests to stress > the coalescing? > The current version of my patch contains a lot of verification code, which is activated by default for the debug case. Nightly we run lots of test suites and benchmarks on all our platforms, so the patch already got stressed a lot. We also have tests specific for the metaspace. The patch only makes sense with thorough testing, so I consider writing tests just a part of implementing the JEP. > > > > > > Do you think your proposed solution has low enough overhead (in terms > > > of CPU and memory) to be on "by default"? > > > > > > > We decided to switch it on by default in our VM. > > > > Memory overhead can be almost exactly calculated. Bitmasks take 2 bits > per > > specialized-chunk-sized-area. That means, for specialized-chunk-size = 1k > > (128 meta words): metaspace size / 8192. So, for 1G of metaspace we pay > > 132KB overhead for the bitmasks, or roughly 0.1%. > > > > There is some CPU overhead, but in my tests I could not measure anything > > above noise level. > > Those numbers seems low enough to me in order to not warrant a new flag. > > > > Thanks, > > > Erik > > > > > > > > Btw, I understand that it is difficult to estimate this proposal without > a > > prototype to play around. As I already mentioned, the patch right now > only > > exists in our code base and not yet in the OpenJDK. If you guys are > > seriously interested in this JEP, I will invest the time to port the > patch > > to the OpenJDK, so that you can check it out for yourself. > > Yes, we are seriously interested :) I think the proposal sounds good. I > guess > the devil will be in the details, so I (we) would really appreciate if > you want to port your internal patch to OpenJDK. > > Ok, thanks Erik. I will implement a prototype then and come back to you once it is done. Kind Regards, Thomas > Thanks, > Erik > > > Kind Regards, Thomas > > > > > > > > > > > > Thank you very much! > > > > > > > > Kind Regards, Thomas > > > > > > > > > > > > On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe < > thomas.stuefe at gmail.com> > > > > wrote: > > > > > > > > > Dear all, > > > > > > > > > > please take a look at this Enhancement Proposal for the metaspace > > > > > allocator. I hope these are the right groups for this discussion. > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > > > > > > > Background: > > > > > > > > > > We at SAP see at times at customer installations OOMs in Metaspace > > > > > (usually, with compressed class pointers enabled, in Compressed > Class > > > > > Space). The VM attempts to allocate metaspace and fails, hitting > the > > > > > CompressedClassSpaceSize limit. Note that we usually set the limit > > > lower > > > > > than the default, typically at 256M. > > > > > > > > > > When analyzing, we observed that a large part of the metaspace is > > > indeed > > > > > free but "locked in" into metaspace chunks of the wrong size: > often we > > > > > would find a lot of free small chunks, but the allocation request > was > > > for > > > > > medium chunks, and failed. > > > > > > > > > > The reason was that if at some point in time a lot of class loaders > > > were > > > > > alive, each with only a few small classes loaded. This would lead > to > > > the > > > > > metaspace being swamped with lots of small chunks. This is because > each > > > > > SpaceManager first allocates small chunks, only after a certain > amount > > > of > > > > > allocation requests switches to larger chunks. > > > > > > > > > > These small chunks are free and wait in the freelist, but cannot be > > > reused > > > > > for allocation requests which require larger chunks, even if they > are > > > > > physically adjacent in the virtual space. > > > > > > > > > > We (at SAP) added a patch which allows on-the-fly metaspace chunk > > > merging > > > > > - to merge multiple adjacent smaller chunk to form a larger chunk. > > > This, in > > > > > combination with the reverse direction - splitting a large chunk > to get > > > > > smaller chunks - partly negates the "chunks-are-locked-in-into- > > > their-size" > > > > > limitation and provides for better reuse of metaspace chunks. It > also > > > > > provides better defragmentation as well. > > > > > > > > > > I discussed this fix off-list with Coleen Phillimore and Jon > Masamitsu, > > > > > and instead of just offering this as a fix, both recommended to > open a > > > JEP > > > > > for this, because its scope would be beyond that of a simple fix. > > > > > > > > > > So here is my first JEP :) I hope it follows the right form. > Please, if > > > > > you have time, take a look and tell us what you think. > > > > > > > > > > Thank you, and Kind Regards, > > > > > > > > > > Thomas St?fe > > > > > > > > > > > > > > > > > > > > > > > > From thomas.stuefe at gmail.com Tue Oct 25 07:39:15 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Oct 2016 09:39:15 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: <6482d642-428a-0e53-c8da-d61de968defe@oracle.com> References: <20161013121510.GM19291@ehelin.jrpg.bea.com> <20161024140105.GE19291@ehelin.jrpg.bea.com> <6482d642-428a-0e53-c8da-d61de968defe@oracle.com> Message-ID: Hi Coleen, thank you for feedback and encouragement :) See further comments inline. On Mon, Oct 24, 2016 at 6:32 PM, Coleen Phillimore < coleen.phillimore at oracle.com> wrote: > > Hi Thomas, > > I agree with Erik. If this works well for you, then it should just be > implemented without an option. If done early in JDK10, it'll get a lot of > good testing. > Ok. Was hoping to get this into JDK9, but if I am thinking clearly about this, I see that maybe the risk is too large. So lets implement this in 10 and if it is stable and works well, it can be backported to jdk9, yes? > > This looks like a very good improvement. We had discussed coalescing > blocks and other improvements like this early on, but wanted to wait to see > problems in the field to motivate the changes. We've seen these sorts of > problems now too. > > One of the things we've considered is that we wanted to use the operating > system's version of malloc for the chunks, so that it can split and > coalesce chunks for us. The Solaris malloc is improved recently. But I > don't think it's time to make change to use malloc'ed chunks yet because we > have to consider all of the operating systems that we and the OpenJDK > community supports. > I do not see how you could get CompressedClassPointers to work with native malloc? You would have to be sure that the pointers returned by malloc are within the numerical range for the 32bit class pointers. I thought that was the reason for using a continuous address range when allocating compressed class space. > > So, yes, I think the JEP looks good and your slides are absolutely > beautiful. Everyone should see these slides. > > :) Thanks! So I will provide a prototype, for now based on jdk9, and we will see how we go from there. Thanks, and Kind Regards, Thomas > Thanks, > Coleen > > > > On 10/24/16 10:01 AM, Erik Helin wrote: > >> On 2016-10-13, Thomas St?fe wrote: >> >>> Hi Erik, >>> >>> On Thu, Oct 13, 2016 at 2:15 PM, Erik Helin >>> wrote: >>> >>> Hi Thomas, >>>> >>>> thanks for submitting the JEP and proposing this feature! >>>> >>>> On 2016-10-10, Thomas St?fe wrote: >>>> >>>>> Hi all, >>>>> >>>>> May I have please some feedback for this enhancement proposal? >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8166690 >>>>> >>>>> >>>>> In one very short sentence it proposes a better allocation scheme for >>>>> Metaspace Chunks in order to reduce fragmentation and metaspace waste. >>>>> >>>>> I also added a short presentation which describes the problem and how >>>>> we >>>>> solved it in our VM port. >>>>> >>>>> https://bugs.openjdk.java.net/secure/attachment/63894/ >>>>> >>>> Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf >>>> >>>> Do we really need the flag -XX:+CoalesceMetaspace? Having two differnent >>>> ways to handle the chunk free lists in Metaspace is unfortunate, it >>>> might introduce hard to detect bugs and will also require much more >>>> testing (runnings lots of tests with the flag both on and off). >>>> >>>> You are right. If the new allocator works well, there is no reason to >>> keep >>> the old allocator around. >>> >>> We wanted for a temporary time to be able to switch between both old and >>> new allocator. Just to have a fallback if problems occur. But if it >>> works, >>> it makes sense to only have one allocator, and the "CoalesceMetaspace" >>> flag >>> can be removed, and also the code can be made a lot simpler because we do >>> not need both code paths. >>> >> Yeah, I would strongly prefer to not introduce a new flag for this. Have >> you thought about testing? Do you intend to write new tests to stress >> the coalescing? >> >> Do you think your proposed solution has low enough overhead (in terms >>>> of CPU and memory) to be on "by default"? >>>> >>>> We decided to switch it on by default in our VM. >>> >>> Memory overhead can be almost exactly calculated. Bitmasks take 2 bits >>> per >>> specialized-chunk-sized-area. That means, for specialized-chunk-size = 1k >>> (128 meta words): metaspace size / 8192. So, for 1G of metaspace we pay >>> 132KB overhead for the bitmasks, or roughly 0.1%. >>> >>> There is some CPU overhead, but in my tests I could not measure anything >>> above noise level. >>> >> Those numbers seems low enough to me in order to not warrant a new flag. >> >> Thanks, >>>> Erik >>>> >>>> >>>> Btw, I understand that it is difficult to estimate this proposal >>> without a >>> prototype to play around. As I already mentioned, the patch right now >>> only >>> exists in our code base and not yet in the OpenJDK. If you guys are >>> seriously interested in this JEP, I will invest the time to port the >>> patch >>> to the OpenJDK, so that you can check it out for yourself. >>> >> Yes, we are seriously interested :) I think the proposal sounds good. I >> guess >> the devil will be in the details, so I (we) would really appreciate if >> you want to port your internal patch to OpenJDK. >> >> Thanks, >> Erik >> >> Kind Regards, Thomas >>> >>> >>> >>> >>> Thank you very much! >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>> On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe < >>>>> thomas.stuefe at gmail.com> >>>>> wrote: >>>>> >>>>> Dear all, >>>>>> >>>>>> please take a look at this Enhancement Proposal for the metaspace >>>>>> allocator. I hope these are the right groups for this discussion. >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166690 >>>>>> >>>>>> Background: >>>>>> >>>>>> We at SAP see at times at customer installations OOMs in Metaspace >>>>>> (usually, with compressed class pointers enabled, in Compressed Class >>>>>> Space). The VM attempts to allocate metaspace and fails, hitting the >>>>>> CompressedClassSpaceSize limit. Note that we usually set the limit >>>>>> >>>>> lower >>>> >>>>> than the default, typically at 256M. >>>>>> >>>>>> When analyzing, we observed that a large part of the metaspace is >>>>>> >>>>> indeed >>>> >>>>> free but "locked in" into metaspace chunks of the wrong size: often we >>>>>> would find a lot of free small chunks, but the allocation request was >>>>>> >>>>> for >>>> >>>>> medium chunks, and failed. >>>>>> >>>>>> The reason was that if at some point in time a lot of class loaders >>>>>> >>>>> were >>>> >>>>> alive, each with only a few small classes loaded. This would lead to >>>>>> >>>>> the >>>> >>>>> metaspace being swamped with lots of small chunks. This is because each >>>>>> SpaceManager first allocates small chunks, only after a certain amount >>>>>> >>>>> of >>>> >>>>> allocation requests switches to larger chunks. >>>>>> >>>>>> These small chunks are free and wait in the freelist, but cannot be >>>>>> >>>>> reused >>>> >>>>> for allocation requests which require larger chunks, even if they are >>>>>> physically adjacent in the virtual space. >>>>>> >>>>>> We (at SAP) added a patch which allows on-the-fly metaspace chunk >>>>>> >>>>> merging >>>> >>>>> - to merge multiple adjacent smaller chunk to form a larger chunk. >>>>>> >>>>> This, in >>>> >>>>> combination with the reverse direction - splitting a large chunk to get >>>>>> smaller chunks - partly negates the "chunks-are-locked-in-into- >>>>>> >>>>> their-size" >>>> >>>>> limitation and provides for better reuse of metaspace chunks. It also >>>>>> provides better defragmentation as well. >>>>>> >>>>>> I discussed this fix off-list with Coleen Phillimore and Jon >>>>>> Masamitsu, >>>>>> and instead of just offering this as a fix, both recommended to open a >>>>>> >>>>> JEP >>>> >>>>> for this, because its scope would be beyond that of a simple fix. >>>>>> >>>>>> So here is my first JEP :) I hope it follows the right form. Please, >>>>>> if >>>>>> you have time, take a look and tell us what you think. >>>>>> >>>>>> Thank you, and Kind Regards, >>>>>> >>>>>> Thomas St?fe >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> > From yasuenag at gmail.com Tue Oct 25 09:28:03 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 25 Oct 2016 18:28:03 +0900 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: Hi David, > The completion status will be 0 because we did manage to communicate with the target VM and process the DCmd request. It passes sis back up to the caller. So looking at JCmd.java I see: > > try (InputStream in = hvm.executeJCmd(line);) { > // read to EOF and just print output > byte b[] = new byte[256]; > int n; > boolean messagePrinted = false; > do { > n = in.read(b); > if (n > 0) { > String s = new String(b, 0, n, "UTF-8"); > System.out.print(s); > messagePrinted = true; > } > } while (n > 0); > if (!messagePrinted) { > System.out.println("Command executed successfully"); > } > } > > and that logic, AFAICS, is not parsing out a return code followed by an error message! IMHO, JCmd#executeCommandForPid() should not parse return code from DCmd because most of DCmds will not return the code. For example, HelpDCmd and SystemGCDcmd will not return it. If we should parse return code in JCmd.java, I guess we have to add return code to all DCmds. IMHO, they should be fixed on another issue. Thanks, Yasumasa On 2016/10/25 12:41, David Holmes wrote: > On 24/10/2016 8:24 PM, Yasumasa Suenaga wrote: >> Hi Dmitry, David, >> >> I want to resume to work for this issue because this issue has been >> marked P3. >> Dmitry, do you have any idea for it? >> >> IMHO, we can fix as below: >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8165869/webrev.00/ >> >> According to David, JvmtiExport::load_agent_library() should return zero >> when the operation which is kicked by AttachListener is succeeded. >> AttachListener will write response code from >> JvmtiExport::load_agent_library() at first, and the client (e.g. >> HotSpotVirtualMachine.java) checks it. >> >> Thus I keep current implementation about return code, and I added the >> code to send error message to the client. >> It works fine with jdk/test/com/sun/tools/attach/BasicTests.java . >> >> What do you think about it? > > Sorry but I still see the problem as being in the JCmd code, as per my previous email on the subject. [1] > > David > ----- > > [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-September/024546.html > > > >> >> Thanks, >> >> Yasumasa >> >> >> On 2016/09/15 20:08, Dmitry Samersoff wrote: >>> Yasumasa, David, >>> >>> I'm looking into this issue and come back as soon as I have a clear >>> picture. >>> >>> It takes some time. Sorry! >>> >>> -Dmitry >>> >>> On 2016-09-15 14:01, Yasumasa Suenaga wrote: >>>> Hi David, >>>> >>>> I agree the call sequence which you write in the case of "jcmd" >>>> operation. >>>> However, we should think about "load" operation in AttachListener. >>>> >>>> We can access JvmtiExport::load_agent_library() through two routes: >>>> >>>> Route "load": AttachListener -> JvmtiExport::load_agent_library() >>>> Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> >>>> DCmd::parse_and_execute() >>>> >>>> "load" sets error code (it means first data in the stream) when >>>> JvmtiExport::load_agent_library() returns JNI_ERR. >>>> OTOH "jcmd" sets error code if exception is occurred ONLY. >>>> >>>> If we try to load invalid JVMTI agent, "load" writes JNI_ERR to stream, >>>> however "jcmd" writes JNI_OK because pending exception is not available >>>> at this point. >>>> Currently, JCmd.java shows "Command executed successfully" when it >>>> cannot read the message from the socket. In case of this issue, >>>> JVMTIAgentLoadDCmd does not write any message because it cannot attach >>>> the agent. >>>> If return code which is in the top of stream should mean whether >>>> communication with AttachListener is succeeded, I think HotSpot should >>>> write error message or error code to the stream for JCmd.java . >>>> >>>> I'm not sure this email is the answer for you. >>>> Sorry for my English. >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2016/09/15 18:43, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>>>>> Hi, >>>>>> >>>>>>> If this is done through jcmd then jcmd needs to know how to "unwrap" >>>>>>> the information it gets back from the Dcmd. >>>>>> >>>>>> In case of JVMTI.agent_load dcmd, I think we should be able to get the >>>>>> response as below: >>>>>> >>>>>> 1. Invalid JVMTI agent >>>>>> 2. Agent_OnAttach() did not return JNI_OK >>>>>> 3. Succeeded >>>>>> >>>>>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, >>>>>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>>>>> occurred (in Agent_OnAttach()). >>>>> >>>>> Can you respond to my comment in the bug report about the chain of >>>>> calls and explain exactly where you think things are going wrong in >>>>> this scenario. I'm having a lot of trouble trying to see how the >>>>> information flows back through the layers. >>>>> >>>>>> IMHO, HotSpot should return error code (in top of the stream: >>>>>> written by >>>>>> AttachListener at first). So we should be able to get some error >>>>>> code in >>>>>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>>>>> parse text information in the stream. >>>>> >>>>> It returns the high-level response code first "did communication with >>>>> the target VM succeed", then the actual action response code. That >>>>> seems the right thing to do to me. It is up to the callers reading the >>>>> stream to understand how to read it. To me the issue lies somewhere on >>>>> the jcmd/dcmd side. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> >>>>>> Sorry for my English. >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2016/09/14 7:03, David Holmes wrote: >>>>>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>>>>> Thanks David! >>>>>>>> If we should not change the meaning of return code from >>>>>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>>>>> below: >>>>>>>> ------------------- >>>>>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>>>>>>> +0000 >>>>>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>>>>>>> +0900 >>>>>>>> @@ -2412,6 +2412,10 @@ >>>>>>>> result = JNI_OK; >>>>>>>> } >>>>>>>> } >>>>>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>>>>> + if (result != JNI_OK) { >>>>>>>> + st->print_cr("%d", result); >>>>>>>> + } >>>>>>>> return result; >>>>>>>> } >>>>>>>> ------------------- >>>>>>> >>>>>>> Not sure I see the point. "return result" will put the error code >>>>>>> into >>>>>>> the socket stream and that error will be seen and responded to. I >>>>>>> don't expect anything to then read further into the stream to see the >>>>>>> "result" repeated a second time. In other words if execute() doesn't >>>>>>> see a zero result then you go no further; if execute sees a zero >>>>>>> result then the actual action may have had an issue so you proceed to >>>>>>> read the "action's result". >>>>>>> >>>>>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get >>>>>>>> -1 if >>>>>>>> we try to attach invalid agent. >>>>>>> >>>>>>> Can you please outline your test case for this again. If this is done >>>>>>> through jcmd then jcmd needs to know how to "unwrap" the information >>>>>>> it gets back from the Dcmd. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>>>>> David, >>>>>>>>> >>>>>>>>> Thank you for the evaluation. >>>>>>>>> >>>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>>> is to >>>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>>> Though I'm >>>>>>>>>> not sure how exactly. >>>>>>>>> >>>>>>>>> This logic seems completely broken for me. But I don't see >>>>>>>>> anything we >>>>>>>>> can do right now - for jdk 9. >>>>>>>>> >>>>>>>>> It requires careful changes of both - code and tests. >>>>>>>>> >>>>>>>>> I can help with this task but not today. >>>>>>>>> >>>>>>>>> -Dmitry >>>>>>>>> >>>>>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>>>>> >>>>>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>>>>> difference between what happens with your fix and what happens >>>>>>>>>>> without. >>>>>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>>>>> how we >>>>>>>>>>> get >>>>>>>>>>> AgentInitializationException instead of AgentLoadException. I >>>>>>>>>>> need >>>>>>>>>>> to be >>>>>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>>>>> very >>>>>>>>>>> difficult to arrange. :( >>>>>>>>>> >>>>>>>>>> I finally managed to connect all the pieces on this. >>>>>>>>>> >>>>>>>>>> The basic problem is that with the proposed change >>>>>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which >>>>>>>>>> then >>>>>>>>>> leads to the InternalError. Without the change we reach this >>>>>>>>>> block in >>>>>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>>>>> >>>>>>>>>> int result = readInt(in); >>>>>>>>>> if (result != 0) { >>>>>>>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>>>>>>> result); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> and so get the expected AgentInitializationException. >>>>>>>>>> >>>>>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>>>>> original: >>>>>>>>>> >>>>>>>>>> st->print_cr("%d", result); >>>>>>>>>> result = JNI_OK; >>>>>>>>>> >>>>>>>>>> then execute() manages to read a zero completion status from the >>>>>>>>>> stream, >>>>>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>>>>> whether >>>>>>>>>> zero or not - from the stream. This is because the AttachListener >>>>>>>>>> code >>>>>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>>>>> wrote >>>>>>>>>> the result value above in print_cr. Then if we look at, for >>>>>>>>>> example, >>>>>>>>>> LinuxAttachOperation::complete, we will write "result" to the >>>>>>>>>> socket >>>>>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>>>>> operation >>>>>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>>>>> read 0 >>>>>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>>>>> proposed >>>>>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>>>>> throws >>>>>>>>>> AgentLoadException. >>>>>>>>>> >>>>>>>>>> So in summary there are two different error indicators written >>>>>>>>>> into >>>>>>>>>> the >>>>>>>>>> stream, and we need the first to be zero unless some major error >>>>>>>>>> with >>>>>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>>>>> "load" >>>>>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>>>>> >>>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>>> is to >>>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>>> Though I'm >>>>>>>>>> not sure how exactly. >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>>>>> someone >>>>>>>>>>>> help me? >>>>>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>>>>> I think I see the problem. The attach framework tries to >>>>>>>>>>>>> load the >>>>>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>>>>> actually >>>>>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>>>>> through. >>>>>>>>>>>>> I'm guessing, at the moment, that the failure is due to a >>>>>>>>>>>>> missing >>>>>>>>>>>>> module related option. >>>>>>>>>>>>> >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>>>>> seems it >>>>>>>>>>>>>> may >>>>>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>>>>> ended up >>>>>>>>>>>>>> not >>>>>>>>>>>>>> touching!). >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) >>>>>>>>>>>>>> and >>>>>>>>>>>>>> see >>>>>>>>>>>>>> what >>>>>>>>>>>>>> happens? We see AgentLoadException from the code that >>>>>>>>>>>>>> internally >>>>>>>>>>>>>> tries >>>>>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception >>>>>>>>>>>>>> during >>>>>>>>>>>>>> test >>>>>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>>>>> missing in >>>>>>>>>>>>>> target VM >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>>>>> missing in >>>>>>>>>>>>>> target VM >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to >>>>>>>>>>>>>> load >>>>>>>>>>>>>> agent >>>>>>>>>>>>>> library >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>>>>>>> when >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>>>>> fixed >>>>>>>>>>>>>>> in 9 >>>>>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>>>>> >>>>>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>>>>> appropriate >>>>>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>>>>> problem - >>>>>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>>>>> unrelated >>>>>>>>>>>>>> issue in our testing environment. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Unfortunately I have some high priority tasks to handle right >>>>>>>>>>>>>> now, >>>>>>>>>>>>>> so I >>>>>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>>>>> >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>>>>> For the record it looks like the tests involved are >>>>>>>>>>>>>>>> broken, in >>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>> because the VM used to lie about the success of attaching >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> agent, the >>>>>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Harold >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>> >>> From yasuenag at gmail.com Tue Oct 25 09:29:45 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 25 Oct 2016 18:29:45 +0900 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <1af531b8-fc35-bc25-d1c8-1688517399ac@oracle.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> <1af531b8-fc35-bc25-d1c8-1688517399ac@oracle.com> Message-ID: <8878200f-c683-cd64-2ae7-8922d1f1d581@gmail.com> Hi Dmitry, > It's hard to fix this issue without access to Oracle testing infrastructure. Agree. I'm not an Oracle employee. Thus I cannot access it. Should I change the Assignee on JBS (JDK-8165869) to you? Thanks, Yasumasa On 2016/10/25 15:09, Dmitry Samersoff wrote: > Yasumasa, > > It's hard to fix this issue without access to Oracle testing infrastructure. > > I'm working on the issue but unfortunately I was caught by some internal > problems so it progresses slowly than I expected. > > I'm sorry about it. > > -Dmitry > > > On 2016-10-24 13:24, Yasumasa Suenaga wrote: >> Hi Dmitry, David, >> >> I want to resume to work for this issue because this issue has been >> marked P3. >> Dmitry, do you have any idea for it? >> >> IMHO, we can fix as below: >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8165869/webrev.00/ >> >> According to David, JvmtiExport::load_agent_library() should return zero >> when the operation which is kicked by AttachListener is succeeded. >> AttachListener will write response code from >> JvmtiExport::load_agent_library() at first, and the client (e.g. >> HotSpotVirtualMachine.java) checks it. >> >> Thus I keep current implementation about return code, and I added the >> code to send error message to the client. >> It works fine with jdk/test/com/sun/tools/attach/BasicTests.java . >> >> What do you think about it? >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2016/09/15 20:08, Dmitry Samersoff wrote: >>> Yasumasa, David, >>> >>> I'm looking into this issue and come back as soon as I have a clear >>> picture. >>> >>> It takes some time. Sorry! >>> >>> -Dmitry >>> >>> On 2016-09-15 14:01, Yasumasa Suenaga wrote: >>>> Hi David, >>>> >>>> I agree the call sequence which you write in the case of "jcmd" >>>> operation. >>>> However, we should think about "load" operation in AttachListener. >>>> >>>> We can access JvmtiExport::load_agent_library() through two routes: >>>> >>>> Route "load": AttachListener -> JvmtiExport::load_agent_library() >>>> Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> >>>> DCmd::parse_and_execute() >>>> >>>> "load" sets error code (it means first data in the stream) when >>>> JvmtiExport::load_agent_library() returns JNI_ERR. >>>> OTOH "jcmd" sets error code if exception is occurred ONLY. >>>> >>>> If we try to load invalid JVMTI agent, "load" writes JNI_ERR to stream, >>>> however "jcmd" writes JNI_OK because pending exception is not available >>>> at this point. >>>> Currently, JCmd.java shows "Command executed successfully" when it >>>> cannot read the message from the socket. In case of this issue, >>>> JVMTIAgentLoadDCmd does not write any message because it cannot attach >>>> the agent. >>>> If return code which is in the top of stream should mean whether >>>> communication with AttachListener is succeeded, I think HotSpot should >>>> write error message or error code to the stream for JCmd.java . >>>> >>>> I'm not sure this email is the answer for you. >>>> Sorry for my English. >>>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2016/09/15 18:43, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>>>>> Hi, >>>>>> >>>>>>> If this is done through jcmd then jcmd needs to know how to "unwrap" >>>>>>> the information it gets back from the Dcmd. >>>>>> >>>>>> In case of JVMTI.agent_load dcmd, I think we should be able to get the >>>>>> response as below: >>>>>> >>>>>> 1. Invalid JVMTI agent >>>>>> 2. Agent_OnAttach() did not return JNI_OK >>>>>> 3. Succeeded >>>>>> >>>>>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, >>>>>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>>>>> occurred (in Agent_OnAttach()). >>>>> >>>>> Can you respond to my comment in the bug report about the chain of >>>>> calls and explain exactly where you think things are going wrong in >>>>> this scenario. I'm having a lot of trouble trying to see how the >>>>> information flows back through the layers. >>>>> >>>>>> IMHO, HotSpot should return error code (in top of the stream: >>>>>> written by >>>>>> AttachListener at first). So we should be able to get some error >>>>>> code in >>>>>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>>>>> parse text information in the stream. >>>>> >>>>> It returns the high-level response code first "did communication with >>>>> the target VM succeed", then the actual action response code. That >>>>> seems the right thing to do to me. It is up to the callers reading the >>>>> stream to understand how to read it. To me the issue lies somewhere on >>>>> the jcmd/dcmd side. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> >>>>>> Sorry for my English. >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2016/09/14 7:03, David Holmes wrote: >>>>>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>>>>> Thanks David! >>>>>>>> If we should not change the meaning of return code from >>>>>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>>>>> below: >>>>>>>> ------------------- >>>>>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>>>>>>> +0000 >>>>>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>>>>>>> +0900 >>>>>>>> @@ -2412,6 +2412,10 @@ >>>>>>>> result = JNI_OK; >>>>>>>> } >>>>>>>> } >>>>>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>>>>> + if (result != JNI_OK) { >>>>>>>> + st->print_cr("%d", result); >>>>>>>> + } >>>>>>>> return result; >>>>>>>> } >>>>>>>> ------------------- >>>>>>> >>>>>>> Not sure I see the point. "return result" will put the error code >>>>>>> into >>>>>>> the socket stream and that error will be seen and responded to. I >>>>>>> don't expect anything to then read further into the stream to see the >>>>>>> "result" repeated a second time. In other words if execute() doesn't >>>>>>> see a zero result then you go no further; if execute sees a zero >>>>>>> result then the actual action may have had an issue so you proceed to >>>>>>> read the "action's result". >>>>>>> >>>>>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get >>>>>>>> -1 if >>>>>>>> we try to attach invalid agent. >>>>>>> >>>>>>> Can you please outline your test case for this again. If this is done >>>>>>> through jcmd then jcmd needs to know how to "unwrap" the information >>>>>>> it gets back from the Dcmd. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>>>>> David, >>>>>>>>> >>>>>>>>> Thank you for the evaluation. >>>>>>>>> >>>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>>> is to >>>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>>> Though I'm >>>>>>>>>> not sure how exactly. >>>>>>>>> >>>>>>>>> This logic seems completely broken for me. But I don't see >>>>>>>>> anything we >>>>>>>>> can do right now - for jdk 9. >>>>>>>>> >>>>>>>>> It requires careful changes of both - code and tests. >>>>>>>>> >>>>>>>>> I can help with this task but not today. >>>>>>>>> >>>>>>>>> -Dmitry >>>>>>>>> >>>>>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>>>>> >>>>>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>>>>> difference between what happens with your fix and what happens >>>>>>>>>>> without. >>>>>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>>>>> how we >>>>>>>>>>> get >>>>>>>>>>> AgentInitializationException instead of AgentLoadException. I >>>>>>>>>>> need >>>>>>>>>>> to be >>>>>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>>>>> very >>>>>>>>>>> difficult to arrange. :( >>>>>>>>>> >>>>>>>>>> I finally managed to connect all the pieces on this. >>>>>>>>>> >>>>>>>>>> The basic problem is that with the proposed change >>>>>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which >>>>>>>>>> then >>>>>>>>>> leads to the InternalError. Without the change we reach this >>>>>>>>>> block in >>>>>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>>>>> >>>>>>>>>> int result = readInt(in); >>>>>>>>>> if (result != 0) { >>>>>>>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>>>>>>> result); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> and so get the expected AgentInitializationException. >>>>>>>>>> >>>>>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>>>>> original: >>>>>>>>>> >>>>>>>>>> st->print_cr("%d", result); >>>>>>>>>> result = JNI_OK; >>>>>>>>>> >>>>>>>>>> then execute() manages to read a zero completion status from the >>>>>>>>>> stream, >>>>>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>>>>> whether >>>>>>>>>> zero or not - from the stream. This is because the AttachListener >>>>>>>>>> code >>>>>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>>>>> wrote >>>>>>>>>> the result value above in print_cr. Then if we look at, for >>>>>>>>>> example, >>>>>>>>>> LinuxAttachOperation::complete, we will write "result" to the >>>>>>>>>> socket >>>>>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>>>>> operation >>>>>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>>>>> read 0 >>>>>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>>>>> proposed >>>>>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>>>>> throws >>>>>>>>>> AgentLoadException. >>>>>>>>>> >>>>>>>>>> So in summary there are two different error indicators written >>>>>>>>>> into >>>>>>>>>> the >>>>>>>>>> stream, and we need the first to be zero unless some major error >>>>>>>>>> with >>>>>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>>>>> "load" >>>>>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>>>>> >>>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>>> is to >>>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>>> Though I'm >>>>>>>>>> not sure how exactly. >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>>>>> someone >>>>>>>>>>>> help me? >>>>>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>>>>> I think I see the problem. The attach framework tries to >>>>>>>>>>>>> load the >>>>>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>>>>> actually >>>>>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>>>>> through. >>>>>>>>>>>>> I'm guessing, at the moment, that the failure is due to a >>>>>>>>>>>>> missing >>>>>>>>>>>>> module related option. >>>>>>>>>>>>> >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>>>>> seems it >>>>>>>>>>>>>> may >>>>>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>>>>> ended up >>>>>>>>>>>>>> not >>>>>>>>>>>>>> touching!). >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) >>>>>>>>>>>>>> and >>>>>>>>>>>>>> see >>>>>>>>>>>>>> what >>>>>>>>>>>>>> happens? We see AgentLoadException from the code that >>>>>>>>>>>>>> internally >>>>>>>>>>>>>> tries >>>>>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception >>>>>>>>>>>>>> during >>>>>>>>>>>>>> test >>>>>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>>>>> missing in >>>>>>>>>>>>>> target VM >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>>>>> missing in >>>>>>>>>>>>>> target VM >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to >>>>>>>>>>>>>> load >>>>>>>>>>>>>> agent >>>>>>>>>>>>>> library >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>>>>>>> when >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>>>>> fixed >>>>>>>>>>>>>>> in 9 >>>>>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>>>>> >>>>>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>>>>> appropriate >>>>>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>>>>> problem - >>>>>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>>>>> unrelated >>>>>>>>>>>>>> issue in our testing environment. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Unfortunately I have some high priority tasks to handle right >>>>>>>>>>>>>> now, >>>>>>>>>>>>>> so I >>>>>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>>>>> >>>>>>>>>>>>>> David >>>>>>>>>>>>>> ----- >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>>>>> For the record it looks like the tests involved are >>>>>>>>>>>>>>>> broken, in >>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>> because the VM used to lie about the success of attaching >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> agent, the >>>>>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Harold >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>> >>> > > From david.holmes at oracle.com Tue Oct 25 10:30:26 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 25 Oct 2016 20:30:26 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: <39e1c5cc-ec9b-4312-0bd5-8b2166549dd6@oracle.com> On 25/10/2016 7:28 PM, Yasumasa Suenaga wrote: > Hi David, > >> The completion status will be 0 because we did manage to communicate >> with the target VM and process the DCmd request. It passes sis back up >> to the caller. So looking at JCmd.java I see: >> >> try (InputStream in = hvm.executeJCmd(line);) { >> // read to EOF and just print output >> byte b[] = new byte[256]; >> int n; >> boolean messagePrinted = false; >> do { >> n = in.read(b); >> if (n > 0) { >> String s = new String(b, 0, n, "UTF-8"); >> System.out.print(s); >> messagePrinted = true; >> } >> } while (n > 0); >> if (!messagePrinted) { >> System.out.println("Command executed successfully"); >> } >> } >> >> and that logic, AFAICS, is not parsing out a return code followed by >> an error message! > > IMHO, JCmd#executeCommandForPid() should not parse return code from DCmd > because most of DCmds will not return the code. > For example, HelpDCmd and SystemGCDcmd will not return it. I can't look at this at the moment, but there is a wire protocol here that needs to be followed, and if it isn't followed then that needs to be fixed. If all of Dcmd needs some updates then so be it. David PS. I am about to travel and will be offline till Wednesday afternoon US PST. > If we should parse return code in JCmd.java, I guess we have to add > return code to all > DCmds. IMHO, they should be fixed on another issue. > > Thanks, > > Yasumasa > > > > On 2016/10/25 12:41, David Holmes wrote: >> On 24/10/2016 8:24 PM, Yasumasa Suenaga wrote: >>> Hi Dmitry, David, >>> >>> I want to resume to work for this issue because this issue has been >>> marked P3. >>> Dmitry, do you have any idea for it? >>> >>> IMHO, we can fix as below: >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8165869/webrev.00/ >>> >>> According to David, JvmtiExport::load_agent_library() should return zero >>> when the operation which is kicked by AttachListener is succeeded. >>> AttachListener will write response code from >>> JvmtiExport::load_agent_library() at first, and the client (e.g. >>> HotSpotVirtualMachine.java) checks it. >>> >>> Thus I keep current implementation about return code, and I added the >>> code to send error message to the client. >>> It works fine with jdk/test/com/sun/tools/attach/BasicTests.java . >>> >>> What do you think about it? >> >> Sorry but I still see the problem as being in the JCmd code, as per my >> previous email on the subject. [1] >> >> David >> ----- >> >> [1] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-September/024546.html >> >> >> >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2016/09/15 20:08, Dmitry Samersoff wrote: >>>> Yasumasa, David, >>>> >>>> I'm looking into this issue and come back as soon as I have a clear >>>> picture. >>>> >>>> It takes some time. Sorry! >>>> >>>> -Dmitry >>>> >>>> On 2016-09-15 14:01, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> I agree the call sequence which you write in the case of "jcmd" >>>>> operation. >>>>> However, we should think about "load" operation in AttachListener. >>>>> >>>>> We can access JvmtiExport::load_agent_library() through two routes: >>>>> >>>>> Route "load": AttachListener -> JvmtiExport::load_agent_library() >>>>> Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> >>>>> DCmd::parse_and_execute() >>>>> >>>>> "load" sets error code (it means first data in the stream) when >>>>> JvmtiExport::load_agent_library() returns JNI_ERR. >>>>> OTOH "jcmd" sets error code if exception is occurred ONLY. >>>>> >>>>> If we try to load invalid JVMTI agent, "load" writes JNI_ERR to >>>>> stream, >>>>> however "jcmd" writes JNI_OK because pending exception is not >>>>> available >>>>> at this point. >>>>> Currently, JCmd.java shows "Command executed successfully" when it >>>>> cannot read the message from the socket. In case of this issue, >>>>> JVMTIAgentLoadDCmd does not write any message because it cannot attach >>>>> the agent. >>>>> If return code which is in the top of stream should mean whether >>>>> communication with AttachListener is succeeded, I think HotSpot should >>>>> write error message or error code to the stream for JCmd.java . >>>>> >>>>> I'm not sure this email is the answer for you. >>>>> Sorry for my English. >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2016/09/15 18:43, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>>>>>> Hi, >>>>>>> >>>>>>>> If this is done through jcmd then jcmd needs to know how to >>>>>>>> "unwrap" >>>>>>>> the information it gets back from the Dcmd. >>>>>>> >>>>>>> In case of JVMTI.agent_load dcmd, I think we should be able to >>>>>>> get the >>>>>>> response as below: >>>>>>> >>>>>>> 1. Invalid JVMTI agent >>>>>>> 2. Agent_OnAttach() did not return JNI_OK >>>>>>> 3. Succeeded >>>>>>> >>>>>>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to >>>>>>> caller, >>>>>>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>>>>>> occurred (in Agent_OnAttach()). >>>>>> >>>>>> Can you respond to my comment in the bug report about the chain of >>>>>> calls and explain exactly where you think things are going wrong in >>>>>> this scenario. I'm having a lot of trouble trying to see how the >>>>>> information flows back through the layers. >>>>>> >>>>>>> IMHO, HotSpot should return error code (in top of the stream: >>>>>>> written by >>>>>>> AttachListener at first). So we should be able to get some error >>>>>>> code in >>>>>>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>>>>>> parse text information in the stream. >>>>>> >>>>>> It returns the high-level response code first "did communication with >>>>>> the target VM succeed", then the actual action response code. That >>>>>> seems the right thing to do to me. It is up to the callers reading >>>>>> the >>>>>> stream to understand how to read it. To me the issue lies >>>>>> somewhere on >>>>>> the jcmd/dcmd side. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> >>>>>>> Sorry for my English. >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2016/09/14 7:03, David Holmes wrote: >>>>>>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>>>>>> Thanks David! >>>>>>>>> If we should not change the meaning of return code from >>>>>>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>>>>>> below: >>>>>>>>> ------------------- >>>>>>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>>>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 >>>>>>>>> 2016 >>>>>>>>> +0000 >>>>>>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 >>>>>>>>> 2016 >>>>>>>>> +0900 >>>>>>>>> @@ -2412,6 +2412,10 @@ >>>>>>>>> result = JNI_OK; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>>>>>> + if (result != JNI_OK) { >>>>>>>>> + st->print_cr("%d", result); >>>>>>>>> + } >>>>>>>>> return result; >>>>>>>>> } >>>>>>>>> ------------------- >>>>>>>> >>>>>>>> Not sure I see the point. "return result" will put the error code >>>>>>>> into >>>>>>>> the socket stream and that error will be seen and responded to. I >>>>>>>> don't expect anything to then read further into the stream to >>>>>>>> see the >>>>>>>> "result" repeated a second time. In other words if execute() >>>>>>>> doesn't >>>>>>>> see a zero result then you go no further; if execute sees a zero >>>>>>>> result then the actual action may have had an issue so you >>>>>>>> proceed to >>>>>>>> read the "action's result". >>>>>>>> >>>>>>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get >>>>>>>>> -1 if >>>>>>>>> we try to attach invalid agent. >>>>>>>> >>>>>>>> Can you please outline your test case for this again. If this is >>>>>>>> done >>>>>>>> through jcmd then jcmd needs to know how to "unwrap" the >>>>>>>> information >>>>>>>> it gets back from the Dcmd. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>>>>>> David, >>>>>>>>>> >>>>>>>>>> Thank you for the evaluation. >>>>>>>>>> >>>>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>>>> is to >>>>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>>>> Though I'm >>>>>>>>>>> not sure how exactly. >>>>>>>>>> >>>>>>>>>> This logic seems completely broken for me. But I don't see >>>>>>>>>> anything we >>>>>>>>>> can do right now - for jdk 9. >>>>>>>>>> >>>>>>>>>> It requires careful changes of both - code and tests. >>>>>>>>>> >>>>>>>>>> I can help with this task but not today. >>>>>>>>>> >>>>>>>>>> -Dmitry >>>>>>>>>> >>>>>>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>>>>>> >>>>>>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>>>>>> difference between what happens with your fix and what happens >>>>>>>>>>>> without. >>>>>>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>>>>>> how we >>>>>>>>>>>> get >>>>>>>>>>>> AgentInitializationException instead of AgentLoadException. I >>>>>>>>>>>> need >>>>>>>>>>>> to be >>>>>>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>>>>>> very >>>>>>>>>>>> difficult to arrange. :( >>>>>>>>>>> >>>>>>>>>>> I finally managed to connect all the pieces on this. >>>>>>>>>>> >>>>>>>>>>> The basic problem is that with the proposed change >>>>>>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, >>>>>>>>>>> which >>>>>>>>>>> then >>>>>>>>>>> leads to the InternalError. Without the change we reach this >>>>>>>>>>> block in >>>>>>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>>>>>> >>>>>>>>>>> int result = readInt(in); >>>>>>>>>>> if (result != 0) { >>>>>>>>>>> throw new AgentInitializationException("Agent_OnAttach >>>>>>>>>>> failed", >>>>>>>>>>> result); >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> and so get the expected AgentInitializationException. >>>>>>>>>>> >>>>>>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>>>>>> original: >>>>>>>>>>> >>>>>>>>>>> st->print_cr("%d", result); >>>>>>>>>>> result = JNI_OK; >>>>>>>>>>> >>>>>>>>>>> then execute() manages to read a zero completion status from the >>>>>>>>>>> stream, >>>>>>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>>>>>> whether >>>>>>>>>>> zero or not - from the stream. This is because the >>>>>>>>>>> AttachListener >>>>>>>>>>> code >>>>>>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>>>>>> wrote >>>>>>>>>>> the result value above in print_cr. Then if we look at, for >>>>>>>>>>> example, >>>>>>>>>>> LinuxAttachOperation::complete, we will write "result" to the >>>>>>>>>>> socket >>>>>>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>>>>>> operation >>>>>>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>>>>>> read 0 >>>>>>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>>>>>> proposed >>>>>>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>>>>>> throws >>>>>>>>>>> AgentLoadException. >>>>>>>>>>> >>>>>>>>>>> So in summary there are two different error indicators written >>>>>>>>>>> into >>>>>>>>>>> the >>>>>>>>>>> stream, and we need the first to be zero unless some major error >>>>>>>>>>> with >>>>>>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>>>>>> "load" >>>>>>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>>>>>> >>>>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>>>> is to >>>>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>>>> Though I'm >>>>>>>>>>> not sure how exactly. >>>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>>>>>> someone >>>>>>>>>>>>> help me? >>>>>>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>>>>>> I think I see the problem. The attach framework tries to >>>>>>>>>>>>>> load the >>>>>>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>>>>>> actually >>>>>>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>>>>>> through. >>>>>>>>>>>>>> I'm guessing, at the moment, that the failure is due to a >>>>>>>>>>>>>> missing >>>>>>>>>>>>>> module related option. >>>>>>>>>>>>>> >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>>>>>> seems it >>>>>>>>>>>>>>> may >>>>>>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>>>>>> ended up >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>> touching!). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>> see >>>>>>>>>>>>>>> what >>>>>>>>>>>>>>> happens? We see AgentLoadException from the code that >>>>>>>>>>>>>>> internally >>>>>>>>>>>>>>> tries >>>>>>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception >>>>>>>>>>>>>>> during >>>>>>>>>>>>>>> test >>>>>>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>>>>>> missing in >>>>>>>>>>>>>>> target VM >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>>>>>> missing in >>>>>>>>>>>>>>> target VM >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: >>>>>>>>>>>>>>> Failed to >>>>>>>>>>>>>>> load >>>>>>>>>>>>>>> agent >>>>>>>>>>>>>>> library >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM >>>>>>>>>>>>>>>> might lie >>>>>>>>>>>>>>>> when >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>>>>>> fixed >>>>>>>>>>>>>>>> in 9 >>>>>>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>>>>>> appropriate >>>>>>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>>>>>> problem - >>>>>>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>>>>>> unrelated >>>>>>>>>>>>>>> issue in our testing environment. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Unfortunately I have some high priority tasks to handle >>>>>>>>>>>>>>> right >>>>>>>>>>>>>>> now, >>>>>>>>>>>>>>> so I >>>>>>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>>>>>> For the record it looks like the tests involved are >>>>>>>>>>>>>>>>> broken, in >>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>> because the VM used to lie about the success of attaching >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> agent, the >>>>>>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Harold >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>> >>>> From dmitry.samersoff at oracle.com Tue Oct 25 10:34:02 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 25 Oct 2016 13:34:02 +0300 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <8878200f-c683-cd64-2ae7-8922d1f1d581@gmail.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> <1af531b8-fc35-bc25-d1c8-1688517399ac@oracle.com> <8878200f-c683-cd64-2ae7-8922d1f1d581@gmail.com> Message-ID: Yasumasa, > Should I change the Assignee on JBS (JDK-8165869) to you? Yes. Please do it. -Dmitry On 2016-10-25 12:29, Yasumasa Suenaga wrote: > Hi Dmitry, > >> It's hard to fix this issue without access to Oracle testing >> infrastructure. > > Agree. > I'm not an Oracle employee. Thus I cannot access it. > > Should I change the Assignee on JBS (JDK-8165869) to you? > > > Thanks, > > Yasumasa > > > On 2016/10/25 15:09, Dmitry Samersoff wrote: >> Yasumasa, >> >> It's hard to fix this issue without access to Oracle testing >> infrastructure. >> >> I'm working on the issue but unfortunately I was caught by some internal >> problems so it progresses slowly than I expected. >> >> I'm sorry about it. >> >> -Dmitry >> >> >> On 2016-10-24 13:24, Yasumasa Suenaga wrote: >>> Hi Dmitry, David, >>> >>> I want to resume to work for this issue because this issue has been >>> marked P3. >>> Dmitry, do you have any idea for it? >>> >>> IMHO, we can fix as below: >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8165869/webrev.00/ >>> >>> According to David, JvmtiExport::load_agent_library() should return zero >>> when the operation which is kicked by AttachListener is succeeded. >>> AttachListener will write response code from >>> JvmtiExport::load_agent_library() at first, and the client (e.g. >>> HotSpotVirtualMachine.java) checks it. >>> >>> Thus I keep current implementation about return code, and I added the >>> code to send error message to the client. >>> It works fine with jdk/test/com/sun/tools/attach/BasicTests.java . >>> >>> What do you think about it? >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2016/09/15 20:08, Dmitry Samersoff wrote: >>>> Yasumasa, David, >>>> >>>> I'm looking into this issue and come back as soon as I have a clear >>>> picture. >>>> >>>> It takes some time. Sorry! >>>> >>>> -Dmitry >>>> >>>> On 2016-09-15 14:01, Yasumasa Suenaga wrote: >>>>> Hi David, >>>>> >>>>> I agree the call sequence which you write in the case of "jcmd" >>>>> operation. >>>>> However, we should think about "load" operation in AttachListener. >>>>> >>>>> We can access JvmtiExport::load_agent_library() through two routes: >>>>> >>>>> Route "load": AttachListener -> JvmtiExport::load_agent_library() >>>>> Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> >>>>> DCmd::parse_and_execute() >>>>> >>>>> "load" sets error code (it means first data in the stream) when >>>>> JvmtiExport::load_agent_library() returns JNI_ERR. >>>>> OTOH "jcmd" sets error code if exception is occurred ONLY. >>>>> >>>>> If we try to load invalid JVMTI agent, "load" writes JNI_ERR to >>>>> stream, >>>>> however "jcmd" writes JNI_OK because pending exception is not >>>>> available >>>>> at this point. >>>>> Currently, JCmd.java shows "Command executed successfully" when it >>>>> cannot read the message from the socket. In case of this issue, >>>>> JVMTIAgentLoadDCmd does not write any message because it cannot attach >>>>> the agent. >>>>> If return code which is in the top of stream should mean whether >>>>> communication with AttachListener is succeeded, I think HotSpot should >>>>> write error message or error code to the stream for JCmd.java . >>>>> >>>>> I'm not sure this email is the answer for you. >>>>> Sorry for my English. >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2016/09/15 18:43, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>>>>>> Hi, >>>>>>> >>>>>>>> If this is done through jcmd then jcmd needs to know how to >>>>>>>> "unwrap" >>>>>>>> the information it gets back from the Dcmd. >>>>>>> >>>>>>> In case of JVMTI.agent_load dcmd, I think we should be able to >>>>>>> get the >>>>>>> response as below: >>>>>>> >>>>>>> 1. Invalid JVMTI agent >>>>>>> 2. Agent_OnAttach() did not return JNI_OK >>>>>>> 3. Succeeded >>>>>>> >>>>>>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to >>>>>>> caller, >>>>>>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>>>>>> occurred (in Agent_OnAttach()). >>>>>> >>>>>> Can you respond to my comment in the bug report about the chain of >>>>>> calls and explain exactly where you think things are going wrong in >>>>>> this scenario. I'm having a lot of trouble trying to see how the >>>>>> information flows back through the layers. >>>>>> >>>>>>> IMHO, HotSpot should return error code (in top of the stream: >>>>>>> written by >>>>>>> AttachListener at first). So we should be able to get some error >>>>>>> code in >>>>>>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>>>>>> parse text information in the stream. >>>>>> >>>>>> It returns the high-level response code first "did communication with >>>>>> the target VM succeed", then the actual action response code. That >>>>>> seems the right thing to do to me. It is up to the callers reading >>>>>> the >>>>>> stream to understand how to read it. To me the issue lies >>>>>> somewhere on >>>>>> the jcmd/dcmd side. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> >>>>>>> Sorry for my English. >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2016/09/14 7:03, David Holmes wrote: >>>>>>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>>>>>> Thanks David! >>>>>>>>> If we should not change the meaning of return code from >>>>>>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>>>>>> below: >>>>>>>>> ------------------- >>>>>>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>>>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 >>>>>>>>> 2016 >>>>>>>>> +0000 >>>>>>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 >>>>>>>>> 2016 >>>>>>>>> +0900 >>>>>>>>> @@ -2412,6 +2412,10 @@ >>>>>>>>> result = JNI_OK; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>>>>>> + if (result != JNI_OK) { >>>>>>>>> + st->print_cr("%d", result); >>>>>>>>> + } >>>>>>>>> return result; >>>>>>>>> } >>>>>>>>> ------------------- >>>>>>>> >>>>>>>> Not sure I see the point. "return result" will put the error code >>>>>>>> into >>>>>>>> the socket stream and that error will be seen and responded to. I >>>>>>>> don't expect anything to then read further into the stream to >>>>>>>> see the >>>>>>>> "result" repeated a second time. In other words if execute() >>>>>>>> doesn't >>>>>>>> see a zero result then you go no further; if execute sees a zero >>>>>>>> result then the actual action may have had an issue so you >>>>>>>> proceed to >>>>>>>> read the "action's result". >>>>>>>> >>>>>>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get >>>>>>>>> -1 if >>>>>>>>> we try to attach invalid agent. >>>>>>>> >>>>>>>> Can you please outline your test case for this again. If this is >>>>>>>> done >>>>>>>> through jcmd then jcmd needs to know how to "unwrap" the >>>>>>>> information >>>>>>>> it gets back from the Dcmd. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>>>>>> David, >>>>>>>>>> >>>>>>>>>> Thank you for the evaluation. >>>>>>>>>> >>>>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>>>> is to >>>>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>>>> Though I'm >>>>>>>>>>> not sure how exactly. >>>>>>>>>> >>>>>>>>>> This logic seems completely broken for me. But I don't see >>>>>>>>>> anything we >>>>>>>>>> can do right now - for jdk 9. >>>>>>>>>> >>>>>>>>>> It requires careful changes of both - code and tests. >>>>>>>>>> >>>>>>>>>> I can help with this task but not today. >>>>>>>>>> >>>>>>>>>> -Dmitry >>>>>>>>>> >>>>>>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>>>>>> >>>>>>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>>>>>> difference between what happens with your fix and what happens >>>>>>>>>>>> without. >>>>>>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>>>>>> how we >>>>>>>>>>>> get >>>>>>>>>>>> AgentInitializationException instead of AgentLoadException. I >>>>>>>>>>>> need >>>>>>>>>>>> to be >>>>>>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>>>>>> very >>>>>>>>>>>> difficult to arrange. :( >>>>>>>>>>> >>>>>>>>>>> I finally managed to connect all the pieces on this. >>>>>>>>>>> >>>>>>>>>>> The basic problem is that with the proposed change >>>>>>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, >>>>>>>>>>> which >>>>>>>>>>> then >>>>>>>>>>> leads to the InternalError. Without the change we reach this >>>>>>>>>>> block in >>>>>>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>>>>>> >>>>>>>>>>> int result = readInt(in); >>>>>>>>>>> if (result != 0) { >>>>>>>>>>> throw new AgentInitializationException("Agent_OnAttach >>>>>>>>>>> failed", >>>>>>>>>>> result); >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> and so get the expected AgentInitializationException. >>>>>>>>>>> >>>>>>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>>>>>> original: >>>>>>>>>>> >>>>>>>>>>> st->print_cr("%d", result); >>>>>>>>>>> result = JNI_OK; >>>>>>>>>>> >>>>>>>>>>> then execute() manages to read a zero completion status from the >>>>>>>>>>> stream, >>>>>>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>>>>>> whether >>>>>>>>>>> zero or not - from the stream. This is because the >>>>>>>>>>> AttachListener >>>>>>>>>>> code >>>>>>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>>>>>> wrote >>>>>>>>>>> the result value above in print_cr. Then if we look at, for >>>>>>>>>>> example, >>>>>>>>>>> LinuxAttachOperation::complete, we will write "result" to the >>>>>>>>>>> socket >>>>>>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>>>>>> operation >>>>>>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>>>>>> read 0 >>>>>>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>>>>>> proposed >>>>>>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>>>>>> throws >>>>>>>>>>> AgentLoadException. >>>>>>>>>>> >>>>>>>>>>> So in summary there are two different error indicators written >>>>>>>>>>> into >>>>>>>>>>> the >>>>>>>>>>> stream, and we need the first to be zero unless some major error >>>>>>>>>>> with >>>>>>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>>>>>> "load" >>>>>>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>>>>>> >>>>>>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>>>>>> is to >>>>>>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>>>>>> Though I'm >>>>>>>>>>> not sure how exactly. >>>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>>>>>> someone >>>>>>>>>>>>> help me? >>>>>>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>>>>>> I think I see the problem. The attach framework tries to >>>>>>>>>>>>>> load the >>>>>>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>>>>>> actually >>>>>>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>>>>>> through. >>>>>>>>>>>>>> I'm guessing, at the moment, that the failure is due to a >>>>>>>>>>>>>> missing >>>>>>>>>>>>>> module related option. >>>>>>>>>>>>>> >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>>>>>> seems it >>>>>>>>>>>>>>> may >>>>>>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>>>>>> ended up >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>> touching!). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) >>>>>>>>>>>>>>> and >>>>>>>>>>>>>>> see >>>>>>>>>>>>>>> what >>>>>>>>>>>>>>> happens? We see AgentLoadException from the code that >>>>>>>>>>>>>>> internally >>>>>>>>>>>>>>> tries >>>>>>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception >>>>>>>>>>>>>>> during >>>>>>>>>>>>>>> test >>>>>>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>>>>>> missing in >>>>>>>>>>>>>>> target VM >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>>>>>> missing in >>>>>>>>>>>>>>> target VM >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: >>>>>>>>>>>>>>> Failed to >>>>>>>>>>>>>>> load >>>>>>>>>>>>>>> agent >>>>>>>>>>>>>>> library >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM >>>>>>>>>>>>>>>> might lie >>>>>>>>>>>>>>>> when >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>>>>>> fixed >>>>>>>>>>>>>>>> in 9 >>>>>>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>>>>>> appropriate >>>>>>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>>>>>> problem - >>>>>>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>>>>>> unrelated >>>>>>>>>>>>>>> issue in our testing environment. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Unfortunately I have some high priority tasks to handle >>>>>>>>>>>>>>> right >>>>>>>>>>>>>>> now, >>>>>>>>>>>>>>> so I >>>>>>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>>>>>> For the record it looks like the tests involved are >>>>>>>>>>>>>>>>> broken, in >>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>> because the VM used to lie about the success of attaching >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> agent, the >>>>>>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Harold >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>> >>>> >> >> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From coleen.phillimore at oracle.com Tue Oct 25 12:01:11 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Oct 2016 08:01:11 -0400 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: References: <20161013121510.GM19291@ehelin.jrpg.bea.com> <20161024140105.GE19291@ehelin.jrpg.bea.com> <6482d642-428a-0e53-c8da-d61de968defe@oracle.com> Message-ID: <07350a4d-ee7a-2971-0050-707445ff550c@oracle.com> On 10/25/16 3:39 AM, Thomas St?fe wrote: > Hi Coleen, > > thank you for feedback and encouragement :) See further comments inline. > > On Mon, Oct 24, 2016 at 6:32 PM, Coleen Phillimore > > > wrote: > > > Hi Thomas, > > I agree with Erik. If this works well for you, then it should > just be implemented without an option. If done early in JDK10, > it'll get a lot of good testing. > > > Ok. Was hoping to get this into JDK9, but if I am thinking clearly > about this, I see that maybe the risk is too large. So lets implement > this in 10 and if it is stable and works well, it can be backported to > jdk9, yes? That seems a better plan. Even though JDK9 slipped for jigsaw, we are keeping to the original FC date for everything else, so that it'll be very stable when it ships. JDK10 should open soon. > > > This looks like a very good improvement. We had discussed > coalescing blocks and other improvements like this early on, but > wanted to wait to see problems in the field to motivate the > changes. We've seen these sorts of problems now too. > > One of the things we've considered is that we wanted to use the > operating system's version of malloc for the chunks, so that it > can split and coalesce chunks for us. The Solaris malloc is > improved recently. But I don't think it's time to make change to > use malloc'ed chunks yet because we have to consider all of the > operating systems that we and the OpenJDK community supports. > > > I do not see how you could get CompressedClassPointers to work with > native malloc? You would have to be sure that the pointers returned by > malloc are within the numerical range for the 32bit class pointers. I > thought that was the reason for using a continuous address range when > allocating compressed class space. Oh, yes, you are right. I thought we saw the too many small chunks problem only in the data metaspace, but this would help with class metaspace as well. Thanks, Coleen > > So, yes, I think the JEP looks good and your slides are absolutely > beautiful. Everyone should see these slides. > > > :) Thanks! > > So I will provide a prototype, for now based on jdk9, and we will see > how we go from there. > > Thanks, and Kind Regards, > > Thomas > > Thanks, > Coleen > > > > On 10/24/16 10:01 AM, Erik Helin wrote: > > On 2016-10-13, Thomas St?fe wrote: > > Hi Erik, > > On Thu, Oct 13, 2016 at 2:15 PM, Erik Helin > > wrote: > > Hi Thomas, > > thanks for submitting the JEP and proposing this feature! > > On 2016-10-10, Thomas St?fe wrote: > > Hi all, > > May I have please some feedback for this > enhancement proposal? > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > > In one very short sentence it proposes a better > allocation scheme for > Metaspace Chunks in order to reduce fragmentation > and metaspace waste. > > I also added a short presentation which describes > the problem and how we > solved it in our VM port. > > https://bugs.openjdk.java.net/secure/attachment/63894/ > > > Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf > > Do we really need the flag -XX:+CoalesceMetaspace? > Having two differnent > ways to handle the chunk free lists in Metaspace is > unfortunate, it > might introduce hard to detect bugs and will also > require much more > testing (runnings lots of tests with the flag both on > and off). > > You are right. If the new allocator works well, there is > no reason to keep > the old allocator around. > > We wanted for a temporary time to be able to switch > between both old and > new allocator. Just to have a fallback if problems occur. > But if it works, > it makes sense to only have one allocator, and the > "CoalesceMetaspace" flag > can be removed, and also the code can be made a lot > simpler because we do > not need both code paths. > > Yeah, I would strongly prefer to not introduce a new flag for > this. Have > you thought about testing? Do you intend to write new tests to > stress > the coalescing? > > Do you think your proposed solution has low enough > overhead (in terms > of CPU and memory) to be on "by default"? > > We decided to switch it on by default in our VM. > > Memory overhead can be almost exactly calculated. Bitmasks > take 2 bits per > specialized-chunk-sized-area. That means, for > specialized-chunk-size = 1k > (128 meta words): metaspace size / 8192. So, for 1G of > metaspace we pay > 132KB overhead for the bitmasks, or roughly 0.1%. > > There is some CPU overhead, but in my tests I could not > measure anything > above noise level. > > Those numbers seems low enough to me in order to not warrant a > new flag. > > Thanks, > Erik > > > Btw, I understand that it is difficult to estimate this > proposal without a > prototype to play around. As I already mentioned, the > patch right now only > exists in our code base and not yet in the OpenJDK. If you > guys are > seriously interested in this JEP, I will invest the time > to port the patch > to the OpenJDK, so that you can check it out for yourself. > > Yes, we are seriously interested :) I think the proposal > sounds good. I guess > the devil will be in the details, so I (we) would really > appreciate if > you want to port your internal patch to OpenJDK. > > Thanks, > Erik > > Kind Regards, Thomas > > > > > Thank you very much! > > Kind Regards, Thomas > > > On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe > > > wrote: > > Dear all, > > please take a look at this Enhancement > Proposal for the metaspace > allocator. I hope these are the right groups > for this discussion. > > https://bugs.openjdk.java.net/browse/JDK-8166690 > > > Background: > > We at SAP see at times at customer > installations OOMs in Metaspace > (usually, with compressed class pointers > enabled, in Compressed Class > Space). The VM attempts to allocate metaspace > and fails, hitting the > CompressedClassSpaceSize limit. Note that we > usually set the limit > > lower > > than the default, typically at 256M. > > When analyzing, we observed that a large part > of the metaspace is > > indeed > > free but "locked in" into metaspace chunks of > the wrong size: often we > would find a lot of free small chunks, but the > allocation request was > > for > > medium chunks, and failed. > > The reason was that if at some point in time a > lot of class loaders > > were > > alive, each with only a few small classes > loaded. This would lead to > > the > > metaspace being swamped with lots of small > chunks. This is because each > SpaceManager first allocates small chunks, > only after a certain amount > > of > > allocation requests switches to larger chunks. > > These small chunks are free and wait in the > freelist, but cannot be > > reused > > for allocation requests which require larger > chunks, even if they are > physically adjacent in the virtual space. > > We (at SAP) added a patch which allows > on-the-fly metaspace chunk > > merging > > - to merge multiple adjacent smaller chunk to > form a larger chunk. > > This, in > > combination with the reverse direction - > splitting a large chunk to get > smaller chunks - partly negates the > "chunks-are-locked-in-into- > > their-size" > > limitation and provides for better reuse of > metaspace chunks. It also > provides better defragmentation as well. > > I discussed this fix off-list with Coleen > Phillimore and Jon Masamitsu, > and instead of just offering this as a fix, > both recommended to open a > > JEP > > for this, because its scope would be beyond > that of a simple fix. > > So here is my first JEP :) I hope it follows > the right form. Please, if > you have time, take a look and tell us what > you think. > > Thank you, and Kind Regards, > > Thomas St?fe > > > > > > From tobias.hartmann at oracle.com Tue Oct 25 12:43:07 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 25 Oct 2016 14:43:07 +0200 Subject: [9] RFR(S): 8164612: NoSuchMethodException when method name contains NULL or Latin-1 supplement character In-Reply-To: <5808692E.9090905@oracle.com> References: <580622AE.9080802@oracle.com> <5808692E.9090905@oracle.com> Message-ID: <580F535B.8040205@oracle.com> [Ping] As Coleen requested, I executed the JCK/VM tests (see comment in bug). Best regards, Tobias On 20.10.2016 08:50, Tobias Hartmann wrote: > Hi, > > since this is affecting runtime code, could someone from the runtime team please have a look as well? > > Thanks, > Tobias > > On 18.10.2016 15:25, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8164612 >> http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/ >> >> The test executes Java Script code that defines getter methods containing Latin-1 supplement characters (0x80 - 0xFF). Those methods are registered at runtime through anonymous classes via Unsafe_DefineAnonymousClass. When calling a method, the VM fails with a NoSuchMethodException in MethodHandles::resolve_MemberName(). >> >> The failure happens while looking up the method name symbol in java_lang_String::as_symbol_or_null() [1]: >> 544 jbyte* position = (length == 0) ? NULL : value->byte_at_addr(0); >> 545 const char* base = UNICODE::as_utf8(position, length); >> 546 return SymbolTable::probe(base, length); >> >> If Compact Strings is enabled, we pass the Latin-1 encoded method name to UNICODE::as_utf8() and probe for the UTF-8 String in the SymbolTable. Since the Latin-1 method name contains non-ASCII characters, the length of the resulting UTF-8 String is larger (characters >= 0x80 are encoded as two bytes in UTF-8). However, we pass the shorter Latin-1 length to SymbolTable::probe() resulting in a lookup failure. >> >> I fixed this by passing the String length by reference to UNICODE::as_utf8(). I also refactored the related code in utf8.cpp, added comments and updated the callers. >> >> Tested with regression test and hs-comp PIT RBT (running). >> >> Thanks, >> Tobias, >> >> [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/652537a80080/src/share/vm/classfile/javaClasses.cpp#l535 >> From thomas.stuefe at gmail.com Tue Oct 25 13:21:17 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Oct 2016 15:21:17 +0200 Subject: PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. In-Reply-To: <07350a4d-ee7a-2971-0050-707445ff550c@oracle.com> References: <20161013121510.GM19291@ehelin.jrpg.bea.com> <20161024140105.GE19291@ehelin.jrpg.bea.com> <6482d642-428a-0e53-c8da-d61de968defe@oracle.com> <07350a4d-ee7a-2971-0050-707445ff550c@oracle.com> Message-ID: On Tue, Oct 25, 2016 at 2:01 PM, Coleen Phillimore < coleen.phillimore at oracle.com> wrote: > > > On 10/25/16 3:39 AM, Thomas St?fe wrote: > > Hi Coleen, > > thank you for feedback and encouragement :) See further comments inline. > > On Mon, Oct 24, 2016 at 6:32 PM, Coleen Phillimore < > coleen.phillimore at oracle.com> wrote: > >> >> Hi Thomas, >> >> I agree with Erik. If this works well for you, then it should just be >> implemented without an option. If done early in JDK10, it'll get a lot of >> good testing. >> > > Ok. Was hoping to get this into JDK9, but if I am thinking clearly about > this, I see that maybe the risk is too large. So lets implement this in 10 > and if it is stable and works well, it can be backported to jdk9, yes? > > > That seems a better plan. Even though JDK9 slipped for jigsaw, we are > keeping to the original FC date for everything else, so that it'll be very > stable when it ships. > > JDK10 should open soon. > > > >> >> This looks like a very good improvement. We had discussed coalescing >> blocks and other improvements like this early on, but wanted to wait to see >> problems in the field to motivate the changes. We've seen these sorts of >> problems now too. >> >> One of the things we've considered is that we wanted to use the operating >> system's version of malloc for the chunks, so that it can split and >> coalesce chunks for us. The Solaris malloc is improved recently. But I >> don't think it's time to make change to use malloc'ed chunks yet because we >> have to consider all of the operating systems that we and the OpenJDK >> community supports. >> > > I do not see how you could get CompressedClassPointers to work with native > malloc? You would have to be sure that the pointers returned by malloc are > within the numerical range for the 32bit class pointers. I thought that was > the reason for using a continuous address range when allocating compressed > class space. > > > Oh, yes, you are right. I thought we saw the too many small chunks > problem only in the data metaspace, but this would help with class > metaspace as well. > > Actually, we saw the problem mostly in compressed class space. The problem is the same on both sides, but customers hit CompressedClassSpaceSize more often, if only because MaxMetaspaceSize is often left unlimited. I'm not sure how to proceed with the JEP - it is still in draft state, should I submit it or leave it as draft until the prototype is done? Thanks, Thomas Thanks, > Coleen > > > > >> >> So, yes, I think the JEP looks good and your slides are absolutely >> beautiful. Everyone should see these slides. >> >> > :) Thanks! > > So I will provide a prototype, for now based on jdk9, and we will see how > we go from there. > > Thanks, and Kind Regards, > > Thomas > > >> Thanks, >> Coleen >> >> >> >> On 10/24/16 10:01 AM, Erik Helin wrote: >> >>> On 2016-10-13, Thomas St?fe wrote: >>> >>>> Hi Erik, >>>> >>>> On Thu, Oct 13, 2016 at 2:15 PM, Erik Helin >>>> wrote: >>>> >>>> Hi Thomas, >>>>> >>>>> thanks for submitting the JEP and proposing this feature! >>>>> >>>>> On 2016-10-10, Thomas St?fe wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> May I have please some feedback for this enhancement proposal? >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166690 >>>>>> >>>>>> >>>>>> In one very short sentence it proposes a better allocation scheme for >>>>>> Metaspace Chunks in order to reduce fragmentation and metaspace waste. >>>>>> >>>>>> I also added a short presentation which describes the problem and how >>>>>> we >>>>>> solved it in our VM port. >>>>>> >>>>>> https://bugs.openjdk.java.net/secure/attachment/63894/ >>>>>> >>>>> Metaspace%20Coalescation%20in%20the%20SAP%20JVM.pdf >>>>> >>>>> Do we really need the flag -XX:+CoalesceMetaspace? Having two >>>>> differnent >>>>> ways to handle the chunk free lists in Metaspace is unfortunate, it >>>>> might introduce hard to detect bugs and will also require much more >>>>> testing (runnings lots of tests with the flag both on and off). >>>>> >>>>> You are right. If the new allocator works well, there is no reason to >>>> keep >>>> the old allocator around. >>>> >>>> We wanted for a temporary time to be able to switch between both old and >>>> new allocator. Just to have a fallback if problems occur. But if it >>>> works, >>>> it makes sense to only have one allocator, and the "CoalesceMetaspace" >>>> flag >>>> can be removed, and also the code can be made a lot simpler because we >>>> do >>>> not need both code paths. >>>> >>> Yeah, I would strongly prefer to not introduce a new flag for this. Have >>> you thought about testing? Do you intend to write new tests to stress >>> the coalescing? >>> >>> Do you think your proposed solution has low enough overhead (in terms >>>>> of CPU and memory) to be on "by default"? >>>>> >>>>> We decided to switch it on by default in our VM. >>>> >>>> Memory overhead can be almost exactly calculated. Bitmasks take 2 bits >>>> per >>>> specialized-chunk-sized-area. That means, for specialized-chunk-size = >>>> 1k >>>> (128 meta words): metaspace size / 8192. So, for 1G of metaspace we pay >>>> 132KB overhead for the bitmasks, or roughly 0.1%. >>>> >>>> There is some CPU overhead, but in my tests I could not measure anything >>>> above noise level. >>>> >>> Those numbers seems low enough to me in order to not warrant a new flag. >>> >>> Thanks, >>>>> Erik >>>>> >>>>> >>>>> Btw, I understand that it is difficult to estimate this proposal >>>> without a >>>> prototype to play around. As I already mentioned, the patch right now >>>> only >>>> exists in our code base and not yet in the OpenJDK. If you guys are >>>> seriously interested in this JEP, I will invest the time to port the >>>> patch >>>> to the OpenJDK, so that you can check it out for yourself. >>>> >>> Yes, we are seriously interested :) I think the proposal sounds good. I >>> guess >>> the devil will be in the details, so I (we) would really appreciate if >>> you want to port your internal patch to OpenJDK. >>> >>> Thanks, >>> Erik >>> >>> Kind Regards, Thomas >>>> >>>> >>>> >>>> >>>> Thank you very much! >>>>>> >>>>>> Kind Regards, Thomas >>>>>> >>>>>> >>>>>> On Tue, Sep 27, 2016 at 10:45 AM, Thomas St?fe < >>>>>> thomas.stuefe at gmail.com> >>>>>> wrote: >>>>>> >>>>>> Dear all, >>>>>>> >>>>>>> please take a look at this Enhancement Proposal for the metaspace >>>>>>> allocator. I hope these are the right groups for this discussion. >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166690 >>>>>>> >>>>>>> Background: >>>>>>> >>>>>>> We at SAP see at times at customer installations OOMs in Metaspace >>>>>>> (usually, with compressed class pointers enabled, in Compressed Class >>>>>>> Space). The VM attempts to allocate metaspace and fails, hitting the >>>>>>> CompressedClassSpaceSize limit. Note that we usually set the limit >>>>>>> >>>>>> lower >>>>> >>>>>> than the default, typically at 256M. >>>>>>> >>>>>>> When analyzing, we observed that a large part of the metaspace is >>>>>>> >>>>>> indeed >>>>> >>>>>> free but "locked in" into metaspace chunks of the wrong size: often we >>>>>>> would find a lot of free small chunks, but the allocation request was >>>>>>> >>>>>> for >>>>> >>>>>> medium chunks, and failed. >>>>>>> >>>>>>> The reason was that if at some point in time a lot of class loaders >>>>>>> >>>>>> were >>>>> >>>>>> alive, each with only a few small classes loaded. This would lead to >>>>>>> >>>>>> the >>>>> >>>>>> metaspace being swamped with lots of small chunks. This is because >>>>>>> each >>>>>>> SpaceManager first allocates small chunks, only after a certain >>>>>>> amount >>>>>>> >>>>>> of >>>>> >>>>>> allocation requests switches to larger chunks. >>>>>>> >>>>>>> These small chunks are free and wait in the freelist, but cannot be >>>>>>> >>>>>> reused >>>>> >>>>>> for allocation requests which require larger chunks, even if they are >>>>>>> physically adjacent in the virtual space. >>>>>>> >>>>>>> We (at SAP) added a patch which allows on-the-fly metaspace chunk >>>>>>> >>>>>> merging >>>>> >>>>>> - to merge multiple adjacent smaller chunk to form a larger chunk. >>>>>>> >>>>>> This, in >>>>> >>>>>> combination with the reverse direction - splitting a large chunk to >>>>>>> get >>>>>>> smaller chunks - partly negates the "chunks-are-locked-in-into- >>>>>>> >>>>>> their-size" >>>>> >>>>>> limitation and provides for better reuse of metaspace chunks. It also >>>>>>> provides better defragmentation as well. >>>>>>> >>>>>>> I discussed this fix off-list with Coleen Phillimore and Jon >>>>>>> Masamitsu, >>>>>>> and instead of just offering this as a fix, both recommended to open >>>>>>> a >>>>>>> >>>>>> JEP >>>>> >>>>>> for this, because its scope would be beyond that of a simple fix. >>>>>>> >>>>>>> So here is my first JEP :) I hope it follows the right form. Please, >>>>>>> if >>>>>>> you have time, take a look and tell us what you think. >>>>>>> >>>>>>> Thank you, and Kind Regards, >>>>>>> >>>>>>> Thomas St?fe >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >> > > From volker.simonis at gmail.com Tue Oct 25 14:46:51 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 25 Oct 2016 16:46:51 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: <9371661e-99d1-6035-b9c4-d93d65fe3c72@oracle.com> References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> <9371661e-99d1-6035-b9c4-d93d65fe3c72@oracle.com> Message-ID: On Tue, Oct 25, 2016 at 5:58 AM, David Holmes wrote: > On 25/10/2016 1:36 AM, Volker Simonis wrote: >> >> So here comes the new webrev: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v2/ >> >> It moves the definition of CONST64/UCONST64 and min_jlong/max_jlong >> from the compiler-specific files to globalDefinitions.hpp. > > > Seems okay. > >> I also had to remove some AIX-specific constant definitions and >> replace them by the more standard '*[K,M,G]' notation. > > > Assuming AIX is 64-bit only this seems okay - the numeric values have > changed from uint64_t to size_t, which should still be a 64-bit unsigned > value. You're right but I hope we'll never have to do a 32-bit AIX port :) > > BTW I'm traveling tomorrow so would not be able to sponsor this till late > Wednesday at the earliest. > That would be still acceptable :) and very much appreciated! Thanks, Volker > Thanks, > David > > >> I did build and smoke tested this on Linux/amd64, MaxOS X, AIX, >> Solaris and Windows. >> >> Thanks, >> Volker >> >> >> On Mon, Oct 24, 2016 at 2:51 PM, Volker Simonis >> wrote: >>> >>> Hi Mikael, >>> >>> you're right - the OpenJDK now has the same definitions for >>> CONST64/UCONST64 on all platforms. If you don't have other settings >>> for you're closed ports I can certainly generalize the definition into >>> globalDefinitions.hpp (it also works for HP-UX which is our last >>> closed platform :) >>> >>> Notice that this would also require to move the definition of >>> min_jlong/max_jlong from the compiler-specific files to >>> globalDefinitions.hpp. >>> >>> I can do a new webrev with these additional changes. >>> >>> Regards, >>> Volker >>> >>> >>> On Mon, Oct 24, 2016 at 11:21 AM, Mikael Gerdin >>> wrote: >>>> >>>> Hi Volker, >>>> >>>> >>>> On 2016-10-24 10:53, Volker Simonis wrote: >>>>> >>>>> >>>>> On Mon, Oct 24, 2016 at 9:14 AM, David Holmes >>>>> wrote: >>>>>> >>>>>> >>>>>> On 24/10/2016 5:12 PM, Volker Simonis wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes >>>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi Volker, >>>>>>>> >>>>>>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> can I please have a review and sponsor for the following trivial >>>>>>>>> fix: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 >>>>>>>>> >>>>>>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >>>>>>>>> literals on Windows (see >>>>>>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >>>>>>>>> >>>>>>>>> Unfortunately this suffix is not known to Eclipse so most of a >>>>>>>>> hotspot >>>>>>>>> files have errors in an Eclipse project (e.g. NULL is defined as >>>>>>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in >>>>>>>>> most >>>>>>>>> hotspot of the source files). >>>>>>>>> >>>>>>>>> Fortunately VS supports the more standard conforming LL/ULL >>>>>>>>> suffixes >>>>>>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) >>>>>>>>> since >>>>>>>>> at least VS2010 (see >>>>>>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >>>>>>>>> therefore propose to change the suffix for integer literals from >>>>>>>>> i64/ui64 to LL/ULL on Windows. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> In the sense that this changes the VIS specific code to use a more >>>>>>>> stand >>>>>>>> conforming form this seems fine. >>>>>>>> >>>>>>>> But as I wrote in the bug report, should we even be using this file >>>>>>>> with >>>>>>>> non >>>>>>>> Visual Studio compilers? >>>>>>>> >>>>>>> >>>>>>> What I meant with "Eclipse doesn't understand" the suffix is that the >>>>>>> Eclipse C++ parser doesn't understand it. Eclipse just parses all the >>>>>>> C++ files which belong to a project in order to build up its code >>>>>>> database for code navigation. As such it reads >>>>>>> globalDefinitions_visCPP.hpp on Windows (if we configure a >>>>>>> corresponding project) or globalDefinitions_gcc.hpp on Linux, etc... >>>>>>> >>>>>>> This does not mean that we are using globalDefinitions_visCPP.hpp for >>>>>>> building with another compiler. And we have no IDE-specific headers >>>>>>> (this actually makes no sense since the IDEs should use the same >>>>>>> settings which are used for the actual builds). >>>>>>> >>>>>>> Not sure about other IDEs, but Eclipse's C++ parser doesn't >>>>>>> understand >>>>>>> the i64/ui64 suffix and as there is a better, more >>>>>>> standard-conforming >>>>>>> way of declaring 64-bit integer literals on Windows I've just >>>>>>> proposed >>>>>>> the cleanup. >>>>>>> >>>>>>> Does this answer your question? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Yes - thanks for clarifying. >>>>>> >>>>> >>>>> Your welcome! Could you then please sponsor this change :) >>>> >>>> >>>> >>>> Does this change make the definition of CONST64 / UCONST64 the same on >>>> all >>>> globalDefinitions_x variants? >>>> If so can we move the definition to the common file instead of having it >>>> in >>>> the per-compiler files? >>>> >>>> Thanks >>>> /Mikael >>>> >>>> >>>>> >>>>>> >>>>>> David >>>>>> >>>>>>> Thank you and best egards, >>>>>>> Volker >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>>>> Thank you and best regards, >>>>>>>>> Volker >>>>>>>>> >>>>>>>> >>>>>> >>>> > From volker.simonis at gmail.com Tue Oct 25 14:57:40 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 25 Oct 2016 16:57:40 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> Message-ID: On Tue, Oct 25, 2016 at 8:09 AM, Thomas St?fe wrote: > Hi Volker, > > so you didn't like my old SIZE_.. macros huh :) > I loved them :) but it was not so easy to keep them in the compiler specifix _xlc file now that the CONST64/UCONST64 macros are defined in the general globalDefinitions.hpp file. > The changes are fine, and strictly speaking make the AIX port more correct, > because before it was assumed all over the place that size_t is 64bit, and > now we use the correct size_t typed constants. So should we ever do an AIX > 32bit port this will help :) > > Nitpicks: > > - globals_aix.hpp: you can remove the UCONST64 macro on > MaxExpectedDataSegmentSize. I fixed that. Without UCONST64 it is only correct on 64-bit platforms but it looks nicer and I doubt we'll ever do a 32bit AIX port :) > - virtualspace.cpp: can you remove pls also remove the SIZE_xx definitions > and usage around get_attach_addresses_for_disjoint_mode()? > I'd rather leave that for another clean-up because it is not really related to this change. Following the hopefully final webrev: http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v3/ Thanks, Volker > Thanks, Thomas > > > On Mon, Oct 24, 2016 at 5:36 PM, Volker Simonis > wrote: >> >> So here comes the new webrev: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v2/ >> >> It moves the definition of CONST64/UCONST64 and min_jlong/max_jlong >> from the compiler-specific files to globalDefinitions.hpp. >> >> I also had to remove some AIX-specific constant definitions and >> replace them by the more standard '*[K,M,G]' notation. >> >> I did build and smoke tested this on Linux/amd64, MaxOS X, AIX, >> Solaris and Windows. >> >> Thanks, >> Volker >> >> >> On Mon, Oct 24, 2016 at 2:51 PM, Volker Simonis >> wrote: >> > Hi Mikael, >> > >> > you're right - the OpenJDK now has the same definitions for >> > CONST64/UCONST64 on all platforms. If you don't have other settings >> > for you're closed ports I can certainly generalize the definition into >> > globalDefinitions.hpp (it also works for HP-UX which is our last >> > closed platform :) >> > >> > Notice that this would also require to move the definition of >> > min_jlong/max_jlong from the compiler-specific files to >> > globalDefinitions.hpp. >> > >> > I can do a new webrev with these additional changes. >> > >> > Regards, >> > Volker >> > >> > >> > On Mon, Oct 24, 2016 at 11:21 AM, Mikael Gerdin >> > wrote: >> >> Hi Volker, >> >> >> >> >> >> On 2016-10-24 10:53, Volker Simonis wrote: >> >>> >> >>> On Mon, Oct 24, 2016 at 9:14 AM, David Holmes >> >>> >> >>> wrote: >> >>>> >> >>>> On 24/10/2016 5:12 PM, Volker Simonis wrote: >> >>>>> >> >>>>> >> >>>>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes >> >>>>> >> >>>>> wrote: >> >>>>>> >> >>>>>> >> >>>>>> Hi Volker, >> >>>>>> >> >>>>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> Hi, >> >>>>>>> >> >>>>>>> can I please have a review and sponsor for the following trivial >> >>>>>>> fix: >> >>>>>>> >> >>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 >> >>>>>>> >> >>>>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >> >>>>>>> literals on Windows (see >> >>>>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >> >>>>>>> >> >>>>>>> Unfortunately this suffix is not known to Eclipse so most of a >> >>>>>>> hotspot >> >>>>>>> files have errors in an Eclipse project (e.g. NULL is defined as >> >>>>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in >> >>>>>>> most >> >>>>>>> hotspot of the source files). >> >>>>>>> >> >>>>>>> Fortunately VS supports the more standard conforming LL/ULL >> >>>>>>> suffixes >> >>>>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) >> >>>>>>> since >> >>>>>>> at least VS2010 (see >> >>>>>>> >> >>>>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >> >>>>>>> therefore propose to change the suffix for integer literals from >> >>>>>>> i64/ui64 to LL/ULL on Windows. >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> In the sense that this changes the VIS specific code to use a more >> >>>>>> stand >> >>>>>> conforming form this seems fine. >> >>>>>> >> >>>>>> But as I wrote in the bug report, should we even be using this file >> >>>>>> with >> >>>>>> non >> >>>>>> Visual Studio compilers? >> >>>>>> >> >>>>> >> >>>>> What I meant with "Eclipse doesn't understand" the suffix is that >> >>>>> the >> >>>>> Eclipse C++ parser doesn't understand it. Eclipse just parses all >> >>>>> the >> >>>>> C++ files which belong to a project in order to build up its code >> >>>>> database for code navigation. As such it reads >> >>>>> globalDefinitions_visCPP.hpp on Windows (if we configure a >> >>>>> corresponding project) or globalDefinitions_gcc.hpp on Linux, >> >>>>> etc... >> >>>>> >> >>>>> This does not mean that we are using globalDefinitions_visCPP.hpp >> >>>>> for >> >>>>> building with another compiler. And we have no IDE-specific headers >> >>>>> (this actually makes no sense since the IDEs should use the same >> >>>>> settings which are used for the actual builds). >> >>>>> >> >>>>> Not sure about other IDEs, but Eclipse's C++ parser doesn't >> >>>>> understand >> >>>>> the i64/ui64 suffix and as there is a better, more >> >>>>> standard-conforming >> >>>>> way of declaring 64-bit integer literals on Windows I've just >> >>>>> proposed >> >>>>> the cleanup. >> >>>>> >> >>>>> Does this answer your question? >> >>>> >> >>>> >> >>>> >> >>>> Yes - thanks for clarifying. >> >>>> >> >>> >> >>> Your welcome! Could you then please sponsor this change :) >> >> >> >> >> >> Does this change make the definition of CONST64 / UCONST64 the same on >> >> all >> >> globalDefinitions_x variants? >> >> If so can we move the definition to the common file instead of having >> >> it in >> >> the per-compiler files? >> >> >> >> Thanks >> >> /Mikael >> >> >> >> >> >>> >> >>>> >> >>>> David >> >>>> >> >>>>> Thank you and best egards, >> >>>>> Volker >> >>>>> >> >>>>> >> >>>>>> Thanks, >> >>>>>> David >> >>>>>> >> >>>>>> >> >>>>>>> Thank you and best regards, >> >>>>>>> Volker >> >>>>>>> >> >>>>>> >> >>>> >> >> > > From thomas.stuefe at gmail.com Tue Oct 25 15:31:04 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Oct 2016 17:31:04 +0200 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> Message-ID: Looks fine! On Tuesday, 25 October 2016, Volker Simonis wrote: > On Tue, Oct 25, 2016 at 8:09 AM, Thomas St?fe > wrote: > > Hi Volker, > > > > so you didn't like my old SIZE_.. macros huh :) > > > > I loved them :) but it was not so easy to keep them in the compiler > specifix _xlc file now that the CONST64/UCONST64 macros are defined in > the general globalDefinitions.hpp file. > > > The changes are fine, and strictly speaking make the AIX port more > correct, > > because before it was assumed all over the place that size_t is 64bit, > and > > now we use the correct size_t typed constants. So should we ever do an > AIX > > 32bit port this will help :) > > > > Nitpicks: > > > > - globals_aix.hpp: you can remove the UCONST64 macro on > > MaxExpectedDataSegmentSize. > > I fixed that. Without UCONST64 it is only correct on 64-bit platforms > but it looks nicer and I doubt we'll ever do a 32bit AIX port :) > > > - virtualspace.cpp: can you remove pls also remove the SIZE_xx > definitions > > and usage around get_attach_addresses_for_disjoint_mode()? > > > > I'd rather leave that for another clean-up because it is not really > related to this change. > > Following the hopefully final webrev: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v3/ > > Thanks, > Volker > > > Thanks, Thomas > > > > > > On Mon, Oct 24, 2016 at 5:36 PM, Volker Simonis < > volker.simonis at gmail.com > > > wrote: > >> > >> So here comes the new webrev: > >> > >> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v2/ > >> > >> It moves the definition of CONST64/UCONST64 and min_jlong/max_jlong > >> from the compiler-specific files to globalDefinitions.hpp. > >> > >> I also had to remove some AIX-specific constant definitions and > >> replace them by the more standard '*[K,M,G]' notation. > >> > >> I did build and smoke tested this on Linux/amd64, MaxOS X, AIX, > >> Solaris and Windows. > >> > >> Thanks, > >> Volker > >> > >> > >> On Mon, Oct 24, 2016 at 2:51 PM, Volker Simonis > >> > wrote: > >> > Hi Mikael, > >> > > >> > you're right - the OpenJDK now has the same definitions for > >> > CONST64/UCONST64 on all platforms. If you don't have other settings > >> > for you're closed ports I can certainly generalize the definition into > >> > globalDefinitions.hpp (it also works for HP-UX which is our last > >> > closed platform :) > >> > > >> > Notice that this would also require to move the definition of > >> > min_jlong/max_jlong from the compiler-specific files to > >> > globalDefinitions.hpp. > >> > > >> > I can do a new webrev with these additional changes. > >> > > >> > Regards, > >> > Volker > >> > > >> > > >> > On Mon, Oct 24, 2016 at 11:21 AM, Mikael Gerdin > >> > > wrote: > >> >> Hi Volker, > >> >> > >> >> > >> >> On 2016-10-24 10:53, Volker Simonis wrote: > >> >>> > >> >>> On Mon, Oct 24, 2016 at 9:14 AM, David Holmes > >> >>> > > >> >>> wrote: > >> >>>> > >> >>>> On 24/10/2016 5:12 PM, Volker Simonis wrote: > >> >>>>> > >> >>>>> > >> >>>>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes > >> >>>>> > > >> >>>>> wrote: > >> >>>>>> > >> >>>>>> > >> >>>>>> Hi Volker, > >> >>>>>> > >> >>>>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> Hi, > >> >>>>>>> > >> >>>>>>> can I please have a review and sponsor for the following trivial > >> >>>>>>> fix: > >> >>>>>>> > >> >>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ > >> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 > >> >>>>>>> > >> >>>>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer > >> >>>>>>> literals on Windows (see > >> >>>>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). > >> >>>>>>> > >> >>>>>>> Unfortunately this suffix is not known to Eclipse so most of a > >> >>>>>>> hotspot > >> >>>>>>> files have errors in an Eclipse project (e.g. NULL is defined as > >> >>>>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used > in > >> >>>>>>> most > >> >>>>>>> hotspot of the source files). > >> >>>>>>> > >> >>>>>>> Fortunately VS supports the more standard conforming LL/ULL > >> >>>>>>> suffixes > >> >>>>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) > >> >>>>>>> since > >> >>>>>>> at least VS2010 (see > >> >>>>>>> > >> >>>>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100). > aspx).I > >> >>>>>>> therefore propose to change the suffix for integer literals from > >> >>>>>>> i64/ui64 to LL/ULL on Windows. > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> > >> >>>>>> In the sense that this changes the VIS specific code to use a > more > >> >>>>>> stand > >> >>>>>> conforming form this seems fine. > >> >>>>>> > >> >>>>>> But as I wrote in the bug report, should we even be using this > file > >> >>>>>> with > >> >>>>>> non > >> >>>>>> Visual Studio compilers? > >> >>>>>> > >> >>>>> > >> >>>>> What I meant with "Eclipse doesn't understand" the suffix is that > >> >>>>> the > >> >>>>> Eclipse C++ parser doesn't understand it. Eclipse just parses all > >> >>>>> the > >> >>>>> C++ files which belong to a project in order to build up its code > >> >>>>> database for code navigation. As such it reads > >> >>>>> globalDefinitions_visCPP.hpp on Windows (if we configure a > >> >>>>> corresponding project) or globalDefinitions_gcc.hpp on Linux, > >> >>>>> etc... > >> >>>>> > >> >>>>> This does not mean that we are using globalDefinitions_visCPP.hpp > >> >>>>> for > >> >>>>> building with another compiler. And we have no IDE-specific > headers > >> >>>>> (this actually makes no sense since the IDEs should use the same > >> >>>>> settings which are used for the actual builds). > >> >>>>> > >> >>>>> Not sure about other IDEs, but Eclipse's C++ parser doesn't > >> >>>>> understand > >> >>>>> the i64/ui64 suffix and as there is a better, more > >> >>>>> standard-conforming > >> >>>>> way of declaring 64-bit integer literals on Windows I've just > >> >>>>> proposed > >> >>>>> the cleanup. > >> >>>>> > >> >>>>> Does this answer your question? > >> >>>> > >> >>>> > >> >>>> > >> >>>> Yes - thanks for clarifying. > >> >>>> > >> >>> > >> >>> Your welcome! Could you then please sponsor this change :) > >> >> > >> >> > >> >> Does this change make the definition of CONST64 / UCONST64 the same > on > >> >> all > >> >> globalDefinitions_x variants? > >> >> If so can we move the definition to the common file instead of having > >> >> it in > >> >> the per-compiler files? > >> >> > >> >> Thanks > >> >> /Mikael > >> >> > >> >> > >> >>> > >> >>>> > >> >>>> David > >> >>>> > >> >>>>> Thank you and best egards, > >> >>>>> Volker > >> >>>>> > >> >>>>> > >> >>>>>> Thanks, > >> >>>>>> David > >> >>>>>> > >> >>>>>> > >> >>>>>>> Thank you and best regards, > >> >>>>>>> Volker > >> >>>>>>> > >> >>>>>> > >> >>>> > >> >> > > > > > From kim.barrett at oracle.com Tue Oct 25 23:11:37 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 25 Oct 2016 19:11:37 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <788C53B9-1CB1-45C9-BFA9-58D4DAE94C3E@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <20161010135436.GB11489@ehelin.jrpg.bea.com> <20161013101149.GL19291@ehelin.jrpg.bea.com> <01DA7656-1AFD-4DF7-9FE8-743092ADEDBA@oracle.com> <328E1EF5-9DD6-41A9-900A-56E66DF245F3@oracle.com> <20161018085351.GA19291@ehelin.jrpg.bea.com> <7A66BDA6-87B3-417A-8BF5-61E8EFFB8F29@oracle.com> <299b4c9a-6090-cf9a-ad01-8a010146e647@oracle.com> <36E3C19E-F2B7-4EA3-BBF5-2A09540EDE16@oracle.com> <788C53B9-1CB1-45C9-BFA9-58D4DAE94C3E@oracle.com> Message-ID: > On Oct 21, 2016, at 9:54 PM, Kim Barrett wrote: > >> On Oct 21, 2016, at 8:46 PM, Kim Barrett wrote: >> In the humongous case, if it bails because klass_or_null == NULL, we must re-enqueue >> the card ? This update (webrev.02) reverts part of the previous change. In the original RFR I said: As a result of the changes in oops_on_card_seq_iterate_careful, we now almost never fail to process the card. The only place where that can occur is a stale card in a humongous region with an in-progress allocation, where we can just ignore it. So the only caller, refine_card, no longer needs to examine the result of the call and enqueue the card for later reconsideration. Ignoring such a stale card is incorrect at the point where it was being done. At that point we've already cleaned the card, so we must either process the designated object(s) or, if we can't do the processing because of in-progress allocation (klass_or_null returned NULL), then re-queue the card for later reconsideration. So the change to refine_card to eliminate that behavior, and the associated changes to oops_on_card_seq_iterate_careful, were a mistake, and are being reverted by this new version. As a result, refine_card is no longer changed at all. CR: https://bugs.openjdk.java.net/browse/JDK-8166607 Webrev: Full: http://cr.openjdk.java.net/~kbarrett/8166607/webrev.02/ Incr: http://cr.openjdk.java.net/~kbarrett/8166607/webrev.02.inc/ Testing: Local specjbb2015 (non-perf). Local jtreg hotspot_all. Also tested as baseline of changes for JDK-8166811. Additionally, in the original RFR I also said: Note that [...] At present the only source of stale cards in the concurrent case seems to be HCC eviction. [...] Doing HCC cleanup when freeing regions might remove the need for klass_or_null checking in the humongous case for concurrent refinement, so might be worth looking into later. That was also incorrect; there are other sources of stale cards. That doesn't affect this change, but may effect how JDK-8166811 should be fixed, and removes the rationale for JDK-8166995 (which has been resolved Won't Fix because of that). See also the RFR for the followup JDK-8166811. From kim.barrett at oracle.com Tue Oct 25 23:13:08 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 25 Oct 2016 19:13:08 -0400 Subject: RFR: 8166811: Missing memory fences between memory allocation and refinement Message-ID: Please review this change to address missing memory barriers needed to ensure ordering between allocation and refinement in G1. Rather than simply adding the "obvious" barriers, this change modifies refinement to not need any additional memory barriers. First, the heap region type predicate used to decide whether the card should be processed has been changed. Previously, !is_young was used, but that isn't really the state of interest. Rather, processing should only occur if the region is old or humongous, not if young or *free*. The free case (and so other cases that should be filtered out) can happen if the card is stale, and there are several ways to get stale cards here. So added is_old_or_humongous type predicate and use it for filtering based on the region's type. Second, moved to refine_card the card region trimming to the heap region's allocated space, and the associated filtering, to be co-located with the type-based filtering. An empty trimmed card region is another indication of a stale card. We should filter out cards that are !is_old_or_humongous or when the card's region is beyond the allocated space for the heap region. Only if the card is old/humongous and covers allocated space should we proceed with processing, and then only for the subset of the card covering allocated space. Moved the card cleaning to refine_card. Having the cleaning in the iterator seemed misplaced. Placing it in refine_card, after the card trimming and type-based filtering also allows the fence needed for the cleaning to serve double duty; in addition to ensuring processing occurs after card cleaning (the original purpose), it now also ensures processing follows the filtering. And this ensures the necessary synchronization with allocation; we can't safely examine some parts of the heap region object or the memory designated by the card until after the card has been trimmed and filtered. Part of this involved changing the storeload to a full fence, though for supported platforms that makes no difference in the underlying implementation. (This change to card cleaning would benefit from a store_fence operation on some platforms, but that operation was phased out, and a release_store_fence is stronger (and more expensive) than needed on some platforms.) There is still a situation where processing can fail, namely an in-progress humongous allocation that hasn't set the klass yet. We continue to handle that as before. CR: https://bugs.openjdk.java.net/browse/JDK-8166811 Webrev: http://cr.openjdk.java.net/~kbarrett/8166811/webrev.00/ [Based on http://cr.openjdk.java.net/~kbarrett/8166607/webrev.02/] Testing: Local jtreg hotspot_all. Local specjbb2015 preset.ir for several hours. Local kitchensink for several hours. Nightly test equiv. via rbt. (in progress) From coleen.phillimore at oracle.com Wed Oct 26 12:11:58 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 26 Oct 2016 08:11:58 -0400 Subject: [9] RFR(S): 8164612: NoSuchMethodException when method name contains NULL or Latin-1 supplement character In-Reply-To: <580F535B.8040205@oracle.com> References: <580622AE.9080802@oracle.com> <5808692E.9090905@oracle.com> <580F535B.8040205@oracle.com> Message-ID: This looks good. Thank you for running the jck tests. http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/src/share/vm/utilities/utf8.hpp.udiff.html I really like the refactoring into a template function. Can you add a short comment here that the parameter 'length' is an input and output parameter? ie. not just output. http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/src/share/vm/utilities/utf8.cpp.udiff.html And a comment here at + length = utf8_len; Just something to point out that the length passed in has been adjusted for utf8 encoding. I don't need to see another webrev for these comment changes. Thanks, Coleen On 10/25/16 8:43 AM, Tobias Hartmann wrote: > [Ping] > > As Coleen requested, I executed the JCK/VM tests (see comment in bug). > > Best regards, > Tobias > > On 20.10.2016 08:50, Tobias Hartmann wrote: >> Hi, >> >> since this is affecting runtime code, could someone from the runtime team please have a look as well? >> >> Thanks, >> Tobias >> >> On 18.10.2016 15:25, Tobias Hartmann wrote: >>> Hi, >>> >>> please review the following patch: >>> https://bugs.openjdk.java.net/browse/JDK-8164612 >>> http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/ >>> >>> The test executes Java Script code that defines getter methods containing Latin-1 supplement characters (0x80 - 0xFF). Those methods are registered at runtime through anonymous classes via Unsafe_DefineAnonymousClass. When calling a method, the VM fails with a NoSuchMethodException in MethodHandles::resolve_MemberName(). >>> >>> The failure happens while looking up the method name symbol in java_lang_String::as_symbol_or_null() [1]: >>> 544 jbyte* position = (length == 0) ? NULL : value->byte_at_addr(0); >>> 545 const char* base = UNICODE::as_utf8(position, length); >>> 546 return SymbolTable::probe(base, length); >>> >>> If Compact Strings is enabled, we pass the Latin-1 encoded method name to UNICODE::as_utf8() and probe for the UTF-8 String in the SymbolTable. Since the Latin-1 method name contains non-ASCII characters, the length of the resulting UTF-8 String is larger (characters >= 0x80 are encoded as two bytes in UTF-8). However, we pass the shorter Latin-1 length to SymbolTable::probe() resulting in a lookup failure. >>> >>> I fixed this by passing the String length by reference to UNICODE::as_utf8(). I also refactored the related code in utf8.cpp, added comments and updated the callers. >>> >>> Tested with regression test and hs-comp PIT RBT (running). >>> >>> Thanks, >>> Tobias, >>> >>> [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/652537a80080/src/share/vm/classfile/javaClasses.cpp#l535 >>> From tobias.hartmann at oracle.com Wed Oct 26 12:17:56 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 26 Oct 2016 14:17:56 +0200 Subject: [9] RFR(S): 8164612: NoSuchMethodException when method name contains NULL or Latin-1 supplement character In-Reply-To: References: <580622AE.9080802@oracle.com> <5808692E.9090905@oracle.com> <580F535B.8040205@oracle.com> Message-ID: <58109EF4.3010707@oracle.com> Hi Coleen, thanks for looking at this! I'll add the comments you suggested before pushing. Best regards, Tobias On 26.10.2016 14:11, Coleen Phillimore wrote: > > This looks good. Thank you for running the jck tests. > > http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/src/share/vm/utilities/utf8.hpp.udiff.html > > I really like the refactoring into a template function. Can you add a short comment here that the parameter 'length' is an input and output parameter? ie. not just output. > > > http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/src/share/vm/utilities/utf8.cpp.udiff.html > > And a comment here at > > + length = utf8_len; > > > Just something to point out that the length passed in has been adjusted for utf8 encoding. > > I don't need to see another webrev for these comment changes. > > Thanks, > Coleen > > On 10/25/16 8:43 AM, Tobias Hartmann wrote: >> [Ping] >> >> As Coleen requested, I executed the JCK/VM tests (see comment in bug). >> >> Best regards, >> Tobias >> >> On 20.10.2016 08:50, Tobias Hartmann wrote: >>> Hi, >>> >>> since this is affecting runtime code, could someone from the runtime team please have a look as well? >>> >>> Thanks, >>> Tobias >>> >>> On 18.10.2016 15:25, Tobias Hartmann wrote: >>>> Hi, >>>> >>>> please review the following patch: >>>> https://bugs.openjdk.java.net/browse/JDK-8164612 >>>> http://cr.openjdk.java.net/~thartmann/8164612/webrev.00/ >>>> >>>> The test executes Java Script code that defines getter methods containing Latin-1 supplement characters (0x80 - 0xFF). Those methods are registered at runtime through anonymous classes via Unsafe_DefineAnonymousClass. When calling a method, the VM fails with a NoSuchMethodException in MethodHandles::resolve_MemberName(). >>>> >>>> The failure happens while looking up the method name symbol in java_lang_String::as_symbol_or_null() [1]: >>>> 544 jbyte* position = (length == 0) ? NULL : value->byte_at_addr(0); >>>> 545 const char* base = UNICODE::as_utf8(position, length); >>>> 546 return SymbolTable::probe(base, length); >>>> >>>> If Compact Strings is enabled, we pass the Latin-1 encoded method name to UNICODE::as_utf8() and probe for the UTF-8 String in the SymbolTable. Since the Latin-1 method name contains non-ASCII characters, the length of the resulting UTF-8 String is larger (characters >= 0x80 are encoded as two bytes in UTF-8). However, we pass the shorter Latin-1 length to SymbolTable::probe() resulting in a lookup failure. >>>> >>>> I fixed this by passing the String length by reference to UNICODE::as_utf8(). I also refactored the related code in utf8.cpp, added comments and updated the callers. >>>> >>>> Tested with regression test and hs-comp PIT RBT (running). >>>> >>>> Thanks, >>>> Tobias, >>>> >>>> [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/652537a80080/src/share/vm/classfile/javaClasses.cpp#l535 >>>> > From david.holmes at oracle.com Wed Oct 26 23:36:38 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Oct 2016 09:36:38 +1000 Subject: RFR(XS): 8168490: Use the LL/ULL suffixes to define 64-bit integer literals on Windows In-Reply-To: References: <2d818ec4-7a65-cff0-46af-4c682937f899@oracle.com> <3c092c6f-bb21-7c98-bcf6-489d64524d09@oracle.com> Message-ID: <733adfba-99df-7eb8-1a83-96f07fae9d71@oracle.com> Looks good. Submitting the push job now. Thanks, David On 26/10/2016 12:57 AM, Volker Simonis wrote: > On Tue, Oct 25, 2016 at 8:09 AM, Thomas St?fe wrote: >> Hi Volker, >> >> so you didn't like my old SIZE_.. macros huh :) >> > > I loved them :) but it was not so easy to keep them in the compiler > specifix _xlc file now that the CONST64/UCONST64 macros are defined in > the general globalDefinitions.hpp file. > >> The changes are fine, and strictly speaking make the AIX port more correct, >> because before it was assumed all over the place that size_t is 64bit, and >> now we use the correct size_t typed constants. So should we ever do an AIX >> 32bit port this will help :) >> >> Nitpicks: >> >> - globals_aix.hpp: you can remove the UCONST64 macro on >> MaxExpectedDataSegmentSize. > > I fixed that. Without UCONST64 it is only correct on 64-bit platforms > but it looks nicer and I doubt we'll ever do a 32bit AIX port :) > >> - virtualspace.cpp: can you remove pls also remove the SIZE_xx definitions >> and usage around get_attach_addresses_for_disjoint_mode()? >> > > I'd rather leave that for another clean-up because it is not really > related to this change. > > Following the hopefully final webrev: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v3/ > > Thanks, > Volker > >> Thanks, Thomas >> >> >> On Mon, Oct 24, 2016 at 5:36 PM, Volker Simonis >> wrote: >>> >>> So here comes the new webrev: >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490.v2/ >>> >>> It moves the definition of CONST64/UCONST64 and min_jlong/max_jlong >>> from the compiler-specific files to globalDefinitions.hpp. >>> >>> I also had to remove some AIX-specific constant definitions and >>> replace them by the more standard '*[K,M,G]' notation. >>> >>> I did build and smoke tested this on Linux/amd64, MaxOS X, AIX, >>> Solaris and Windows. >>> >>> Thanks, >>> Volker >>> >>> >>> On Mon, Oct 24, 2016 at 2:51 PM, Volker Simonis >>> wrote: >>>> Hi Mikael, >>>> >>>> you're right - the OpenJDK now has the same definitions for >>>> CONST64/UCONST64 on all platforms. If you don't have other settings >>>> for you're closed ports I can certainly generalize the definition into >>>> globalDefinitions.hpp (it also works for HP-UX which is our last >>>> closed platform :) >>>> >>>> Notice that this would also require to move the definition of >>>> min_jlong/max_jlong from the compiler-specific files to >>>> globalDefinitions.hpp. >>>> >>>> I can do a new webrev with these additional changes. >>>> >>>> Regards, >>>> Volker >>>> >>>> >>>> On Mon, Oct 24, 2016 at 11:21 AM, Mikael Gerdin >>>> wrote: >>>>> Hi Volker, >>>>> >>>>> >>>>> On 2016-10-24 10:53, Volker Simonis wrote: >>>>>> >>>>>> On Mon, Oct 24, 2016 at 9:14 AM, David Holmes >>>>>> >>>>>> wrote: >>>>>>> >>>>>>> On 24/10/2016 5:12 PM, Volker Simonis wrote: >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Oct 24, 2016 at 2:43 AM, David Holmes >>>>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Volker, >>>>>>>>> >>>>>>>>> On 22/10/2016 1:35 AM, Volker Simonis wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> can I please have a review and sponsor for the following trivial >>>>>>>>>> fix: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/8168490/ >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8168490 >>>>>>>>>> >>>>>>>>>> Currently we use the i64/ui64 suffixes to declare 64-bit integer >>>>>>>>>> literals on Windows (see >>>>>>>>>> src/share/vm/utilities/globalDefinitions_visCPP.hpp). >>>>>>>>>> >>>>>>>>>> Unfortunately this suffix is not known to Eclipse so most of a >>>>>>>>>> hotspot >>>>>>>>>> files have errors in an Eclipse project (e.g. NULL is defined as >>>>>>>>>> '0i64' which gives a syntax error in Eclipse, but NULL is used in >>>>>>>>>> most >>>>>>>>>> hotspot of the source files). >>>>>>>>>> >>>>>>>>>> Fortunately VS supports the more standard conforming LL/ULL >>>>>>>>>> suffixes >>>>>>>>>> (see http://en.cppreference.com/w/cpp/language/integer_literal) >>>>>>>>>> since >>>>>>>>>> at least VS2010 (see >>>>>>>>>> >>>>>>>>>> https://msdn.microsoft.com/en-us/library/00a1awxf(v=vs.100).aspx).I >>>>>>>>>> therefore propose to change the suffix for integer literals from >>>>>>>>>> i64/ui64 to LL/ULL on Windows. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> In the sense that this changes the VIS specific code to use a more >>>>>>>>> stand >>>>>>>>> conforming form this seems fine. >>>>>>>>> >>>>>>>>> But as I wrote in the bug report, should we even be using this file >>>>>>>>> with >>>>>>>>> non >>>>>>>>> Visual Studio compilers? >>>>>>>>> >>>>>>>> >>>>>>>> What I meant with "Eclipse doesn't understand" the suffix is that >>>>>>>> the >>>>>>>> Eclipse C++ parser doesn't understand it. Eclipse just parses all >>>>>>>> the >>>>>>>> C++ files which belong to a project in order to build up its code >>>>>>>> database for code navigation. As such it reads >>>>>>>> globalDefinitions_visCPP.hpp on Windows (if we configure a >>>>>>>> corresponding project) or globalDefinitions_gcc.hpp on Linux, >>>>>>>> etc... >>>>>>>> >>>>>>>> This does not mean that we are using globalDefinitions_visCPP.hpp >>>>>>>> for >>>>>>>> building with another compiler. And we have no IDE-specific headers >>>>>>>> (this actually makes no sense since the IDEs should use the same >>>>>>>> settings which are used for the actual builds). >>>>>>>> >>>>>>>> Not sure about other IDEs, but Eclipse's C++ parser doesn't >>>>>>>> understand >>>>>>>> the i64/ui64 suffix and as there is a better, more >>>>>>>> standard-conforming >>>>>>>> way of declaring 64-bit integer literals on Windows I've just >>>>>>>> proposed >>>>>>>> the cleanup. >>>>>>>> >>>>>>>> Does this answer your question? >>>>>>> >>>>>>> >>>>>>> >>>>>>> Yes - thanks for clarifying. >>>>>>> >>>>>> >>>>>> Your welcome! Could you then please sponsor this change :) >>>>> >>>>> >>>>> Does this change make the definition of CONST64 / UCONST64 the same on >>>>> all >>>>> globalDefinitions_x variants? >>>>> If so can we move the definition to the common file instead of having >>>>> it in >>>>> the per-compiler files? >>>>> >>>>> Thanks >>>>> /Mikael >>>>> >>>>> >>>>>> >>>>>>> >>>>>>> David >>>>>>> >>>>>>>> Thank you and best egards, >>>>>>>> Volker >>>>>>>> >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thank you and best regards, >>>>>>>>>> Volker >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >> >> From vladimir.kozlov at oracle.com Thu Oct 27 00:45:00 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 26 Oct 2016 17:45:00 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler Message-ID: <58114E0C.107@oracle.com> AOT JEP: https://bugs.openjdk.java.net/browse/JDK-8166089 Subtask: https://bugs.openjdk.java.net/browse/JDK-8166416 Webrev: http://cr.openjdk.java.net/~kvn/aot/top.webrev/ http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ Please, review build changes for AOT. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. Changes include new 'jaotc' launcher, makefile changes to build jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. Both modules sources are located in Hotspot: hotspot/src/jdk.aot and hotspot/src/jdk.vm.compiler. 'jaotc' requires installed libelf package on a system to build native part of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of AOT compilation. Hotspot makefile changes will be pushed together with Hotspot AOT changes. Thanks, Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 00:50:06 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 26 Oct 2016 17:50:06 -0700 Subject: [9] RFR(M) 8166418: Integrate AOT tests Message-ID: <58114F3E.5030309@oracle.com> AOT JEP: https://bugs.openjdk.java.net/browse/JDK-8166089 Subtask: https://bugs.openjdk.java.net/browse/JDK-8166418 Webrev: http://cr.openjdk.java.net/~kvn/aot/top.test.webrev/ http://cr.openjdk.java.net/~kvn/aot/hs.test.webrev/ Please, review new AOT tests and existing tests changes. Hotspot tests changes will be pushed together with Hotspot AOT changes. And top tests changes will be pushed together with top AOT changes. Thanks, Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 01:01:33 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 26 Oct 2016 18:01:33 -0700 Subject: [9] RFR(L) 8166415: Integrate AOT tool JAOTC Message-ID: <581151ED.7080904@oracle.com> AOT JEP: https://bugs.openjdk.java.net/browse/JDK-8166089 Subtask: https://bugs.openjdk.java.net/browse/JDK-8166415 Webrev: http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ Please, review implementation of jaotc - Java Ahead-Of-Time static compiler. It produces native code for compiled java methods. It uses Graal as the code-generating backend and libelf for generating shared .so AOT libraries. It is written mostly in java. Code interface to libelf is written in C. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. Corresponding build changes are reviewed in a separate thread for https://bugs.openjdk.java.net/browse/JDK-8166416 http://cr.openjdk.java.net/~kvn/aot/top.webrev/ http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ Thanks, Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 01:15:34 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 26 Oct 2016 18:15:34 -0700 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes Message-ID: <58115536.5080205@oracle.com> AOT JEP: https://bugs.openjdk.java.net/browse/JDK-8166089 Subtask: https://bugs.openjdk.java.net/browse/JDK-8166415 Webrev: http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ Please, review Hotspot VM part of AOT changes. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. AOT code is NOT linked during AOT libraries load as it happens with normal .so libraries. AOT code entry points are not exposed (not global) in AOT libraries. Only class data has global labels which we look for with dlsym(klass_name). AOT-compiled code in AOT libraries is treated by JVM as *extension* of existing CodeCache. When a java class is loaded JVM looks if corresponding AOT-compiled methods exist in loaded AOT libraries and add links to them from java methods descriptors (we have new field Method::_aot_code). AOT-compiled code follows the same invocation/deoptimization/unloading rules as normal JIT-compiled code. Calls in AOT code use the same methods resolution runtime code as calls in JITed code. The difference is call's destination address is loaded indirectly because we can't patch AOT code - it is immutable (to share between multiple JVM instances). Classes and Strings referenced in AOT code are resolved lazily by calling into runtime. All mutable pointers (oops (mostly strings), metadata) are stored and modified in a separate mutable memory (GOT cells) - they are not embedded into AOT code. Changes includes klass fingerprint generation since we need it to find correct klass data in loaded AOT libraries. Thanks, Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 04:27:15 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 26 Oct 2016 21:27:15 -0700 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap> References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap> Message-ID: <58118223.2030600@oracle.com> JEP is Targeted and RFEs are approved. I pushed 3 Hotspot changes which have touch shared code. I left for you "JDK-8167673 [s390] The s390 port." because it has only s390 specific code. Regards, Vladimir On 10/13/16 9:22 AM, Lindenmaier, Goetz wrote: > Hi Vladimir, > > I made a new webrev containing all outstanding changes merged into one patch > http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/ > > You probably saw my RFR with the s390 files. > > Best regards, > Goetz. > >> -----Original Message----- >> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >> Sent: Thursday, October 13, 2016 1:09 AM >> To: Lindenmaier, Goetz ; Volker Simonis >> >> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; build- >> dev ; HotSpot Open Source Developers >> ; Java Core Libs > dev at openjdk.java.net> >> Subject: Re: RFR: JEP draft for Linux/s3990x port >> >> Hi Goetz and Volker, >> >> Positive news is that JEP status is moving, changes are reviewed and some >> changes were already pushed. >> >> But because of our testing issues during past week I was not able to execute >> the testing. >> >> We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge >> hs-comp repository first. hs/ repo will be >> opened for other pushes soon after that. >> >> I added estimated integration date to the JEP (Oct 28). We would like to test >> and integrate this port before JDK 10 >> forest is forked. Do you think all s390 changes and new code will be ready by >> that date? >> >> Do you have all shared changes reviewed and approved for push? >> >> Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare >> changeset based on hs/ repo for changes which >> are not pushed yet? I will try to submit testing over weekend. >> >> Regards, >> Vladimir >> >> On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote: >>> Hi Vladimir, >>> >>> This webrev contains all the changes to hotspot needed for the port: >>> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390- >> all/hotspot.wr01/ >>> >>> It includes >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >> basic_s390/hotspot.wr03/ >>> http://cr.openjdk.java.net/~goetz/wr16/8166561- >> basic_C1C2_s390/webrev.01/ >>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >> scratch_emit/webrev.01/ >>> which are out for review. Further it includes >>> the one change to relocate the pc-relative instructions where we didn't >> open >>> a bug for yet, and the new s390-files. >>> >>> Altogether this passed all our tests that were running on the weekend >>> on linuxs390. >>> >>> The s390-files though are not yet fully in shape, I'm still editing them to get >>> rid of legacy stuff and SAP JVM specific code. E.g. all the code guarded by >>> #ifdef SAPJVM will go away in the end. >>> >>> I hope to have the final versions by end of this week. >>> >>> Best regards, >>> Goetz. >>> >>> >>>> -----Original Message----- >>>> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] >>>> On Behalf Of Vladimir Kozlov >>>> Sent: Montag, 3. Oktober 2016 23:50 >>>> To: Volker Simonis >>>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; >>>> build-dev ; HotSpot Open Source >> Developers >>>> ; Java Core Libs >>> dev at openjdk.java.net> >>>> Subject: Re: RFR: JEP draft for Linux/s3990x port >>>> >>>> Hi Volker, >>>> >>>> Can you prepare combined patch (or set of patches) based on latest >>>> reviews together with s390 code as it will be in final push? >>>> >>>> We want to run it through our pre-integration testing to verify that it >>>> does not have problems. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote: >>>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or >>>>> other) state: >>>>> >>>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html >>>>> >>>>> Vladimir >>>>> >>>>> On 9/29/16 9:55 AM, Volker Simonis wrote: >>>>>> Hi Vladimir, >>>>>> >>>>>> thanks a lot for reviewing and endorsing the JEP. >>>>>> >>>>>> I've linked all the relevant issues to the JEP (they all have a link >>>>>> to a webrev) and change the state to "Submitted". >>>>>> >>>>>> There's just one more small shared change we need for the port for >>>>>> which we haven't opened a bug now because we are still working on >>>>>> simplifying it. The current version looks as follows: >>>>>> >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- >>>> constant_table_offset.patch >>>>>> >>>>>> >>>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t >>>>>> he JEP and add a corresponding "FC Extension Request" comment to it? >>>>>> Or will this be done automatically once I move it to "Candidate"? >>>>>> >>>>>> Is there anything left to do before I can move it to "Candidate" state? >>>>>> >>>>>> Thanks a lot and best regards, >>>>>> Volker >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov >>>>>> wrote: >>>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> can you please review and endorse the following draft JEP for the >>>>>>>> integration of the Linux/s390x port into the jkd9 master repository: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730 >>>>>>> >>>>>>> >>>>>>> Good. >>>>>>> Add links to webrevs in a comment. It will help to get umbrella FC >>>>>>> extension >>>>>>> approval. >>>>>>> >>>>>>>> >>>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared >>>>>>>> changes and we therefore don't foresee any impact on the existing >>>>>>>> platforms at all. Following you can find a short description of the >>>>>>>> planned changes: >>>>>>>> >>>>>>>> hotspot: >>>>>>>> ======= >>>>>>>> >>>>>>>> Out for review: >>>>>>>> 8166560: [s390] Basic enablement of s390 port. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>>> basic_s390/hotspot.wr01/ >>>>>>>> >>>>>>>> Reviewed: >>>>>>>> 8166562: C2: Suppress relocations in scratch emit. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>>> scratch_emit/webrev.01/ >>>>>>>> >>>>>>>> Will send RFR soon (depends on 8166560): >>>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>>> scratch_emit/webrev.01 >>>>>>> >>>>>>> >>>>>>> Wrong link. >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> We are still investigating the need of these shared changes: >>>>>>>> >>>>>>>> >>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>> 011-pass_PC_to_retAddrOffset.patch >>>>>>>> >>>>>>>> >>>>>>>> >>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>> 016-constant_table_offset.patch >>>>>>>> >>>>>>>> >>>>>>>> And finally the patch with the s390x-only platform files. We are still >>>>>>>> editing these to get them into OpenJdk style and shape. >>>>>>>> Hotspot passes most jck, jtreg and spec tests with these. >>>>>>>> >>>>>>>> >>>> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>> 101-zFiles.patch >>>>>>>> >>>>>>>> >>>>>>>> top-level repository: >>>>>>>> =============== >>>>>>>> >>>>>>>> The following is just adding some s390x specific compiler flags to >>>>>>>> flags.m4 >>>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800 >>>>>>>> >>>>>>>> jdk repository: >>>>>>>> ============ >>>>>>>> >>>>>>>> This one just adds a new jvm.cfg file for s390x >>>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801 >>>>>>>> >>>>>>>> >>>>>>>> And finally we plan to do one more change which fixes the jtreg test >>>>>>>> on Linux/s390x. But this is mainly for the correct detection of the >>>>>>>> platform and for excluding the tests which are not appropriate for >>>>>>>> s390x. >>>>>>>> >>>>>>>> Thank you and best regards, >>>>>>>> Volker >>>>>>>> >>>>>>> From volker.simonis at gmail.com Thu Oct 27 07:37:21 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 27 Oct 2016 09:37:21 +0200 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: <58118223.2030600@oracle.com> References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap> <58118223.2030600@oracle.com> Message-ID: Vladimir, thanks a lot for your great help! Bringing the s390x port into jdk9 in time wouldn't have been possible without your support. Regards, The SAP JVM Team On Thu, Oct 27, 2016 at 6:27 AM, Vladimir Kozlov wrote: > JEP is Targeted and RFEs are approved. > > I pushed 3 Hotspot changes which have touch shared code. > I left for you "JDK-8167673 [s390] The s390 port." because it has only s390 > specific code. > > Regards, > Vladimir > > > On 10/13/16 9:22 AM, Lindenmaier, Goetz wrote: >> >> Hi Vladimir, >> >> I made a new webrev containing all outstanding changes merged into one >> patch >> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390-all/hotspot.wr01/ >> >> You probably saw my RFR with the s390 files. >> >> Best regards, >> Goetz. >> >>> -----Original Message----- >>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>> Sent: Thursday, October 13, 2016 1:09 AM >>> To: Lindenmaier, Goetz ; Volker Simonis >>> >>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; build- >>> dev ; HotSpot Open Source Developers >>> ; Java Core Libs >> dev at openjdk.java.net> >>> Subject: Re: RFR: JEP draft for Linux/s3990x port >>> >>> Hi Goetz and Volker, >>> >>> Positive news is that JEP status is moving, changes are reviewed and some >>> changes were already pushed. >>> >>> But because of our testing issues during past week I was not able to >>> execute >>> the testing. >>> >>> We hope jdk9/hs will be open soon but we want to sync jdk9/dev and merge >>> hs-comp repository first. hs/ repo will be >>> opened for other pushes soon after that. >>> >>> I added estimated integration date to the JEP (Oct 28). We would like to >>> test >>> and integrate this port before JDK 10 >>> forest is forked. Do you think all s390 changes and new code will be >>> ready by >>> that date? >>> >>> Do you have all shared changes reviewed and approved for push? >>> >>> Goetz, I saw you updated RFEs with latest webrevs. Can you again prepare >>> changeset based on hs/ repo for changes which >>> are not pushed yet? I will try to submit testing over weekend. >>> >>> Regards, >>> Vladimir >>> >>> On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote: >>>> >>>> Hi Vladimir, >>>> >>>> This webrev contains all the changes to hotspot needed for the port: >>>> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390- >>> >>> all/hotspot.wr01/ >>>> >>>> >>>> It includes >>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>> >>> basic_s390/hotspot.wr03/ >>>> >>>> http://cr.openjdk.java.net/~goetz/wr16/8166561- >>> >>> basic_C1C2_s390/webrev.01/ >>>> >>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>> >>> scratch_emit/webrev.01/ >>>> >>>> which are out for review. Further it includes >>>> the one change to relocate the pc-relative instructions where we didn't >>> >>> open >>>> >>>> a bug for yet, and the new s390-files. >>>> >>>> Altogether this passed all our tests that were running on the weekend >>>> on linuxs390. >>>> >>>> The s390-files though are not yet fully in shape, I'm still editing them >>>> to get >>>> rid of legacy stuff and SAP JVM specific code. E.g. all the code >>>> guarded by >>>> #ifdef SAPJVM will go away in the end. >>>> >>>> I hope to have the final versions by end of this week. >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>>> -----Original Message----- >>>>> From: s390x-port-dev [mailto:s390x-port-dev-bounces at openjdk.java.net] >>>>> On Behalf Of Vladimir Kozlov >>>>> Sent: Montag, 3. Oktober 2016 23:50 >>>>> To: Volker Simonis >>>>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; >>>>> build-dev ; HotSpot Open Source >>> >>> Developers >>>>> >>>>> ; Java Core Libs >>>> dev at openjdk.java.net> >>>>> Subject: Re: RFR: JEP draft for Linux/s3990x port >>>>> >>>>> Hi Volker, >>>>> >>>>> Can you prepare combined patch (or set of patches) based on latest >>>>> reviews together with s390 code as it will be in final push? >>>>> >>>>> We want to run it through our pre-integration testing to verify that it >>>>> does not have problems. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote: >>>>>> >>>>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or >>>>>> other) state: >>>>>> >>>>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html >>>>>> >>>>>> Vladimir >>>>>> >>>>>> On 9/29/16 9:55 AM, Volker Simonis wrote: >>>>>>> >>>>>>> Hi Vladimir, >>>>>>> >>>>>>> thanks a lot for reviewing and endorsing the JEP. >>>>>>> >>>>>>> I've linked all the relevant issues to the JEP (they all have a link >>>>>>> to a webrev) and change the state to "Submitted". >>>>>>> >>>>>>> There's just one more small shared change we need for the port for >>>>>>> which we haven't opened a bug now because we are still working on >>>>>>> simplifying it. The current version looks as follows: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- >>>>> >>>>> constant_table_offset.patch >>>>>>> >>>>>>> >>>>>>> >>>>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t >>>>>>> he JEP and add a corresponding "FC Extension Request" comment to it? >>>>>>> Or will this be done automatically once I move it to "Candidate"? >>>>>>> >>>>>>> Is there anything left to do before I can move it to "Candidate" >>>>>>> state? >>>>>>> >>>>>>> Thanks a lot and best regards, >>>>>>> Volker >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov >>>>>>> wrote: >>>>>>>> >>>>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> can you please review and endorse the following draft JEP for the >>>>>>>>> integration of the Linux/s390x port into the jkd9 master >>>>>>>>> repository: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Good. >>>>>>>> Add links to webrevs in a comment. It will help to get umbrella FC >>>>>>>> extension >>>>>>>> approval. >>>>>>>> >>>>>>>>> >>>>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared >>>>>>>>> changes and we therefore don't foresee any impact on the existing >>>>>>>>> platforms at all. Following you can find a short description of the >>>>>>>>> planned changes: >>>>>>>>> >>>>>>>>> hotspot: >>>>>>>>> ======= >>>>>>>>> >>>>>>>>> Out for review: >>>>>>>>> 8166560: [s390] Basic enablement of s390 port. >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- >>>>> >>>>> basic_s390/hotspot.wr01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Reviewed: >>>>>>>>> 8166562: C2: Suppress relocations in scratch emit. >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>>>> >>>>> scratch_emit/webrev.01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Will send RFR soon (depends on 8166560): >>>>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- >>>>> >>>>> scratch_emit/webrev.01 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Wrong link. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimir >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> We are still investigating the need of these shared changes: >>>>>>>>> >>>>>>>>> >>>>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>>> >>>>> 011-pass_PC_to_retAddrOffset.patch >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>>> >>>>> 016-constant_table_offset.patch >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> And finally the patch with the s390x-only platform files. We are >>>>>>>>> still >>>>>>>>> editing these to get them into OpenJdk style and shape. >>>>>>>>> Hotspot passes most jck, jtreg and spec tests with these. >>>>>>>>> >>>>>>>>> >>>>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 >>>>> >>>>> 101-zFiles.patch >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> top-level repository: >>>>>>>>> =============== >>>>>>>>> >>>>>>>>> The following is just adding some s390x specific compiler flags to >>>>>>>>> flags.m4 >>>>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800 >>>>>>>>> >>>>>>>>> jdk repository: >>>>>>>>> ============ >>>>>>>>> >>>>>>>>> This one just adds a new jvm.cfg file for s390x >>>>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801 >>>>>>>>> >>>>>>>>> >>>>>>>>> And finally we plan to do one more change which fixes the jtreg >>>>>>>>> test >>>>>>>>> on Linux/s390x. But this is mainly for the correct detection of the >>>>>>>>> platform and for excluding the tests which are not appropriate for >>>>>>>>> s390x. >>>>>>>>> >>>>>>>>> Thank you and best regards, >>>>>>>>> Volker >>>>>>>>> >>>>>>>> > From aph at redhat.com Thu Oct 27 09:09:38 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 27 Oct 2016 10:09:38 +0100 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: <58115536.5080205@oracle.com> References: <58115536.5080205@oracle.com> Message-ID: <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> On 27/10/16 02:15, Vladimir Kozlov wrote: > Please, review Hotspot VM part of AOT changes. > > Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. Of course I'll port this to AArch64 asap. Andrew. From goetz.lindenmaier at sap.com Thu Oct 27 09:47:07 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 27 Oct 2016 09:47:07 +0000 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: <58118223.2030600@oracle.com> References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> <69e91f16-30b8-427b-a792-b6d1bf98580c@oracle.com> <2b52f6217ba941dcaf95a62dd6e5dab9@DEWDFE13DE50.global.corp.sap> <58118223.2030600@oracle.com> Message-ID: Hi Vladimir I pushed the remaining change. It is now possible to build a s390 VM directly out of the jdk9/hs repo. Thanks a lot for your help with the JEP, testing and all the other issues with the port! For me, this has been a very uncomplicated straight forward project without too much overhead! Also a big thanks to the other people at Oracle that have been involved. Best regards, Goetz > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Donnerstag, 27. Oktober 2016 06:27 > To: Lindenmaier, Goetz ; Volker Simonis > > Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; > HotSpot Open Source Developers > Subject: Re: RFR: JEP draft for Linux/s3990x port > > JEP is Targeted and RFEs are approved. > > I pushed 3 Hotspot changes which have touch shared code. > I left for you "JDK-8167673 [s390] The s390 port." because it has only s390 > specific code. > > Regards, > Vladimir > > On 10/13/16 9:22 AM, Lindenmaier, Goetz wrote: > > Hi Vladimir, > > > > I made a new webrev containing all outstanding changes merged into one > patch > > http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390- > all/hotspot.wr01/ > > > > You probably saw my RFR with the s390 files. > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > >> Sent: Thursday, October 13, 2016 1:09 AM > >> To: Lindenmaier, Goetz ; Volker Simonis > >> > >> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; > build- > >> dev ; HotSpot Open Source Developers > >> ; Java Core Libs >> dev at openjdk.java.net> > >> Subject: Re: RFR: JEP draft for Linux/s3990x port > >> > >> Hi Goetz and Volker, > >> > >> Positive news is that JEP status is moving, changes are reviewed and some > >> changes were already pushed. > >> > >> But because of our testing issues during past week I was not able to > execute > >> the testing. > >> > >> We hope jdk9/hs will be open soon but we want to sync jdk9/dev and > merge > >> hs-comp repository first. hs/ repo will be > >> opened for other pushes soon after that. > >> > >> I added estimated integration date to the JEP (Oct 28). We would like to > test > >> and integrate this port before JDK 10 > >> forest is forked. Do you think all s390 changes and new code will be ready > by > >> that date? > >> > >> Do you have all shared changes reviewed and approved for push? > >> > >> Goetz, I saw you updated RFEs with latest webrevs. Can you again > prepare > >> changeset based on hs/ repo for changes which > >> are not pushed yet? I will try to submit testing over weekend. > >> > >> Regards, > >> Vladimir > >> > >> On 10/4/16 9:48 AM, Lindenmaier, Goetz wrote: > >>> Hi Vladimir, > >>> > >>> This webrev contains all the changes to hotspot needed for the port: > >>> http://cr.openjdk.java.net/~goetz/wr16/8166730-linuxs390- > >> all/hotspot.wr01/ > >>> > >>> It includes > >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >> basic_s390/hotspot.wr03/ > >>> http://cr.openjdk.java.net/~goetz/wr16/8166561- > >> basic_C1C2_s390/webrev.01/ > >>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > >> scratch_emit/webrev.01/ > >>> which are out for review. Further it includes > >>> the one change to relocate the pc-relative instructions where we didn't > >> open > >>> a bug for yet, and the new s390-files. > >>> > >>> Altogether this passed all our tests that were running on the weekend > >>> on linuxs390. > >>> > >>> The s390-files though are not yet fully in shape, I'm still editing them to > get > >>> rid of legacy stuff and SAP JVM specific code. E.g. all the code guarded > by > >>> #ifdef SAPJVM will go away in the end. > >>> > >>> I hope to have the final versions by end of this week. > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>>> -----Original Message----- > >>>> From: s390x-port-dev [mailto:s390x-port-dev- > bounces at openjdk.java.net] > >>>> On Behalf Of Vladimir Kozlov > >>>> Sent: Montag, 3. Oktober 2016 23:50 > >>>> To: Volker Simonis > >>>> Cc: s390x-port-dev at openjdk.java.net; porters-dev at openjdk.java.net; > >>>> build-dev ; HotSpot Open Source > >> Developers > >>>> ; Java Core Libs >>>> dev at openjdk.java.net> > >>>> Subject: Re: RFR: JEP draft for Linux/s3990x port > >>>> > >>>> Hi Volker, > >>>> > >>>> Can you prepare combined patch (or set of patches) based on latest > >>>> reviews together with s390 code as it will be in final push? > >>>> > >>>> We want to run it through our pre-integration testing to verify that it > >>>> does not have problems. > >>>> > >>>> Thanks, > >>>> Vladimir > >>>> > >>>> On 9/29/16 11:25 AM, Vladimir Kozlov wrote: > >>>>> You need to wait when Mark (OpenJDK Lead) move it to Candidate > (or > >>>>> other) state: > >>>>> > >>>>> http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html > >>>>> > >>>>> Vladimir > >>>>> > >>>>> On 9/29/16 9:55 AM, Volker Simonis wrote: > >>>>>> Hi Vladimir, > >>>>>> > >>>>>> thanks a lot for reviewing and endorsing the JEP. > >>>>>> > >>>>>> I've linked all the relevant issues to the JEP (they all have a link > >>>>>> to a webrev) and change the state to "Submitted". > >>>>>> > >>>>>> There's just one more small shared change we need for the port for > >>>>>> which we haven't opened a bug now because we are still working on > >>>>>> simplifying it. The current version looks as follows: > >>>>>> > >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016- > >>>> constant_table_offset.patch > >>>>>> > >>>>>> > >>>>>> What are the next steps? Should I add a "jdk9-fc-request" label to t > >>>>>> he JEP and add a corresponding "FC Extension Request" comment to > it? > >>>>>> Or will this be done automatically once I move it to "Candidate"? > >>>>>> > >>>>>> Is there anything left to do before I can move it to "Candidate" > state? > >>>>>> > >>>>>> Thanks a lot and best regards, > >>>>>> Volker > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov > >>>>>> wrote: > >>>>>>> On 9/27/16 10:49 AM, Volker Simonis wrote: > >>>>>>>> > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> can you please review and endorse the following draft JEP for the > >>>>>>>> integration of the Linux/s390x port into the jkd9 master > repository: > >>>>>>>> > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166730 > >>>>>>> > >>>>>>> > >>>>>>> Good. > >>>>>>> Add links to webrevs in a comment. It will help to get umbrella FC > >>>>>>> extension > >>>>>>> approval. > >>>>>>> > >>>>>>>> > >>>>>>>> As detailed in the JEP, the Linux/s390x requires very few shared > >>>>>>>> changes and we therefore don't foresee any impact on the > existing > >>>>>>>> platforms at all. Following you can find a short description of the > >>>>>>>> planned changes: > >>>>>>>> > >>>>>>>> hotspot: > >>>>>>>> ======= > >>>>>>>> > >>>>>>>> Out for review: > >>>>>>>> 8166560: [s390] Basic enablement of s390 port. > >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > >>>> basic_s390/hotspot.wr01/ > >>>>>>>> > >>>>>>>> Reviewed: > >>>>>>>> 8166562: C2: Suppress relocations in scratch emit. > >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > >>>> scratch_emit/webrev.01/ > >>>>>>>> > >>>>>>>> Will send RFR soon (depends on 8166560): > >>>>>>>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. > >>>>>>>> http://cr.openjdk.java.net/~goetz/wr16/8166562- > >>>> scratch_emit/webrev.01 > >>>>>>> > >>>>>>> > >>>>>>> Wrong link. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Vladimir > >>>>>>> > >>>>>>> > >>>>>>>> > >>>>>>>> We are still investigating the need of these shared changes: > >>>>>>>> > >>>>>>>> > >>>> > >> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >>>> 011-pass_PC_to_retAddrOffset.patch > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>> > >> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >>>> 016-constant_table_offset.patch > >>>>>>>> > >>>>>>>> > >>>>>>>> And finally the patch with the s390x-only platform files. We are > still > >>>>>>>> editing these to get them into OpenJdk style and shape. > >>>>>>>> Hotspot passes most jck, jtreg and spec tests with these. > >>>>>>>> > >>>>>>>> > >>>> > >> > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000 > >>>> 101-zFiles.patch > >>>>>>>> > >>>>>>>> > >>>>>>>> top-level repository: > >>>>>>>> =============== > >>>>>>>> > >>>>>>>> The following is just adding some s390x specific compiler flags to > >>>>>>>> flags.m4 > >>>>>>>> 8166800: [s390] Top-level build changes required for Linux/s390x > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166800 > >>>>>>>> > >>>>>>>> jdk repository: > >>>>>>>> ============ > >>>>>>>> > >>>>>>>> This one just adds a new jvm.cfg file for s390x > >>>>>>>> 8166801: [s390] Add jvm.cfg file for Linux/s390x > >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8166801 > >>>>>>>> > >>>>>>>> > >>>>>>>> And finally we plan to do one more change which fixes the jtreg > test > >>>>>>>> on Linux/s390x. But this is mainly for the correct detection of the > >>>>>>>> platform and for excluding the tests which are not appropriate for > >>>>>>>> s390x. > >>>>>>>> > >>>>>>>> Thank you and best regards, > >>>>>>>> Volker > >>>>>>>> > >>>>>>> From volker.simonis at gmail.com Thu Oct 27 10:14:37 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 27 Oct 2016 12:14:37 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <58114E0C.107@oracle.com> References: <58114E0C.107@oracle.com> Message-ID: Hi Vladimir, I'm trying to build and test these changes on our platforms and I've just realized that the top-level changes do not contained the generated-configure.sh script. I for myself have generated it for me, but I think it would be useful to always include that into a webrev for convenience because not everybody may have the right version of autoconf at hand. Thanks, Volker On Thu, Oct 27, 2016 at 2:45 AM, Vladimir Kozlov wrote: > AOT JEP: > https://bugs.openjdk.java.net/browse/JDK-8166089 > Subtask: > https://bugs.openjdk.java.net/browse/JDK-8166416 > Webrev: > http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > > Please, review build changes for AOT. Only Linux/x64 platform is supported. > 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. > > Changes include new 'jaotc' launcher, makefile changes to build > jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. > Both modules sources are located in Hotspot: hotspot/src/jdk.aot and > hotspot/src/jdk.vm.compiler. > 'jaotc' requires installed libelf package on a system to build native part > of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of > AOT compilation. > > Hotspot makefile changes will be pushed together with Hotspot AOT changes. > > Thanks, > Vladimir From volker.simonis at gmail.com Thu Oct 27 12:22:32 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 27 Oct 2016 14:22:32 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <58114E0C.107@oracle.com> References: <58114E0C.107@oracle.com> Message-ID: Hi, currently libelf support is checked on every 64-bit Linux and Solaris platform (see lib-elf.m4): if ((test "x$OPENJDK_TARGET_OS" = "xlinux" \ || test "x$OPENJDK_TARGET_OS" = "xsolaris") && \ (test "x$OPENJDK_TARGET_CPU_BITS" = "x64")); then LIBJELFSHIM_ENABLED="true" else LIBJELFSHIM_ENABLED="false" fi As far as I understand, this is only required for AOT and should therefore only be checked for if AOT is enabled. We could do something similar like for FFI (see libraries.m4): # Check if ffi is needed if HOTSPOT_CHECK_JVM_VARIANT(zero) || HOTSPOT_CHECK_JVM_VARIANT(zeroshark); then NEEDS_LIB_FFI=true else NEEDS_LIB_FFI=false fi And then test for "NEEDS_LIB_JELFSHIM" in lib-elf.m4 instead of testing for a specific platform. Otherwise we get ugly warning like: WARNING: Could not find libelf! Not building libjelfshim.so during configuration if we are running on platforms without AOT support or if AOT support is switched of. Regards, Volker On Thu, Oct 27, 2016 at 2:45 AM, Vladimir Kozlov wrote: > AOT JEP: > https://bugs.openjdk.java.net/browse/JDK-8166089 > Subtask: > https://bugs.openjdk.java.net/browse/JDK-8166416 > Webrev: > http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > > Please, review build changes for AOT. Only Linux/x64 platform is supported. > 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. > > Changes include new 'jaotc' launcher, makefile changes to build > jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. > Both modules sources are located in Hotspot: hotspot/src/jdk.aot and > hotspot/src/jdk.vm.compiler. > 'jaotc' requires installed libelf package on a system to build native part > of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of > AOT compilation. > > Hotspot makefile changes will be pushed together with Hotspot AOT changes. > > Thanks, > Vladimir From erik.joelsson at oracle.com Thu Oct 27 12:40:47 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 27 Oct 2016 14:40:47 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <58114E0C.107@oracle.com> References: <58114E0C.107@oracle.com> Message-ID: On 2016-10-27 02:45, Vladimir Kozlov wrote: > AOT JEP: > https://bugs.openjdk.java.net/browse/JDK-8166089 > Subtask: > https://bugs.openjdk.java.net/browse/JDK-8166416 > Webrev: > http://cr.openjdk.java.net/~kvn/aot/top.webrev/ hotspot.m4: 296: Comment is misleading. Should just be removed. CompileJavaModules.gmk: Use of -g flag for java compilation is controlled globally. Please remove. Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK conditional like all other buildtools targets. > http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ The extra exports from java.base needs to go in a new jdk/src/java.base/share/classes/module-info.java.extra since the module jdk.vm.compiler is optional. > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris automatically. No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and controlled globally. /Erik > > Please, review build changes for AOT. Only Linux/x64 platform is > supported. 'jaotc' and AOT part of Hotspot will be build only on > Linux/x64. > > Changes include new 'jaotc' launcher, makefile changes to build > jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. > Both modules sources are located in Hotspot: hotspot/src/jdk.aot and > hotspot/src/jdk.vm.compiler. > 'jaotc' requires installed libelf package on a system to build native > part of 'jaotc'. It is used to generated AOT shared libraries (.so) as > result of AOT compilation. > > Hotspot makefile changes will be pushed together with Hotspot AOT > changes. > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 14:14:46 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 07:14:46 -0700 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> Message-ID: Thank you, Andrew, There is no official port of Graal for AArch64 yet. But the work seems started based on your mails on graal mailing list ;) There are several hardcoded x86 parts in JAOTC have to be fixed. For example: lines 76-82: http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CodeSectionProcessor.java.html lines 392-400: http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CompiledMethodInfo.java.html And jaotc ELF interface code. Hotspot changes are not simple but platform specific mostly. Regards, Vladimir On 10/27/16 2:09 AM, Andrew Haley wrote: > On 27/10/16 02:15, Vladimir Kozlov wrote: >> Please, review Hotspot VM part of AOT changes. >> >> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. > > Of course I'll port this to AArch64 asap. > > Andrew. > From vladimir.kozlov at oracle.com Thu Oct 27 14:17:37 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 07:17:37 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> Message-ID: I see. I specifically removed generated-configure.sh from webrev since it has to be generated. I will add version of webrev with it. Regards, Vladimir On 10/27/16 3:14 AM, Volker Simonis wrote: > Hi Vladimir, > > I'm trying to build and test these changes on our platforms and I've > just realized that the top-level changes do not contained the > generated-configure.sh script. I for myself have generated it for me, > but I think it would be useful to always include that into a webrev > for convenience because not everybody may have the right version of > autoconf at hand. > > Thanks, > Volker > > > On Thu, Oct 27, 2016 at 2:45 AM, Vladimir Kozlov > wrote: >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166416 >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >> >> Please, review build changes for AOT. Only Linux/x64 platform is supported. >> 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >> >> Changes include new 'jaotc' launcher, makefile changes to build >> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and >> hotspot/src/jdk.vm.compiler. >> 'jaotc' requires installed libelf package on a system to build native part >> of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of >> AOT compilation. >> >> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >> >> Thanks, >> Vladimir From volker.simonis at gmail.com Thu Oct 27 15:45:37 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 27 Oct 2016 17:45:37 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> Message-ID: On Thu, Oct 27, 2016 at 2:22 PM, Volker Simonis wrote: > Hi, > > currently libelf support is checked on every 64-bit Linux and Solaris > platform (see lib-elf.m4): > > if ((test "x$OPENJDK_TARGET_OS" = "xlinux" \ > || test "x$OPENJDK_TARGET_OS" = "xsolaris") && \ > (test "x$OPENJDK_TARGET_CPU_BITS" = "x64")); then > LIBJELFSHIM_ENABLED="true" > else > LIBJELFSHIM_ENABLED="false" > fi > > As far as I understand, this is only required for AOT and should > therefore only be checked for if AOT is enabled. We could do something > similar like for FFI (see libraries.m4): > > # Check if ffi is needed > if HOTSPOT_CHECK_JVM_VARIANT(zero) || > HOTSPOT_CHECK_JVM_VARIANT(zeroshark); then > NEEDS_LIB_FFI=true > else > NEEDS_LIB_FFI=false > fi > > And then test for "NEEDS_LIB_JELFSHIM" in lib-elf.m4 instead of > testing for a specific platform. > > Otherwise we get ugly warning like: > > WARNING: Could not find libelf! Not building libjelfshim.so > And I think in the case where we compile for AOT, we should print the usual, platform-dependent hints like for example: libelf.so not found: try to run 'sudo apt-get install libelf-dev' > during configuration if we are running on platforms without AOT > support or if AOT support is switched of. > > Regards, > Volker > > > > On Thu, Oct 27, 2016 at 2:45 AM, Vladimir Kozlov > wrote: >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166416 >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >> >> Please, review build changes for AOT. Only Linux/x64 platform is supported. >> 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >> >> Changes include new 'jaotc' launcher, makefile changes to build >> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and >> hotspot/src/jdk.vm.compiler. >> 'jaotc' requires installed libelf package on a system to build native part >> of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of >> AOT compilation. >> >> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >> >> Thanks, >> Vladimir From volker.simonis at gmail.com Thu Oct 27 15:45:59 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 27 Oct 2016 17:45:59 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> Message-ID: I can't even compile this on Linux/x86_64. I get the following error: /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:159: error: module not found: jdk.vm.compiler jdk.vm.compiler; ^ /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:180: error: module not found: jdk.vm.compiler jdk.vm.compiler, ^ /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:309: error: module not found: jdk.vm.compiler jdk.vm.compiler; I have integrated this two webrevs and the hotspot webrev from http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ What do I have to do, to get it at least compiled? Regards, Volker On Thu, Oct 27, 2016 at 2:40 PM, Erik Joelsson wrote: > > > On 2016-10-27 02:45, Vladimir Kozlov wrote: >> >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166416 >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > > hotspot.m4: 296: Comment is misleading. Should just be removed. > > CompileJavaModules.gmk: Use of -g flag for java compilation is controlled > globally. Please remove. > > Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK > conditional like all other buildtools targets. > >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > > The extra exports from java.base needs to go in a new > jdk/src/java.base/share/classes/module-info.java.extra since the module > jdk.vm.compiler is optional. >> >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > > Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to > LIBS since that will provide -lc on Solaris automatically. No need to set > DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and > controlled globally. > > /Erik > >> >> Please, review build changes for AOT. Only Linux/x64 platform is >> supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >> >> Changes include new 'jaotc' launcher, makefile changes to build >> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and >> hotspot/src/jdk.vm.compiler. >> 'jaotc' requires installed libelf package on a system to build native part >> of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of >> AOT compilation. >> >> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >> >> Thanks, >> Vladimir > > From vladimir.kozlov at oracle.com Thu Oct 27 16:20:47 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 09:20:47 -0700 Subject: [9] RFR(L) 8166417: Graal-core into JDK for AOT compiler Message-ID: <5812295F.4000107@oracle.com> AOT JEP: https://bugs.openjdk.java.net/browse/JDK-8166089 Subtask: https://bugs.openjdk.java.net/browse/JDK-8166417 Graal-core: http://hg.openjdk.java.net/graal/graal-core/ Webrev: http://cr.openjdk.java.net/~kvn/aot/graal.webrev/ This is last part of AOT project. AOT requires Graal-core to be part of JDK. AOT compiler uses Graal-core as backend compiler - we use its ability to generate immutable PIC code. We are working with Oracle's Labs on this integration. Meanwhile, if you want to experiment do next: hg clone http://hg.openjdk.java.net/graal/graal-core/ apply patch from graal.webrev cp graal-core/graal myjdk/hotspot/src/jdk.vm.compiler After that you can build JDK with AOT. Thanks, Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 16:24:48 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 09:24:48 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> Message-ID: <58122A50.10002@oracle.com> Yes, you need to integrate graal-core into hotspot for that. I just send final RFR which explains what to do: 8166417: Graal-core into JDK for AOT compiler Regards, Vladimir On 10/27/16 8:45 AM, Volker Simonis wrote: > I can't even compile this on Linux/x86_64. I get the following error: > > /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:159: > error: module not found: jdk.vm.compiler > jdk.vm.compiler; > ^ > /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:180: > error: module not found: jdk.vm.compiler > jdk.vm.compiler, > ^ > /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:309: > error: module not found: jdk.vm.compiler > jdk.vm.compiler; > > I have integrated this two webrevs and the hotspot webrev from > http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ > > What do I have to do, to get it at least compiled? > > Regards, > Volker > > On Thu, Oct 27, 2016 at 2:40 PM, Erik Joelsson wrote: >> >> >> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>> >>> AOT JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>> Subtask: >>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>> Webrev: >>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> >> hotspot.m4: 296: Comment is misleading. Should just be removed. >> >> CompileJavaModules.gmk: Use of -g flag for java compilation is controlled >> globally. Please remove. >> >> Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK >> conditional like all other buildtools targets. >> >>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> >> The extra exports from java.base needs to go in a new >> jdk/src/java.base/share/classes/module-info.java.extra since the module >> jdk.vm.compiler is optional. >>> >>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >> >> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to >> LIBS since that will provide -lc on Solaris automatically. No need to set >> DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and >> controlled globally. >> >> /Erik >> >>> >>> Please, review build changes for AOT. Only Linux/x64 platform is >>> supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>> >>> Changes include new 'jaotc' launcher, makefile changes to build >>> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and >>> hotspot/src/jdk.vm.compiler. >>> 'jaotc' requires installed libelf package on a system to build native part >>> of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of >>> AOT compilation. >>> >>> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >>> >>> Thanks, >>> Vladimir >> >> From volker.simonis at gmail.com Thu Oct 27 16:42:56 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 27 Oct 2016 18:42:56 +0200 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: <58115536.5080205@oracle.com> References: <58115536.5080205@oracle.com> Message-ID: Hi, first of all I had some problems importing the patch from the webrev because of this sequence: --- old/./.hgignore 2016-10-26 14:56:22.000000000 -0700 +++ new/./.hgignore 2016-10-26 14:56:22.000000000 -0700 @@ -24,3 +24,19 @@ ^test/compiler/jvmci/\w[\w\.]*/.*\.iml ^test/compiler/jvmci/\w[\w\.]*/nbproject The error (when using hg qpush -v) was: applying 8166413_AOT.patch path './.hgignore' is inside nested repo '.' committing changelog patch failed, rejects left in working directory errors during apply, please fix and qrefresh 8166413_AOT.patch I don't know how it was generated, but I could easily fix it by replacing the above sequence by: --- old/.hgignore 2016-10-26 14:56:22.000000000 -0700 +++ new/.hgignore 2016-10-26 14:56:22.000000000 -0700 Besides that, it would be great if you could integrate the followinf ppc/s390 specific changes which are required to keep the corresponding ports building: http://cr.openjdk.java.net/~simonis/webrevs/2016/8166413_ppc_s390_addon/ Thanks, Volker On Thu, Oct 27, 2016 at 3:15 AM, Vladimir Kozlov wrote: > AOT JEP: > https://bugs.openjdk.java.net/browse/JDK-8166089 > Subtask: > https://bugs.openjdk.java.net/browse/JDK-8166415 > Webrev: > http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ > > Please, review Hotspot VM part of AOT changes. > > Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will > be build only on Linux/x64. > > AOT code is NOT linked during AOT libraries load as it happens with normal > .so libraries. AOT code entry points are not exposed (not global) in AOT > libraries. Only class data has global labels which we look for with > dlsym(klass_name). > > AOT-compiled code in AOT libraries is treated by JVM as *extension* of > existing CodeCache. When a java class is loaded JVM looks if corresponding > AOT-compiled methods exist in loaded AOT libraries and add links to them > from java methods descriptors (we have new field Method::_aot_code). > AOT-compiled code follows the same invocation/deoptimization/unloading rules > as normal JIT-compiled code. > > Calls in AOT code use the same methods resolution runtime code as calls in > JITed code. The difference is call's destination address is loaded > indirectly because we can't patch AOT code - it is immutable (to share > between multiple JVM instances). > > Classes and Strings referenced in AOT code are resolved lazily by calling > into runtime. All mutable pointers (oops (mostly strings), metadata) are > stored and modified in a separate mutable memory (GOT cells) - they are not > embedded into AOT code. > > Changes includes klass fingerprint generation since we need it to find > correct klass data in loaded AOT libraries. > > Thanks, > Vladimir From volker.simonis at gmail.com Thu Oct 27 16:47:54 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 27 Oct 2016 18:47:54 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <58122A50.10002@oracle.com> References: <58114E0C.107@oracle.com> <58122A50.10002@oracle.com> Message-ID: OK, thanks. I'll do that tomorrow. For now I did built 'hotspot-only' and sent out a webrev with the requiredppc/s390 changes for hotspot to the corresponding review thread. Would be nice if you could integrate them into your patch right away. Thanks, Volker On Thu, Oct 27, 2016 at 6:24 PM, Vladimir Kozlov wrote: > Yes, you need to integrate graal-core into hotspot for that. > > I just send final RFR which explains what to do: > > 8166417: Graal-core into JDK for AOT compiler > > Regards, > Vladimir > > > On 10/27/16 8:45 AM, Volker Simonis wrote: >> >> I can't even compile this on Linux/x86_64. I get the following error: >> >> >> /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:159: >> error: module not found: jdk.vm.compiler >> jdk.vm.compiler; >> ^ >> >> /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:180: >> error: module not found: jdk.vm.compiler >> jdk.vm.compiler, >> ^ >> >> /usr/work/d046063/OpenJDK/jdk9-hs/jdk/src/java.base/share/classes/module-info.java:309: >> error: module not found: jdk.vm.compiler >> jdk.vm.compiler; >> >> I have integrated this two webrevs and the hotspot webrev from >> http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ >> >> What do I have to do, to get it at least compiled? >> >> Regards, >> Volker >> >> On Thu, Oct 27, 2016 at 2:40 PM, Erik Joelsson >> wrote: >>> >>> >>> >>> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>>> >>>> >>>> AOT JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>> Subtask: >>>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>>> Webrev: >>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>> >>> >>> hotspot.m4: 296: Comment is misleading. Should just be removed. >>> >>> CompileJavaModules.gmk: Use of -g flag for java compilation is controlled >>> globally. Please remove. >>> >>> Main.gmk: buildtools-hotspot should be declared inside the >>> CREATING_BUILDJDK >>> conditional like all other buildtools targets. >>> >>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>> >>> >>> The extra exports from java.base needs to go in a new >>> jdk/src/java.base/share/classes/module-info.java.extra since the module >>> jdk.vm.compiler is optional. >>>> >>>> >>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> >>> >>> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to >>> LIBS since that will provide -lc on Solaris automatically. No need to set >>> DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and >>> controlled globally. >>> >>> /Erik >>> >>>> >>>> Please, review build changes for AOT. Only Linux/x64 platform is >>>> supported. 'jaotc' and AOT part of Hotspot will be build only on >>>> Linux/x64. >>>> >>>> Changes include new 'jaotc' launcher, makefile changes to build >>>> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and >>>> hotspot/src/jdk.vm.compiler. >>>> 'jaotc' requires installed libelf package on a system to build native >>>> part >>>> of 'jaotc'. It is used to generated AOT shared libraries (.so) as result >>>> of >>>> AOT compilation. >>>> >>>> Hotspot makefile changes will be pushed together with Hotspot AOT >>>> changes. >>>> >>>> Thanks, >>>> Vladimir >>> >>> >>> > From vladimir.kozlov at oracle.com Thu Oct 27 17:20:18 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 10:20:18 -0700 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: References: <58115536.5080205@oracle.com> Message-ID: <58123752.8060308@oracle.com> Thank you, Volker I will integrate ppc and s390 changes after I merge jdk9/hs into AOT repo. And I will look on .hgignore. I use 'hg import --no-commit AOT.patch' command to apply patches. Thanks, Vladimir On 10/27/16 9:42 AM, Volker Simonis wrote: > Hi, > > first of all I had some problems importing the patch from the webrev > because of this sequence: > > --- old/./.hgignore 2016-10-26 14:56:22.000000000 -0700 > +++ new/./.hgignore 2016-10-26 14:56:22.000000000 -0700 > @@ -24,3 +24,19 @@ > ^test/compiler/jvmci/\w[\w\.]*/.*\.iml > ^test/compiler/jvmci/\w[\w\.]*/nbproject > > The error (when using hg qpush -v) was: > > applying 8166413_AOT.patch > path './.hgignore' is inside nested repo '.' > committing changelog > patch failed, rejects left in working directory > errors during apply, please fix and qrefresh 8166413_AOT.patch > > I don't know how it was generated, but I could easily fix it by > replacing the above sequence by: > > --- old/.hgignore 2016-10-26 14:56:22.000000000 -0700 > +++ new/.hgignore 2016-10-26 14:56:22.000000000 -0700 > > > Besides that, it would be great if you could integrate the followinf > ppc/s390 specific changes which are required to keep the corresponding > ports building: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8166413_ppc_s390_addon/ > > Thanks, > Volker > > > On Thu, Oct 27, 2016 at 3:15 AM, Vladimir Kozlov > wrote: >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166415 >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ >> >> Please, review Hotspot VM part of AOT changes. >> >> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will >> be build only on Linux/x64. >> >> AOT code is NOT linked during AOT libraries load as it happens with normal >> .so libraries. AOT code entry points are not exposed (not global) in AOT >> libraries. Only class data has global labels which we look for with >> dlsym(klass_name). >> >> AOT-compiled code in AOT libraries is treated by JVM as *extension* of >> existing CodeCache. When a java class is loaded JVM looks if corresponding >> AOT-compiled methods exist in loaded AOT libraries and add links to them >> from java methods descriptors (we have new field Method::_aot_code). >> AOT-compiled code follows the same invocation/deoptimization/unloading rules >> as normal JIT-compiled code. >> >> Calls in AOT code use the same methods resolution runtime code as calls in >> JITed code. The difference is call's destination address is loaded >> indirectly because we can't patch AOT code - it is immutable (to share >> between multiple JVM instances). >> >> Classes and Strings referenced in AOT code are resolved lazily by calling >> into runtime. All mutable pointers (oops (mostly strings), metadata) are >> stored and modified in a separate mutable memory (GOT cells) - they are not >> embedded into AOT code. >> >> Changes includes klass fingerprint generation since we need it to find >> correct klass data in loaded AOT libraries. >> >> Thanks, >> Vladimir From cthalinger at twitter.com Thu Oct 27 17:55:08 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 27 Oct 2016 07:55:08 -1000 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> Message-ID: > On Oct 27, 2016, at 2:40 AM, Erik Joelsson wrote: > > > > On 2016-10-27 02:45, Vladimir Kozlov wrote: >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166416 >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > hotspot.m4: 296: Comment is misleading. Should just be removed. > > CompileJavaModules.gmk: Use of -g flag for java compilation is controlled globally. Please remove. There is a reason for that. Some debugging related Graal code makes use of classfile information to provide better information. Since this is Java and not C++ it *is* possible to have pleasant debugging experience even in product builds. I want this to be there. > > Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK conditional like all other buildtools targets. > >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > The extra exports from java.base needs to go in a new jdk/src/java.base/share/classes/module-info.java.extra since the module jdk.vm.compiler is optional. >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris automatically. No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and controlled globally. > > /Erik >> >> Please, review build changes for AOT. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >> >> Changes include new 'jaotc' launcher, makefile changes to build jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and hotspot/src/jdk.vm.compiler. >> 'jaotc' requires installed libelf package on a system to build native part of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of AOT compilation. >> >> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >> >> Thanks, >> Vladimir > From cthalinger at twitter.com Thu Oct 27 18:00:47 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 27 Oct 2016 08:00:47 -1000 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> Message-ID: > On Oct 27, 2016, at 4:14 AM, Vladimir Kozlov wrote: > > Thank you, Andrew, > > There is no official port of Graal for AArch64 yet. But the work seems started based on your mails on graal mailing list ;) > > There are several hardcoded x86 parts in JAOTC have to be fixed. For example: > > lines 76-82: > > http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CodeSectionProcessor.java.html > > lines 392-400: > > http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CompiledMethodInfo.java.html > > And jaotc ELF interface code. Btw. are we still planning to replace the ELF requirement with a Java solution? If yes, is there an RFE for that? > > Hotspot changes are not simple but platform specific mostly. > > Regards, > Vladimir > > On 10/27/16 2:09 AM, Andrew Haley wrote: >> On 27/10/16 02:15, Vladimir Kozlov wrote: >>> Please, review Hotspot VM part of AOT changes. >>> >>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >> >> Of course I'll port this to AArch64 asap. >> >> Andrew. >> From vladimir.kozlov at oracle.com Thu Oct 27 18:05:19 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 11:05:19 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> Message-ID: <581241DF.1050507@oracle.com> Done as you suggested. I will update webrevs later. One problem left. How to switch off ENABLE_AOT if we can't build libjelfshim.so (libelf not found or wrong version)? ENABLE_AOT and JVM_FEATURES_server are defined before LIBJELFSHIM_ENABLED is defined. Can I reset ENABLE_AOT (based on LIBJELFSHIM_ENABLED) in make/common/Modules.gmk (where AOT modules filtered out)? http://cr.openjdk.java.net/~kvn/aot/top.webrev/make/common/Modules.gmk.udiff.html Thanks, Vladimir On 10/27/16 8:45 AM, Volker Simonis wrote: > On Thu, Oct 27, 2016 at 2:22 PM, Volker Simonis > wrote: >> Hi, >> >> currently libelf support is checked on every 64-bit Linux and Solaris >> platform (see lib-elf.m4): >> >> if ((test "x$OPENJDK_TARGET_OS" = "xlinux" \ >> || test "x$OPENJDK_TARGET_OS" = "xsolaris") && \ >> (test "x$OPENJDK_TARGET_CPU_BITS" = "x64")); then >> LIBJELFSHIM_ENABLED="true" >> else >> LIBJELFSHIM_ENABLED="false" >> fi >> >> As far as I understand, this is only required for AOT and should >> therefore only be checked for if AOT is enabled. We could do something >> similar like for FFI (see libraries.m4): >> >> # Check if ffi is needed >> if HOTSPOT_CHECK_JVM_VARIANT(zero) || >> HOTSPOT_CHECK_JVM_VARIANT(zeroshark); then >> NEEDS_LIB_FFI=true >> else >> NEEDS_LIB_FFI=false >> fi >> >> And then test for "NEEDS_LIB_JELFSHIM" in lib-elf.m4 instead of >> testing for a specific platform. >> >> Otherwise we get ugly warning like: >> >> WARNING: Could not find libelf! Not building libjelfshim.so >> > > And I think in the case where we compile for AOT, we should print the > usual, platform-dependent hints like for example: > > libelf.so not found: try to run 'sudo apt-get install libelf-dev' > >> during configuration if we are running on platforms without AOT >> support or if AOT support is switched of. >> >> Regards, >> Volker >> >> >> >> On Thu, Oct 27, 2016 at 2:45 AM, Vladimir Kozlov >> wrote: >>> AOT JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>> Subtask: >>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>> Webrev: >>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> >>> Please, review build changes for AOT. Only Linux/x64 platform is supported. >>> 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>> >>> Changes include new 'jaotc' launcher, makefile changes to build >>> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and >>> hotspot/src/jdk.vm.compiler. >>> 'jaotc' requires installed libelf package on a system to build native part >>> of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of >>> AOT compilation. >>> >>> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >>> >>> Thanks, >>> Vladimir From cthalinger at twitter.com Thu Oct 27 18:07:36 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 27 Oct 2016 08:07:36 -1000 Subject: [9] RFR(L) 8166415: Integrate AOT tool JAOTC In-Reply-To: <581151ED.7080904@oracle.com> References: <581151ED.7080904@oracle.com> Message-ID: > On Oct 26, 2016, at 3:01 PM, Vladimir Kozlov wrote: > > AOT JEP: > https://bugs.openjdk.java.net/browse/JDK-8166089 > Subtask: > https://bugs.openjdk.java.net/browse/JDK-8166415 > Webrev: > http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ > > Please, review implementation of jaotc - Java Ahead-Of-Time static compiler. It produces native code for compiled java methods. It uses Graal as the code-generating backend and libelf for generating shared .so AOT libraries. It is written mostly in java. Code interface to libelf is written in C. > > Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. > > Corresponding build changes are reviewed in a separate thread for https://bugs.openjdk.java.net/browse/JDK-8166416 > > http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ I always planned on moving the tool into another package than com.oracle.graal.aot before integrating. com.oracle.graal.aot suggests that it is part of Graal but it?s just an application that is using Graal. I think we should follow what javac has: com.sun.tools.javac So, maybe com.oracle.tools.jaotc. > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 18:12:14 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 11:12:14 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> Message-ID: <5812437E.9070404@oracle.com> On 10/27/16 10:55 AM, Christian Thalinger wrote: > >> On Oct 27, 2016, at 2:40 AM, Erik Joelsson wrote: >> >> >> >> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>> AOT JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>> Subtask: >>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>> Webrev: >>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> hotspot.m4: 296: Comment is misleading. Should just be removed. >> >> CompileJavaModules.gmk: Use of -g flag for java compilation is controlled globally. Please remove. > > There is a reason for that. Some debugging related Graal code makes use of classfile information to provide better information. Since this is Java and not C++ it *is* possible to have pleasant debugging experience even in product builds. I want this to be there. Chris, do we need -g for JVMCI module too for that? And do we need -Xlint:-exports for jdk.vm.compiler (Graal)? FOR JVMCI it was added by: http://hg.openjdk.java.net/jdk9/hs/rev/81435a812f59 Thanks, Vladimir > >> >> Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK conditional like all other buildtools targets. >> >>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> The extra exports from java.base needs to go in a new jdk/src/java.base/share/classes/module-info.java.extra since the module jdk.vm.compiler is optional. >>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris automatically. No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and controlled globally. >> >> /Erik >>> >>> Please, review build changes for AOT. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>> >>> Changes include new 'jaotc' launcher, makefile changes to build jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and hotspot/src/jdk.vm.compiler. >>> 'jaotc' requires installed libelf package on a system to build native part of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of AOT compilation. >>> >>> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >>> >>> Thanks, >>> Vladimir >> > From doug.simon at oracle.com Thu Oct 27 18:18:24 2016 From: doug.simon at oracle.com (Doug Simon) Date: Thu, 27 Oct 2016 20:18:24 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <5812437E.9070404@oracle.com> References: <58114E0C.107@oracle.com> <5812437E.9070404@oracle.com> Message-ID: > On 27 Oct 2016, at 20:12, Vladimir Kozlov wrote: > > On 10/27/16 10:55 AM, Christian Thalinger wrote: >> >>> On Oct 27, 2016, at 2:40 AM, Erik Joelsson wrote: >>> >>> >>> >>> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>>> AOT JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>> Subtask: >>>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>>> Webrev: >>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>> hotspot.m4: 296: Comment is misleading. Should just be removed. >>> >>> CompileJavaModules.gmk: Use of -g flag for java compilation is controlled globally. Please remove. >> >> There is a reason for that. Some debugging related Graal code makes use of classfile information to provide better information. Since this is Java and not C++ it *is* possible to have pleasant debugging experience even in product builds. I want this to be there. > > Chris, do we need -g for JVMCI module too for that? I?m assuming Chris is referring to the use of debug info for snippet parameter names: https://github.com/graalvm/graal-core/blob/7c94891d06f08a635367df6078696b9388332f3b/graal/com.oracle.graal.replacements/src/com/oracle/graal/replacements/SnippetTemplate.java#L191 For that use case, -g is not necessary for JVMCI. That said, I think it?s always useful for Java code to be compiled with -g ;-) -Doug >> >>> >>> Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK conditional like all other buildtools targets. >>> >>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>> The extra exports from java.base needs to go in a new jdk/src/java.base/share/classes/module-info.java.extra since the module jdk.vm.compiler is optional. >>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris automatically. No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and controlled globally. >>> >>> /Erik >>>> >>>> Please, review build changes for AOT. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>> >>>> Changes include new 'jaotc' launcher, makefile changes to build jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and hotspot/src/jdk.vm.compiler. >>>> 'jaotc' requires installed libelf package on a system to build native part of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of AOT compilation. >>>> >>>> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >>>> >>>> Thanks, >>>> Vladimir From cthalinger at twitter.com Thu Oct 27 18:19:08 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 27 Oct 2016 08:19:08 -1000 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <5812437E.9070404@oracle.com> References: <58114E0C.107@oracle.com> <5812437E.9070404@oracle.com> Message-ID: > On Oct 27, 2016, at 8:12 AM, Vladimir Kozlov wrote: > > On 10/27/16 10:55 AM, Christian Thalinger wrote: >> >>> On Oct 27, 2016, at 2:40 AM, Erik Joelsson wrote: >>> >>> >>> >>> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>>> AOT JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>> Subtask: >>>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>>> Webrev: >>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>> hotspot.m4: 296: Comment is misleading. Should just be removed. >>> >>> CompileJavaModules.gmk: Use of -g flag for java compilation is controlled globally. Please remove. >> >> There is a reason for that. Some debugging related Graal code makes use of classfile information to provide better information. Since this is Java and not C++ it *is* possible to have pleasant debugging experience even in product builds. I want this to be there. > > Chris, do we need -g for JVMCI module too for that? Yes. > > And do we need -Xlint:-exports for jdk.vm.compiler (Graal)? FOR JVMCI it was added by: > > http://hg.openjdk.java.net/jdk9/hs/rev/81435a812f59 I can?t remember what the reason for -Xlint was but? wait. That was added 3 weeks ago. No clue then. > > Thanks, > Vladimir > >> >>> >>> Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK conditional like all other buildtools targets. >>> >>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>> The extra exports from java.base needs to go in a new jdk/src/java.base/share/classes/module-info.java.extra since the module jdk.vm.compiler is optional. >>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris automatically. No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and controlled globally. >>> >>> /Erik >>>> >>>> Please, review build changes for AOT. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>> >>>> Changes include new 'jaotc' launcher, makefile changes to build jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and hotspot/src/jdk.vm.compiler. >>>> 'jaotc' requires installed libelf package on a system to build native part of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of AOT compilation. >>>> >>>> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >>>> >>>> Thanks, >>>> Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 18:41:11 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 11:41:11 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> Message-ID: <58124A47.5010604@oracle.com> Thank you, Erik On 10/27/16 5:40 AM, Erik Joelsson wrote: > > On 2016-10-27 02:45, Vladimir Kozlov wrote: >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166416 >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > hotspot.m4: 296: Comment is misleading. Should just be removed. Removed. > CompileJavaModules.gmk: Use of -g flag for java compilation is controlled globally. Please remove. See Chris comment on this. We want to have JAOTC java debug information for debugging in product builds. > Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK conditional like all other buildtools targets. Done. > >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > The extra exports from java.base needs to go in a new jdk/src/java.base/share/classes/module-info.java.extra since the module jdk.vm.compiler is optional. Done. >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris automatically. Like this?: LDFLAGS := $(LDFLAGS_JDKLIB), \ LIBS := $(ELF_LIBS) $(LIBS_JDKLIB), \ > No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and controlled globally. Done. Thanks, Vladimir > > /Erik >> >> Please, review build changes for AOT. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >> >> Changes include new 'jaotc' launcher, makefile changes to build jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and hotspot/src/jdk.vm.compiler. >> 'jaotc' requires installed libelf package on a system to build native part of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of AOT compilation. >> >> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >> >> Thanks, >> Vladimir > From staffan.larsen at oracle.com Thu Oct 27 19:21:58 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 27 Oct 2016 21:21:58 +0200 Subject: Fwd: IntelliJ plugin for running jtreg tests References: <90b80926-2b5e-0738-80ff-6d28fa3034bc@oracle.com> Message-ID: <0E5A40C5-3FFF-4240-8AE4-DE7CFB657C19@oracle.com> Interesting if you use IntelliJ. > Begin forwarded message: > > From: Maurizio Cimadamore > Subject: IntelliJ plugin for running jtreg tests > Date: 27 October 2016 at 20:28:23 GMT+2 > To: jtreg-dev at openjdk.java.net, jtreg-use at openjdk.java.net > > Hi, > As you might have noted, Jon has recently pushed [1] the initial version of the IntelliJ jtreg plugin that Chris and I have been working hard to put together. This plugin allows you to run jtreg tests without leaving the IDE. As tests are executed, a green/red progress bar is displayed (similar to standard junit test support). Test output (jtr files) can also be recovered within the IDE. There are a number of options that can be configured when running tests - some global, some test-specific - this should allow you to fine tune jtreg execution to reproduce the combination of settings that you use today. > > In addition to test execution, this plugin also allows you to debug tests, which is a feature I find very handy for complex tests. > > Finally, this plugin interacts with the way in which IntelliJ discovers sources in your project - so with the plugin enabled, IntelliJ can 'understand' the sourcepath of a test, which will eliminate all the red squiggly lines from your tests. > > For a more detailed documentation of the plugin functionalities, as well as to how to build and install the plugin, please refer to this document [2]. > > Happy testing > Maurizio > > [1] - http://hg.openjdk.java.net/code-tools/jtreg/rev/b106e5bbe2e0 > [2] - http://hg.openjdk.java.net/code-tools/jtreg/file/b106e5bbe2e0/plugins/idea/README.md > > > > > From paul.sandoz at oracle.com Thu Oct 27 19:24:56 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Oct 2016 12:24:56 -0700 Subject: RFR 8163553 java.lang.LinkageError from test java/lang/ThreadGroup/Stop.java In-Reply-To: <5EA4A44D-3E66-4B76-8160-163580606FF1@oracle.com> References: <5EA4A44D-3E66-4B76-8160-163580606FF1@oracle.com> Message-ID: <8C56B19F-22EC-4E1E-AA47-E0D629231B07@oracle.com> Gentle reminder. Paul. > On 18 Oct 2016, at 11:41, Paul Sandoz wrote: > > Hi, > > Please review: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8163553-vh-mh-link-errors-not-wrapped/webrev/ > > This is the issue that motivated a change in the behaviour of indy wrapping Errors in BootstrapMethodError, JDK-8166974. I plan to push this issue with JDK-8166974 to hs, since they are related in behaviour even though there is no direct dependency between the patches. > > > When invoking signature-polymorphic methods a similar but hardcoded dance occurs, with an appeal to Java code, to link the call site. > > - MethodHandle.invoke/invokeExact (and the VH methods) would wrap all Errors in LinkageError. Now they are passed through, thus an Error like ThreadDeath is not wrapped. > > - MethodHandle.invoke/invokeExact/invokeBasic throw Throwable, and in certain cases the Throwable is wrapped in an InternalError. In many other cases Error and RuntimeException are propagated, which i think in general is the right pattern, so i consistently applied that. > > - I updated StringConcatFactory to also pass through Errors and avoid unduly wrapping StringConcatException in another instance of StringConcatException. (LambdaMetafactory and associated classes required no changes.) > > Thanks, > Paul. From vladimir.kozlov at oracle.com Thu Oct 27 19:52:16 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 12:52:16 -0700 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> Message-ID: <58125AF0.2000902@oracle.com> On 10/27/16 11:00 AM, Christian Thalinger wrote: > >> On Oct 27, 2016, at 4:14 AM, Vladimir Kozlov wrote: >> >> Thank you, Andrew, >> >> There is no official port of Graal for AArch64 yet. But the work seems started based on your mails on graal mailing list ;) >> >> There are several hardcoded x86 parts in JAOTC have to be fixed. For example: >> >> lines 76-82: >> >> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CodeSectionProcessor.java.html >> >> lines 392-400: >> >> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CompiledMethodInfo.java.html >> >> And jaotc ELF interface code. > > Btw. are we still planning to replace the ELF requirement with a Java solution? If yes, is there an RFE for that? Filed https://bugs.openjdk.java.net/browse/JDK-8168858 It is not clear what container format we should use or may be simple integrate with CDS. We start discussing it because we can't use ELF on all other platforms. Vladimir > >> >> Hotspot changes are not simple but platform specific mostly. >> >> Regards, >> Vladimir >> >> On 10/27/16 2:09 AM, Andrew Haley wrote: >>> On 27/10/16 02:15, Vladimir Kozlov wrote: >>>> Please, review Hotspot VM part of AOT changes. >>>> >>>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>> >>> Of course I'll port this to AArch64 asap. >>> >>> Andrew. >>> > From vladimir.kozlov at oracle.com Thu Oct 27 19:54:47 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 12:54:47 -0700 Subject: [9] RFR(L) 8166415: Integrate AOT tool JAOTC In-Reply-To: References: <581151ED.7080904@oracle.com> Message-ID: <58125B87.7030206@oracle.com> I like com.oracle.tools.jaotc. I will try it. Thanks, Vladimir On 10/27/16 11:07 AM, Christian Thalinger wrote: > >> On Oct 26, 2016, at 3:01 PM, Vladimir Kozlov wrote: >> >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166415 >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ >> >> Please, review implementation of jaotc - Java Ahead-Of-Time static compiler. It produces native code for compiled java methods. It uses Graal as the code-generating backend and libelf for generating shared .so AOT libraries. It is written mostly in java. Code interface to libelf is written in C. >> >> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >> >> Corresponding build changes are reviewed in a separate thread for https://bugs.openjdk.java.net/browse/JDK-8166416 >> >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > > I always planned on moving the tool into another package than com.oracle.graal.aot before integrating. com.oracle.graal.aot suggests that it is part of Graal but it?s just an application that is using Graal. > > I think we should follow what javac has: com.sun.tools.javac > > So, maybe com.oracle.tools.jaotc. > >> >> Thanks, >> Vladimir > From cthalinger at twitter.com Thu Oct 27 20:18:22 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 27 Oct 2016 10:18:22 -1000 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: <58125AF0.2000902@oracle.com> References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> <58125AF0.2000902@oracle.com> Message-ID: > On Oct 27, 2016, at 9:52 AM, Vladimir Kozlov wrote: > > On 10/27/16 11:00 AM, Christian Thalinger wrote: >> >>> On Oct 27, 2016, at 4:14 AM, Vladimir Kozlov wrote: >>> >>> Thank you, Andrew, >>> >>> There is no official port of Graal for AArch64 yet. But the work seems started based on your mails on graal mailing list ;) >>> >>> There are several hardcoded x86 parts in JAOTC have to be fixed. For example: >>> >>> lines 76-82: >>> >>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CodeSectionProcessor.java.html >>> >>> lines 392-400: >>> >>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CompiledMethodInfo.java.html >>> >>> And jaotc ELF interface code. >> >> Btw. are we still planning to replace the ELF requirement with a Java solution? If yes, is there an RFE for that? > > Filed https://bugs.openjdk.java.net/browse/JDK-8168858 > > It is not clear what container format we should use or may be simple integrate with CDS. We start discussing it because we can't use ELF on all other platforms. In my opinion (and John?s, IIRC) moving towards a CDS solution is the wrong direction. CDS has too many restrictions to work reliably for compiled code. I would much rather see CDS moving more towards DSOs. SubstrateVM is using some kind of Java library that is able to write DSOs for different operating systems. Maybe we can open source that library and start integrating it. > > Vladimir > >> >>> >>> Hotspot changes are not simple but platform specific mostly. >>> >>> Regards, >>> Vladimir >>> >>> On 10/27/16 2:09 AM, Andrew Haley wrote: >>>> On 27/10/16 02:15, Vladimir Kozlov wrote: >>>>> Please, review Hotspot VM part of AOT changes. >>>>> >>>>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>> >>>> Of course I'll port this to AArch64 asap. >>>> >>>> Andrew. From paul.sandoz at oracle.com Thu Oct 27 20:56:14 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 27 Oct 2016 13:56:14 -0700 Subject: [9] RFR(L) 8166415: Integrate AOT tool JAOTC In-Reply-To: <581151ED.7080904@oracle.com> References: <581151ED.7080904@oracle.com> Message-ID: Hi Vladimir, I think we should adjust to depend on the internal Unsafe and update the java.base module to export qualified to jdk.aot (like that for jdk.vm.compiler). It could easily be done later rather than now, if so I can log an issue. Paul. > On 26 Oct 2016, at 18:01, Vladimir Kozlov wrote: > > AOT JEP: > https://bugs.openjdk.java.net/browse/JDK-8166089 > Subtask: > https://bugs.openjdk.java.net/browse/JDK-8166415 > Webrev: > http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ > > Please, review implementation of jaotc - Java Ahead-Of-Time static compiler. It produces native code for compiled java methods. It uses Graal as the code-generating backend and libelf for generating shared .so AOT libraries. It is written mostly in java. Code interface to libelf is written in C. > > Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. > > Corresponding build changes are reviewed in a separate thread for https://bugs.openjdk.java.net/browse/JDK-8166416 > > http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Thu Oct 27 21:39:32 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 27 Oct 2016 14:39:32 -0700 Subject: [9] RFR(L) 8166415: Integrate AOT tool JAOTC In-Reply-To: References: <581151ED.7080904@oracle.com> Message-ID: <58127414.8020907@oracle.com> Thanks Paul, I think it is not difficult to adjust jaotc code to use internal Unsafe instead of sun.misc.Unsafe. Give me an example and I will do that. We have one UnsafeAccess.java class through which we do access: http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.jnilibelf/src/com/oracle/jnilibelf/UnsafeAccess.java.html Thanks, Vladimir On 10/27/16 1:56 PM, Paul Sandoz wrote: > Hi Vladimir, > > I think we should adjust to depend on the internal Unsafe and update the java.base module to export qualified to jdk.aot (like that for jdk.vm.compiler). It could easily be done later rather than now, if so I can log an issue. > > Paul. > >> On 26 Oct 2016, at 18:01, Vladimir Kozlov wrote: >> >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166415 >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ >> >> Please, review implementation of jaotc - Java Ahead-Of-Time static compiler. It produces native code for compiled java methods. It uses Graal as the code-generating backend and libelf for generating shared .so AOT libraries. It is written mostly in java. Code interface to libelf is written in C. >> >> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >> >> Corresponding build changes are reviewed in a separate thread for https://bugs.openjdk.java.net/browse/JDK-8166416 >> >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >> >> Thanks, >> Vladimir > From igor.veresov at oracle.com Thu Oct 27 21:44:10 2016 From: igor.veresov at oracle.com (Igor Veresov) Date: Thu, 27 Oct 2016 14:44:10 -0700 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> <58125AF0.2000902@oracle.com> Message-ID: It is very important to integrate with CDS in the future. We?re going to save tons of time if we could pre-link with the class archive. There would be substantially less class resolution and no indirection to get a klass. Same thing goes for heap serialization - direct pointers to strings and possibly other oops resulting from class initialization would be very beneficial. This all points in the the direction of an image format the is not an ELF. It would be just be much easier to do. igor > On Oct 27, 2016, at 1:18 PM, Christian Thalinger wrote: > >> >> On Oct 27, 2016, at 9:52 AM, Vladimir Kozlov > wrote: >> >> On 10/27/16 11:00 AM, Christian Thalinger wrote: >>> >>>> On Oct 27, 2016, at 4:14 AM, Vladimir Kozlov wrote: >>>> >>>> Thank you, Andrew, >>>> >>>> There is no official port of Graal for AArch64 yet. But the work seems started based on your mails on graal mailing list ;) >>>> >>>> There are several hardcoded x86 parts in JAOTC have to be fixed. For example: >>>> >>>> lines 76-82: >>>> >>>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CodeSectionProcessor.java.html >>>> >>>> lines 392-400: >>>> >>>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CompiledMethodInfo.java.html >>>> >>>> And jaotc ELF interface code. >>> >>> Btw. are we still planning to replace the ELF requirement with a Java solution? If yes, is there an RFE for that? >> >> Filed https://bugs.openjdk.java.net/browse/JDK-8168858 > >> >> It is not clear what container format we should use or may be simple integrate with CDS. We start discussing it because we can't use ELF on all other platforms. > > In my opinion (and John?s, IIRC) moving towards a CDS solution is the wrong direction. CDS has too many restrictions to work reliably for compiled code. I would much rather see CDS moving more towards DSOs. > > SubstrateVM is using some kind of Java library that is able to write DSOs for different operating systems. Maybe we can open source that library and start integrating it. > >> >> Vladimir >> >>> >>>> >>>> Hotspot changes are not simple but platform specific mostly. >>>> >>>> Regards, >>>> Vladimir >>>> >>>> On 10/27/16 2:09 AM, Andrew Haley wrote: >>>>> On 27/10/16 02:15, Vladimir Kozlov wrote: >>>>>> Please, review Hotspot VM part of AOT changes. >>>>>> >>>>>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>>> >>>>> Of course I'll port this to AArch64 asap. >>>>> >>>>> Andrew. From cthalinger at twitter.com Thu Oct 27 22:02:52 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 27 Oct 2016 12:02:52 -1000 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> <58125AF0.2000902@oracle.com> Message-ID: > On Oct 27, 2016, at 11:44 AM, Igor Veresov wrote: > > It is very important to integrate with CDS in the future. We?re going to save tons of time if we could pre-link with the class archive. There would be substantially less class resolution and no indirection to get a klass. Same thing goes for heap serialization - direct pointers to strings and possibly other oops resulting from class initialization would be very beneficial. I?m not disagreeing with any of this. > This all points in the the direction of an image format the is not an ELF. It would be just be much easier to do. John and I discussed this before we decided to go with standard formats. The biggest issue with the current CDS archive is that it?s not relocatable. That means if the address you want to map at is not available you?re out of luck. So, the proprietary CDS format needs to become relocatable; pretty much make it a DSO. And then you?re just reinventing the wheel. Also, every operating system had decades to tune loading and linking their format while you have to do it yourself. > > igor > >> On Oct 27, 2016, at 1:18 PM, Christian Thalinger > wrote: >> >>> >>> On Oct 27, 2016, at 9:52 AM, Vladimir Kozlov > wrote: >>> >>> On 10/27/16 11:00 AM, Christian Thalinger wrote: >>>> >>>>> On Oct 27, 2016, at 4:14 AM, Vladimir Kozlov > wrote: >>>>> >>>>> Thank you, Andrew, >>>>> >>>>> There is no official port of Graal for AArch64 yet. But the work seems started based on your mails on graal mailing list ;) >>>>> >>>>> There are several hardcoded x86 parts in JAOTC have to be fixed. For example: >>>>> >>>>> lines 76-82: >>>>> >>>>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CodeSectionProcessor.java.html >>>>> >>>>> lines 392-400: >>>>> >>>>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CompiledMethodInfo.java.html >>>>> >>>>> And jaotc ELF interface code. >>>> >>>> Btw. are we still planning to replace the ELF requirement with a Java solution? If yes, is there an RFE for that? >>> >>> Filed https://bugs.openjdk.java.net/browse/JDK-8168858 > >>> >>> It is not clear what container format we should use or may be simple integrate with CDS. We start discussing it because we can't use ELF on all other platforms. >> >> In my opinion (and John?s, IIRC) moving towards a CDS solution is the wrong direction. CDS has too many restrictions to work reliably for compiled code. I would much rather see CDS moving more towards DSOs. >> >> SubstrateVM is using some kind of Java library that is able to write DSOs for different operating systems. Maybe we can open source that library and start integrating it. >> >>> >>> Vladimir >>> >>>> >>>>> >>>>> Hotspot changes are not simple but platform specific mostly. >>>>> >>>>> Regards, >>>>> Vladimir >>>>> >>>>> On 10/27/16 2:09 AM, Andrew Haley wrote: >>>>>> On 27/10/16 02:15, Vladimir Kozlov wrote: >>>>>>> Please, review Hotspot VM part of AOT changes. >>>>>>> >>>>>>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>>>> >>>>>> Of course I'll port this to AArch64 asap. >>>>>> >>>>>> Andrew. From igor.veresov at oracle.com Thu Oct 27 22:40:23 2016 From: igor.veresov at oracle.com (Igor Veresov) Date: Thu, 27 Oct 2016 15:40:23 -0700 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> <58125AF0.2000902@oracle.com> Message-ID: <65F676D5-4BC4-4018-BF88-9C5EBB0E95EE@oracle.com> The whole point of having our own image format is that it won?t have to be relocatable. It?s hard to make CDS relocatable, you'd have to either do indirections or you?d have to do load-time relocations that would destroy sharing (most pages will be copied on write) and cause massive page-ins. Either alternative wastes time. Anyways, this all is pretty far down the road. igor > On Oct 27, 2016, at 3:02 PM, Christian Thalinger wrote: > > >> On Oct 27, 2016, at 11:44 AM, Igor Veresov > wrote: >> >> It is very important to integrate with CDS in the future. We?re going to save tons of time if we could pre-link with the class archive. There would be substantially less class resolution and no indirection to get a klass. Same thing goes for heap serialization - direct pointers to strings and possibly other oops resulting from class initialization would be very beneficial. > > I?m not disagreeing with any of this. > >> This all points in the the direction of an image format the is not an ELF. It would be just be much easier to do. > > John and I discussed this before we decided to go with standard formats. The biggest issue with the current CDS archive is that it?s not relocatable. That means if the address you want to map at is not available you?re out of luck. So, the proprietary CDS format needs to become relocatable; pretty much make it a DSO. And then you?re just reinventing the wheel. Also, every operating system had decades to tune loading and linking their format while you have to do it yourself. > >> >> igor >> >>> On Oct 27, 2016, at 1:18 PM, Christian Thalinger > wrote: >>> >>>> >>>> On Oct 27, 2016, at 9:52 AM, Vladimir Kozlov > wrote: >>>> >>>> On 10/27/16 11:00 AM, Christian Thalinger wrote: >>>>> >>>>>> On Oct 27, 2016, at 4:14 AM, Vladimir Kozlov > wrote: >>>>>> >>>>>> Thank you, Andrew, >>>>>> >>>>>> There is no official port of Graal for AArch64 yet. But the work seems started based on your mails on graal mailing list ;) >>>>>> >>>>>> There are several hardcoded x86 parts in JAOTC have to be fixed. For example: >>>>>> >>>>>> lines 76-82: >>>>>> >>>>>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CodeSectionProcessor.java.html >>>>>> >>>>>> lines 392-400: >>>>>> >>>>>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.graal.aot/src/com/oracle/graal/aot/CompiledMethodInfo.java.html >>>>>> >>>>>> And jaotc ELF interface code. >>>>> >>>>> Btw. are we still planning to replace the ELF requirement with a Java solution? If yes, is there an RFE for that? >>>> >>>> Filed https://bugs.openjdk.java.net/browse/JDK-8168858 > >>>> >>>> It is not clear what container format we should use or may be simple integrate with CDS. We start discussing it because we can't use ELF on all other platforms. >>> >>> In my opinion (and John?s, IIRC) moving towards a CDS solution is the wrong direction. CDS has too many restrictions to work reliably for compiled code. I would much rather see CDS moving more towards DSOs. >>> >>> SubstrateVM is using some kind of Java library that is able to write DSOs for different operating systems. Maybe we can open source that library and start integrating it. >>> >>>> >>>> Vladimir >>>> >>>>> >>>>>> >>>>>> Hotspot changes are not simple but platform specific mostly. >>>>>> >>>>>> Regards, >>>>>> Vladimir >>>>>> >>>>>> On 10/27/16 2:09 AM, Andrew Haley wrote: >>>>>>> On 27/10/16 02:15, Vladimir Kozlov wrote: >>>>>>>> Please, review Hotspot VM part of AOT changes. >>>>>>>> >>>>>>>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>>>>> >>>>>>> Of course I'll port this to AArch64 asap. >>>>>>> >>>>>>> Andrew. > From 1072213404 at qq.com Fri Oct 28 03:29:23 2016 From: 1072213404 at qq.com (=?gb18030?B?tvHB6cbvyr8=?=) Date: Fri, 28 Oct 2016 11:29:23 +0800 Subject: help understanding the "putLongRelease" implemetation in openjdk9 Message-ID: for jdk8 according to src/share/vm/classfile/vmSymbols.hpp do_intrinsic(_putOrderedLong, sun_misc_Unsafe, putOrderedLong_name, putOrderedLong_signature, F_RN) do_name( putOrderedLong_name, "putOrderedLong") i can find method "putOrderedLong" in src/share/vm/prims/unsafe.hpp, but for jdk9 src/share/vm/classfile/vmSymbols.hpp do_intrinsic(_getLongAcquire, jdk_internal_misc_Unsafe, getLongAcquire_name, getLong_signature, F_R) do_name(getLongAcquire_name, "getLongAcquire") do_name(putLongRelease_name, "putLongRelease") i can not find method "putLongRelease" in src/share/vm/prims/unsafe.hpp, why ? Is the method "putLongRelease" of interpreter userd version discarded? only reserve the implemetation of compiler used version? From mikael.vidstedt at oracle.com Fri Oct 28 04:04:05 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 27 Oct 2016 21:04:05 -0700 Subject: help understanding the "putLongRelease" implemetation in openjdk9 In-Reply-To: References: Message-ID: In jdk/src/java.base/share/classes/jdk/internal/misc/Unsafe.cpp you?ll find this: /** Release version of {@link #putLongVolatile(Object, long, long)} */ @HotSpotIntrinsicCandidate public final void putLongRelease(Object o, long offset, long x) { putLongVolatile(o, offset, x); } That is - in the default/interpreter case the putLongRelease method will delegate to the putLongVolatile method, and putLongVolatile in turn is in the default/interpreter case implemented in unsafe.cpp. However, the @HotSpotIntrinsicCandidate annotation is an indication that the JIT compiler handles the method differently, and if you have a look at hotspot/src/share/vm/opto/library_call.cpp you?ll find that putLongRelease, along with the other methods in the same family, get intensified and expanded during compilation using the inline_unsafe_access C++ method. Cheers, Mikael > On Oct 27, 2016, at 8:29 PM, ???? <1072213404 at qq.com> wrote: > > for jdk8 according to src/share/vm/classfile/vmSymbols.hpp > do_intrinsic(_putOrderedLong, sun_misc_Unsafe, putOrderedLong_name, putOrderedLong_signature, F_RN) > do_name( putOrderedLong_name, "putOrderedLong") > > > > i can find method "putOrderedLong" in src/share/vm/prims/unsafe.hpp, > > > but for jdk9 src/share/vm/classfile/vmSymbols.hpp > do_intrinsic(_getLongAcquire, jdk_internal_misc_Unsafe, getLongAcquire_name, getLong_signature, F_R) > do_name(getLongAcquire_name, "getLongAcquire") do_name(putLongRelease_name, "putLongRelease") > > > i can not find method "putLongRelease" in src/share/vm/prims/unsafe.hpp, > why ? > Is the method "putLongRelease" of interpreter userd version discarded? > only reserve the implemetation of compiler used version? From david.holmes at oracle.com Fri Oct 28 04:06:34 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Oct 2016 14:06:34 +1000 Subject: help understanding the "putLongRelease" implemetation in openjdk9 In-Reply-To: References: Message-ID: On 28/10/2016 1:29 PM, ???? wrote: > for jdk8 according to src/share/vm/classfile/vmSymbols.hpp > do_intrinsic(_putOrderedLong, sun_misc_Unsafe, putOrderedLong_name, putOrderedLong_signature, F_RN) > do_name( putOrderedLong_name, "putOrderedLong") > > > > i can find method "putOrderedLong" in src/share/vm/prims/unsafe.hpp, That must be the original JDK 8 code. The current JDK 8u code does not define unsafe.hpp, nor mention it in vmSymbols.hpp. > > but for jdk9 src/share/vm/classfile/vmSymbols.hpp > do_intrinsic(_getLongAcquire, jdk_internal_misc_Unsafe, getLongAcquire_name, getLong_signature, F_R) > do_name(getLongAcquire_name, "getLongAcquire") do_name(putLongRelease_name, "putLongRelease") > > > i can not find method "putLongRelease" in src/share/vm/prims/unsafe.hpp, > why ? > Is the method "putLongRelease" of interpreter userd version discarded? > only reserve the implemetation of compiler used version? The "new" unsafe.hpp defines the "interface" to some Unsafe related methods in the VM, called from within VM code. It doesn't define the set of methods that are the API of the Unsafe Java class. David ----- From mikael.vidstedt at oracle.com Fri Oct 28 05:08:31 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 27 Oct 2016 22:08:31 -0700 Subject: help understanding the "putLongRelease" implemetation in openjdk9 In-Reply-To: References: Message-ID: <2AD4B08F-B77A-4EF7-A811-30CF4E46B9F8@oracle.com> Adding back the hotspot-dev mailing list - please keep it on the cc: line. The JIT compiler only compiles methods which are ?hot? (which are being executed a lot). Before the methods are JIT compiled they are executed by the interpreter. The interpreter will *not* short-circuit the method, and will instead execute it exactly like it?s written, delegating to putLongVolatile which will take execution into unsafe.cpp/Unsafe_PutLongVolatile. Leaving the body of the method empty would mean that if the method is not JIT compiled, the interpreter would effectively execute a no-op, which of course will not have the desired effect. Cheers, Mikael > On Oct 27, 2016, at 9:51 PM, ???? <1072213404 at qq.com> wrote: > > > you already know that JIT compiler handles the method > 'public final void putLongRelease(Object o, long offset, long x) ' differently, > > why you still implemeted the method putLongRelease as > '@HotSpotIntrinsicCandidate > public final void putLongRelease(Object o, long offset, long x) { > putLongVolatile(o, offset, x); > }' > > Why can't it be a empty method like '@HotSpotIntrinsicCandidate > public final void putLongRelease(Object o, long offset, long x) { > > }' ? > > Is method putLongRelease used somewhere else not by JIT? > > > ------------------ ???? ------------------ > ???: "Mikael Vidstedt";; > ????: 2016?10?28?(???) ??12:04 > ???: "????"<1072213404 at qq.com>; > ??: "hotspot-dev"; > ??: Re: help understanding the "putLongRelease" implemetation in openjdk9 > > > In jdk/src/java.base/share/classes/jdk/internal/misc/Unsafe.cpp you?ll find this: > > /** Release version of {@link #putLongVolatile(Object, long, long)} */ > @HotSpotIntrinsicCandidate > public final void putLongRelease(Object o, long offset, long x) { > putLongVolatile(o, offset, x); > } > > That is - in the default/interpreter case the putLongRelease method will delegate to the putLongVolatile method, and putLongVolatile in turn is in the default/interpreter case implemented in unsafe.cpp. However, the @HotSpotIntrinsicCandidate annotation is an indication that the JIT compiler handles the method differently, and if you have a look at hotspot/src/share/vm/opto/library_call.cpp you?ll find that putLongRelease, along with the other methods in the same family, get intensified and expanded during compilation using the inline_unsafe_access C++ method. > > Cheers, > Mikael > > > On Oct 27, 2016, at 8:29 PM, ???? <1072213404 at qq.com> wrote: > > > > for jdk8 according to src/share/vm/classfile/vmSymbols.hpp > > do_intrinsic(_putOrderedLong, sun_misc_Unsafe, putOrderedLong_name, putOrderedLong_signature, F_RN) > > do_name( putOrderedLong_name, "putOrderedLong") > > > > > > > > i can find method "putOrderedLong" in src/share/vm/prims/unsafe.hpp, > > > > > > but for jdk9 src/share/vm/classfile/vmSymbols.hpp > > do_intrinsic(_getLongAcquire, jdk_internal_misc_Unsafe, getLongAcquire_name, getLong_signature, F_R) > > do_name(getLongAcquire_name, "getLongAcquire") do_name(putLongRelease_name, "putLongRelease") > > > > > > i can not find method "putLongRelease" in src/share/vm/prims/unsafe.hpp, > > why ? > > Is the method "putLongRelease" of interpreter userd version discarded? > > only reserve the implemetation of compiler used version? From mikael.vidstedt at oracle.com Fri Oct 28 05:46:48 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 27 Oct 2016 22:46:48 -0700 Subject: help understanding the "putLongRelease" implemetation in openjdk9 In-Reply-To: References: <2AD4B08F-B77A-4EF7-A811-30CF4E46B9F8@oracle.com> Message-ID: <3BD4F1B5-1FDE-4204-BF6E-82D8EDAFC84C@oracle.com> Adding back hotspot-dev again. Pretty please, keep the hotspot-dev mailing list on the cc: line - that is, please use ?reply all? instead of just ?reply?. Yes, that?s exactly right. In general, if the code is being executed in the interpreter it isn?t (yet) performance critical. In this specific case, the cost/overhead of going into the unsafe.cpp native code and back out again is significantly higher than the cost/savings of the barrier, so instead of having multiple separate versions of the code in native we simply make the methods fall back on the more heavy-weight barrier version. Only when JIT compiling the code, where we know that the code is hot and therefore likely performance critical, do we pick the best suited/cheapest barrier version. Cheers, Mikael > On Oct 27, 2016, at 10:34 PM, ???? <1072213404 at qq.com> wrote: > > > you means that if the method is not ?hot?, > even if using unsafe.cpp/Unsafe_PutLongVolatile which emits one more store-load barrier than the number of lazySet should emit, > the time costing will be almost equal , so there is no need putLongRelease using a individual implemetation, right? > > thank you , Mikael! > > Arron > > ------------------ ???? ------------------ > ???: "Mikael Vidstedt";; > ????: 2016?10?28?(???) ??1:08 > ???: "????"<1072213404 at qq.com>; > ??: "hotspot-dev"; > ??: Re: help understanding the "putLongRelease" implemetation in openjdk9 > > Adding back the hotspot-dev mailing list - please keep it on the cc: line. > > The JIT compiler only compiles methods which are ?hot? (which are being executed a lot). Before the methods are JIT compiled they are executed by the interpreter. The interpreter will *not* short-circuit the method, and will instead execute it exactly like it?s written, delegating to putLongVolatile which will take execution into unsafe.cpp/Unsafe_PutLongVolatile. Leaving the body of the method empty would mean that if the method is not JIT compiled, the interpreter would effectively execute a no-op, which of course will not have the desired effect. > > Cheers, > Mikael > >> On Oct 27, 2016, at 9:51 PM, ???? <1072213404 at qq.com > wrote: >> >> >> you already know that JIT compiler handles the method >> 'public final void putLongRelease(Object o, long offset, long x) ' differently, >> >> why you still implemeted the method putLongRelease as >> '@HotSpotIntrinsicCandidate >> public final void putLongRelease(Object o, long offset, long x) { >> putLongVolatile(o, offset, x); >> }' >> >> Why can't it be a empty method like '@HotSpotIntrinsicCandidate >> public final void putLongRelease(Object o, long offset, long x) { >> >> }' ? >> >> Is method putLongRelease used somewhere else not by JIT? >> >> >> ------------------ ???? ------------------ >> ???: "Mikael Vidstedt";>; >> ????: 2016?10?28?(???) ??12:04 >> ???: "????"<1072213404 at qq.com >; >> ??: "hotspot-dev">; >> ??: Re: help understanding the "putLongRelease" implemetation in openjdk9 >> >> >> In jdk/src/java.base/share/classes/jdk/internal/misc/Unsafe.cpp you?ll find this: >> >> /** Release version of {@link #putLongVolatile(Object, long, long)} */ >> @HotSpotIntrinsicCandidate >> public final void putLongRelease(Object o, long offset, long x) { >> putLongVolatile(o, offset, x); >> } >> >> That is - in the default/interpreter case the putLongRelease method will delegate to the putLongVolatile method, and putLongVolatile in turn is in the default/interpreter case implemented in unsafe.cpp. However, the @HotSpotIntrinsicCandidate annotation is an indication that the JIT compiler handles the method differently, and if you have a look at hotspot/src/share/vm/opto/library_call.cpp you?ll find that putLongRelease, along with the other methods in the same family, get intensified and expanded during compilation using the inline_unsafe_access C++ method. >> >> Cheers, >> Mikael >> >> > On Oct 27, 2016, at 8:29 PM, ???? <1072213404 at qq.com > wrote: >> > >> > for jdk8 according to src/share/vm/classfile/vmSymbols.hpp >> > do_intrinsic(_putOrderedLong, sun_misc_Unsafe, putOrderedLong_name, putOrderedLong_signature, F_RN) >> > do_name( putOrderedLong_name, "putOrderedLong") >> > >> > >> > >> > i can find method "putOrderedLong" in src/share/vm/prims/unsafe.hpp, >> > >> > >> > but for jdk9 src/share/vm/classfile/vmSymbols.hpp >> > do_intrinsic(_getLongAcquire, jdk_internal_misc_Unsafe, getLongAcquire_name, getLong_signature, F_R) >> > do_name(getLongAcquire_name, "getLongAcquire") do_name(putLongRelease_name, "putLongRelease") >> > >> > >> > i can not find method "putLongRelease" in src/share/vm/prims/unsafe.hpp, >> > why ? >> > Is the method "putLongRelease" of interpreter userd version discarded? >> > only reserve the implemetation of compiler used version? > From vladimir.kozlov at oracle.com Fri Oct 28 07:56:38 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 00:56:38 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <58124A47.5010604@oracle.com> References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> Message-ID: <581304B6.7030804@oracle.com> Webrevs updated in place (please, reload webpage to get new version). generated-configure.sh changes are included. http://cr.openjdk.java.net/~kvn/aot/top.webrev/ http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ libelfshim generation is guarded by ENABLE_AOT as Volker suggested. I did not figure out how switch off ENABLE_AOT if libelf not found. So I changed warning to configuration error: error: libelf not found, cannot build AOT. Disable AOT build: --enable-aot=no. You might be able to fix this by running 'sudo yum install elfutils-libelf-devel'. Which is not correct since AOT build is enabled by default on linux-x64. Need help with this. AOT modules dependencies moved to new module-info.java.extra. And other changes suggested by Erik. Thanks, Vladimir On 10/27/16 11:41 AM, Vladimir Kozlov wrote: > Thank you, Erik > > On 10/27/16 5:40 AM, Erik Joelsson wrote: >> >> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>> AOT JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>> Subtask: >>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>> Webrev: >>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> hotspot.m4: 296: Comment is misleading. Should just be removed. > > Removed. > >> CompileJavaModules.gmk: Use of -g flag for java compilation is controlled globally. Please remove. > > See Chris comment on this. We want to have JAOTC java debug information for debugging in product builds. > >> Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK conditional like all other buildtools targets. > > Done. > >> >>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> The extra exports from java.base needs to go in a new jdk/src/java.base/share/classes/module-info.java.extra since the module jdk.vm.compiler is optional. > > Done. > >>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris automatically. > > Like this?: > > LDFLAGS := $(LDFLAGS_JDKLIB), \ > LIBS := $(ELF_LIBS) $(LIBS_JDKLIB), \ > >> No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and controlled globally. > > Done. > > Thanks, > Vladimir > >> >> /Erik >>> >>> Please, review build changes for AOT. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>> >>> Changes include new 'jaotc' launcher, makefile changes to build jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and hotspot/src/jdk.vm.compiler. >>> 'jaotc' requires installed libelf package on a system to build native part of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of AOT compilation. >>> >>> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >>> >>> Thanks, >>> Vladimir >> From vladimir.kozlov at oracle.com Fri Oct 28 07:56:44 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 00:56:44 -0700 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: <58123752.8060308@oracle.com> References: <58115536.5080205@oracle.com> <58123752.8060308@oracle.com> Message-ID: <581304BC.6090302@oracle.com> Webrevs updated in place (please, reload webpage to get new version). http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ Included ppc64 and s390 changes. Corrected .hgignore in the hotspot.patch. Thanks, Vladimir On 10/27/16 10:20 AM, Vladimir Kozlov wrote: > Thank you, Volker > > I will integrate ppc and s390 changes after I merge jdk9/hs into AOT repo. > And I will look on .hgignore. > > I use 'hg import --no-commit AOT.patch' command to apply patches. > > Thanks, > Vladimir > > On 10/27/16 9:42 AM, Volker Simonis wrote: >> Hi, >> >> first of all I had some problems importing the patch from the webrev >> because of this sequence: >> >> --- old/./.hgignore 2016-10-26 14:56:22.000000000 -0700 >> +++ new/./.hgignore 2016-10-26 14:56:22.000000000 -0700 >> @@ -24,3 +24,19 @@ >> ^test/compiler/jvmci/\w[\w\.]*/.*\.iml >> ^test/compiler/jvmci/\w[\w\.]*/nbproject >> >> The error (when using hg qpush -v) was: >> >> applying 8166413_AOT.patch >> path './.hgignore' is inside nested repo '.' >> committing changelog >> patch failed, rejects left in working directory >> errors during apply, please fix and qrefresh 8166413_AOT.patch >> >> I don't know how it was generated, but I could easily fix it by >> replacing the above sequence by: >> >> --- old/.hgignore 2016-10-26 14:56:22.000000000 -0700 >> +++ new/.hgignore 2016-10-26 14:56:22.000000000 -0700 >> >> >> Besides that, it would be great if you could integrate the followinf >> ppc/s390 specific changes which are required to keep the corresponding >> ports building: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/8166413_ppc_s390_addon/ >> >> Thanks, >> Volker >> >> >> On Thu, Oct 27, 2016 at 3:15 AM, Vladimir Kozlov >> wrote: >>> AOT JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>> Subtask: >>> https://bugs.openjdk.java.net/browse/JDK-8166415 >>> Webrev: >>> http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ >>> >>> Please, review Hotspot VM part of AOT changes. >>> >>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will >>> be build only on Linux/x64. >>> >>> AOT code is NOT linked during AOT libraries load as it happens with normal >>> .so libraries. AOT code entry points are not exposed (not global) in AOT >>> libraries. Only class data has global labels which we look for with >>> dlsym(klass_name). >>> >>> AOT-compiled code in AOT libraries is treated by JVM as *extension* of >>> existing CodeCache. When a java class is loaded JVM looks if corresponding >>> AOT-compiled methods exist in loaded AOT libraries and add links to them >>> from java methods descriptors (we have new field Method::_aot_code). >>> AOT-compiled code follows the same invocation/deoptimization/unloading rules >>> as normal JIT-compiled code. >>> >>> Calls in AOT code use the same methods resolution runtime code as calls in >>> JITed code. The difference is call's destination address is loaded >>> indirectly because we can't patch AOT code - it is immutable (to share >>> between multiple JVM instances). >>> >>> Classes and Strings referenced in AOT code are resolved lazily by calling >>> into runtime. All mutable pointers (oops (mostly strings), metadata) are >>> stored and modified in a separate mutable memory (GOT cells) - they are not >>> embedded into AOT code. >>> >>> Changes includes klass fingerprint generation since we need it to find >>> correct klass data in loaded AOT libraries. >>> >>> Thanks, >>> Vladimir From vladimir.kozlov at oracle.com Fri Oct 28 07:57:06 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 00:57:06 -0700 Subject: [9] RFR(L) 8166415: Integrate AOT tool JAOTC In-Reply-To: <58127414.8020907@oracle.com> References: <581151ED.7080904@oracle.com> <58127414.8020907@oracle.com> Message-ID: <581304D2.5060407@oracle.com> Webrevs updated in place (please, reload webpage to get new version). http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ jaotc tool package was renamed from com.oracle.graal.aot to jdk.tools.jaotc similar to other new jdk tools (jlink). Switched to new internal Unsafe instead of old sun.misc Unsafe. Thanks, Vladimir On 10/27/16 2:39 PM, Vladimir Kozlov wrote: > Thanks Paul, > > I think it is not difficult to adjust jaotc code to use internal Unsafe instead of sun.misc.Unsafe. > Give me an example and I will do that. > > We have one UnsafeAccess.java class through which we do access: > > http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.jnilibelf/src/com/oracle/jnilibelf/UnsafeAccess.java.html > > Thanks, > Vladimir > On 10/27/16 1:56 PM, Paul Sandoz wrote: >> Hi Vladimir, >> >> I think we should adjust to depend on the internal Unsafe and update the java.base module to export qualified to jdk.aot (like that for jdk.vm.compiler). It could easily be done later rather than >> now, if so I can log an issue. >> >> Paul. >> >>> On 26 Oct 2016, at 18:01, Vladimir Kozlov wrote: >>> >>> AOT JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>> Subtask: >>> https://bugs.openjdk.java.net/browse/JDK-8166415 >>> Webrev: >>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ >>> >>> Please, review implementation of jaotc - Java Ahead-Of-Time static compiler. It produces native code for compiled java methods. It uses Graal as the code-generating backend and libelf for >>> generating shared .so AOT libraries. It is written mostly in java. Code interface to libelf is written in C. >>> >>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>> >>> Corresponding build changes are reviewed in a separate thread for https://bugs.openjdk.java.net/browse/JDK-8166416 >>> >>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> >>> Thanks, >>> Vladimir >> From vladimir.kozlov at oracle.com Fri Oct 28 08:00:02 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 01:00:02 -0700 Subject: [9] RFR(L) 8166417: Graal-core into JDK for AOT compiler In-Reply-To: <5812295F.4000107@oracle.com> References: <5812295F.4000107@oracle.com> Message-ID: <58130582.3000108@oracle.com> Webrevs updated in place (please, reload webpage to get new version). http://hg.openjdk.java.net/graal/graal-core/ Synced with latest graal-core changes: changeset: 14134:21c7cc9e9b99 tag: tip parent: 14129:0481cd5ef3bb parent: 14133:65baeca1cf31 user: Tom Rodriguez date: Thu Oct 27 13:13:27 2016 -0700 summary: Merge: InliningUtil.inline should be responsible for recording inlines Thanks, Vladimir On 10/27/16 9:20 AM, Vladimir Kozlov wrote: > AOT JEP: > https://bugs.openjdk.java.net/browse/JDK-8166089 > Subtask: > https://bugs.openjdk.java.net/browse/JDK-8166417 > Graal-core: > http://hg.openjdk.java.net/graal/graal-core/ > Webrev: > http://cr.openjdk.java.net/~kvn/aot/graal.webrev/ > > This is last part of AOT project. AOT requires Graal-core to be part of JDK. > AOT compiler uses Graal-core as backend compiler - we use its ability to generate immutable PIC code. > > We are working with Oracle's Labs on this integration. > > Meanwhile, if you want to experiment do next: > > hg clone http://hg.openjdk.java.net/graal/graal-core/ > > apply patch from graal.webrev > > cp graal-core/graal myjdk/hotspot/src/jdk.vm.compiler > > After that you can build JDK with AOT. > > Thanks, > Vladimir From aph at redhat.com Fri Oct 28 08:11:04 2016 From: aph at redhat.com (Andrew Haley) Date: Fri, 28 Oct 2016 09:11:04 +0100 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> <58125AF0.2000902@oracle.com> Message-ID: On 27/10/16 23:02, Christian Thalinger wrote: > John and I discussed this before we decided to go with standard > formats. The biggest issue with the current CDS archive is that > it?s not relocatable. That means if the address you want to map at > is not available you?re out of luck. Is that really an issue in practice? > So, the proprietary CDS format needs to become relocatable; pretty > much make it a DSO. And then you?re just reinventing the wheel. > Also, every operating system had decades to tune loading and linking > their format while you have to do it yourself. They have, but what they have is still not so efficient: think about GOTs, PLTs, etc. These work, but we do better in JIT-generated code. In other words, being as good as existing DSO formats is a very low bar. Andrew. From erik.joelsson at oracle.com Fri Oct 28 08:34:07 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 28 Oct 2016 10:34:07 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> <5812437E.9070404@oracle.com> Message-ID: On 2016-10-27 20:18, Doug Simon wrote: >> On 27 Oct 2016, at 20:12, Vladimir Kozlov wrote: >> >> On 10/27/16 10:55 AM, Christian Thalinger wrote: >>>> On Oct 27, 2016, at 2:40 AM, Erik Joelsson wrote: >>>> >>>> >>>> >>>> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>>>> AOT JEP: >>>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>>> Subtask: >>>>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>>> hotspot.m4: 296: Comment is misleading. Should just be removed. >>>> >>>> CompileJavaModules.gmk: Use of -g flag for java compilation is controlled globally. Please remove. >>> There is a reason for that. Some debugging related Graal code makes use of classfile information to provide better information. Since this is Java and not C++ it *is* possible to have pleasant debugging experience even in product builds. I want this to be there. >> Chris, do we need -g for JVMCI module too for that? > I?m assuming Chris is referring to the use of debug info for snippet parameter names: > > https://github.com/graalvm/graal-core/blob/7c94891d06f08a635367df6078696b9388332f3b/graal/com.oracle.graal.replacements/src/com/oracle/graal/replacements/SnippetTemplate.java#L191 > > For that use case, -g is not necessary for JVMCI. That said, I think it?s always useful for Java code to be compiled with -g ;-) I'm sure most people do. To my knowledge, -g is disabled by default for footprint reasons in the JRE. This has been up for discussion before and I think the conclusion was that ideally we want -g in the JDK image but not in the JRE image. That has not been possible before (without compiling everything twice), but maybe jlink is able to strip the debug info from java classes now? In that case we could consider globally enabling -g for product builds. For now if you have a special case for needing -g on specific modules, at the very least provide a comment explaining why. Otherwise it will just be cleaned away in the future. /Erik > -Doug > >>>> Main.gmk: buildtools-hotspot should be declared inside the CREATING_BUILDJDK conditional like all other buildtools targets. >>>> >>>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>>> The extra exports from java.base needs to go in a new jdk/src/java.base/share/classes/module-info.java.extra since the module jdk.vm.compiler is optional. >>>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>>> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris automatically. No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be correct and controlled globally. >>>> >>>> /Erik >>>>> Please, review build changes for AOT. Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>>> >>>>> Changes include new 'jaotc' launcher, makefile changes to build jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>>>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and hotspot/src/jdk.vm.compiler. >>>>> 'jaotc' requires installed libelf package on a system to build native part of 'jaotc'. It is used to generated AOT shared libraries (.so) as result of AOT compilation. >>>>> >>>>> Hotspot makefile changes will be pushed together with Hotspot AOT changes. >>>>> >>>>> Thanks, >>>>> Vladimir From Alan.Bateman at oracle.com Fri Oct 28 08:43:42 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 28 Oct 2016 09:43:42 +0100 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> <5812437E.9070404@oracle.com> Message-ID: On 28/10/2016 09:34, Erik Joelsson wrote: > > but maybe jlink is able to strip the debug info from java classes now? > In that case we could consider globally enabling -g for product builds. Yes, it can, `jlink --strip-debug` (or `-G`). A good test would be to build with `javac -g` and then strip the debug information when generating the JRE image to see that it comes out at around the same size as the current build. -Alan. From erik.joelsson at oracle.com Fri Oct 28 08:57:45 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 28 Oct 2016 10:57:45 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <581304B6.7030804@oracle.com> References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> Message-ID: <0acdf13c-88d9-f904-7fcb-1f5d3683e30c@oracle.com> Hello, On 2016-10-28 09:56, Vladimir Kozlov wrote: > Webrevs updated in place (please, reload webpage to get new version). > generated-configure.sh changes are included. > > http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > > libelfshim generation is guarded by ENABLE_AOT as Volker suggested. > > I did not figure out how switch off ENABLE_AOT if libelf not found. So > I changed warning to configuration error: > > error: libelf not found, cannot build AOT. Disable AOT build: > --enable-aot=no. You might be able to fix this by running 'sudo yum > install elfutils-libelf-devel'. > > Which is not correct since AOT build is enabled by default on > linux-x64. Need help with this. > I think it should be fine moving HOTSPOT_SETUP_JVM_FEATURES to after LIB_SETUP_LIBRARIES in configure.ac. (Note that many of these calls are just ordered as they happened to be, though some really need to be before or after others. I cannot find a reason for these two.) Then you can enable/disable AOT after looking for libelf. It depends on what kind of behavior you want. If libelf isn't commonly installed, then having configure use it, and enable AOT if it is, and semi silently just disabling AOT if it's not is fine. We can define our Oracle builds to explicitly set --enable-aot to get an early fail on a missing dependency. That's inline with current configure behavior so I like that. Another note. We usually use the style --disable- rather than --enable-=no when referring to configure options. /Erik > AOT modules dependencies moved to new module-info.java.extra. > > And other changes suggested by Erik. > > Thanks, > Vladimir > > On 10/27/16 11:41 AM, Vladimir Kozlov wrote: >> Thank you, Erik >> >> On 10/27/16 5:40 AM, Erik Joelsson wrote: >>> >>> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>>> AOT JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>> Subtask: >>>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>>> Webrev: >>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>> hotspot.m4: 296: Comment is misleading. Should just be removed. >> >> Removed. >> >>> CompileJavaModules.gmk: Use of -g flag for java compilation is >>> controlled globally. Please remove. >> >> See Chris comment on this. We want to have JAOTC java debug >> information for debugging in product builds. >> >>> Main.gmk: buildtools-hotspot should be declared inside the >>> CREATING_BUILDJDK conditional like all other buildtools targets. >> >> Done. >> >>> >>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>> The extra exports from java.base needs to go in a new >>> jdk/src/java.base/share/classes/module-info.java.extra since the >>> module jdk.vm.compiler is optional. >> >> Done. >> >>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add >>> $(LIBS_JDKLIB) to LIBS since that will provide -lc on Solaris >>> automatically. >> >> Like this?: >> >> LDFLAGS := $(LDFLAGS_JDKLIB), \ >> LIBS := $(ELF_LIBS) $(LIBS_JDKLIB), \ >> >>> No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should >>> be correct and controlled globally. >> >> Done. >> >> Thanks, >> Vladimir >> >>> >>> /Erik >>>> >>>> Please, review build changes for AOT. Only Linux/x64 platform is >>>> supported. 'jaotc' and AOT part of Hotspot will be build only on >>>> Linux/x64. >>>> >>>> Changes include new 'jaotc' launcher, makefile changes to build >>>> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot >>>> and hotspot/src/jdk.vm.compiler. >>>> 'jaotc' requires installed libelf package on a system to build >>>> native part of 'jaotc'. It is used to generated AOT shared >>>> libraries (.so) as result of AOT compilation. >>>> >>>> Hotspot makefile changes will be pushed together with Hotspot AOT >>>> changes. >>>> >>>> Thanks, >>>> Vladimir >>> From volker.simonis at gmail.com Fri Oct 28 09:09:00 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 28 Oct 2016 11:09:00 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <0acdf13c-88d9-f904-7fcb-1f5d3683e30c@oracle.com> References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <0acdf13c-88d9-f904-7fcb-1f5d3683e30c@oracle.com> Message-ID: On Fri, Oct 28, 2016 at 10:57 AM, Erik Joelsson wrote: > Hello, > > > On 2016-10-28 09:56, Vladimir Kozlov wrote: >> >> Webrevs updated in place (please, reload webpage to get new version). >> generated-configure.sh changes are included. >> >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >> >> libelfshim generation is guarded by ENABLE_AOT as Volker suggested. >> >> I did not figure out how switch off ENABLE_AOT if libelf not found. So I >> changed warning to configuration error: >> >> error: libelf not found, cannot build AOT. Disable AOT build: >> --enable-aot=no. You might be able to fix this by running 'sudo yum install >> elfutils-libelf-devel'. >> >> Which is not correct since AOT build is enabled by default on linux-x64. >> Need help with this. >> > I think it should be fine moving HOTSPOT_SETUP_JVM_FEATURES to after > LIB_SETUP_LIBRARIES in configure.ac. (Note that many of these calls are just > ordered as they happened to be, though some really need to be before or > after others. I cannot find a reason for these two.) Then you can > enable/disable AOT after looking for libelf. It depends on what kind of > behavior you want. > > If libelf isn't commonly installed, then having configure use it, and enable > AOT if it is, and semi silently just disabling AOT if it's not is fine. We > can define our Oracle builds to explicitly set --enable-aot to get an early > fail on a missing dependency. That's inline with current configure behavior > so I like that. I'm fine with that. But then we shouldn't warn if we don't find libelf. We should only warn, if '--enable-aot' was explicitly requested by the user and if this request can't be fulfilled because of the missing libelf dependency. I also noticed that currently the configuration for AOT bails out early, because it can not find the "aot src": + AC_MSG_CHECKING([if aot src is present]) + if test -d "$HOTSPOT_TOPDIR/src/jdk.aot"; then + AC_MSG_RESULT([yes]) But I couldn't find the patch which provides "$HOTSPOT_TOPDIR/src/jdk.aot". Should this read "$HOTSPOT_TOPDIR/src/jdk.vm.compiler" which is the location where the graal sources will be copied to (see RFR for "8166417: Graal-core into JDK for AOT compiler")? Or is there yet another patch to come which will provide "$HOTSPOT_TOPDIR/src/jdk.aot"? > > Another note. We usually use the style --disable- rather than > --enable-=no when referring to configure options. > > /Erik > >> AOT modules dependencies moved to new module-info.java.extra. >> >> And other changes suggested by Erik. >> >> Thanks, >> Vladimir >> >> On 10/27/16 11:41 AM, Vladimir Kozlov wrote: >>> >>> Thank you, Erik >>> >>> On 10/27/16 5:40 AM, Erik Joelsson wrote: >>>> >>>> >>>> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>>>> >>>>> AOT JEP: >>>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>>> Subtask: >>>>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>>> >>>> hotspot.m4: 296: Comment is misleading. Should just be removed. >>> >>> >>> Removed. >>> >>>> CompileJavaModules.gmk: Use of -g flag for java compilation is >>>> controlled globally. Please remove. >>> >>> >>> See Chris comment on this. We want to have JAOTC java debug information >>> for debugging in product builds. >>> >>>> Main.gmk: buildtools-hotspot should be declared inside the >>>> CREATING_BUILDJDK conditional like all other buildtools targets. >>> >>> >>> Done. >>> >>>> >>>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>>> >>>> The extra exports from java.base needs to go in a new >>>> jdk/src/java.base/share/classes/module-info.java.extra since the module >>>> jdk.vm.compiler is optional. >>> >>> >>> Done. >>> >>>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>>> >>>> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) >>>> to LIBS since that will provide -lc on Solaris automatically. >>> >>> >>> Like this?: >>> >>> LDFLAGS := $(LDFLAGS_JDKLIB), \ >>> LIBS := $(ELF_LIBS) $(LIBS_JDKLIB), \ >>> >>>> No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be >>>> correct and controlled globally. >>> >>> >>> Done. >>> >>> Thanks, >>> Vladimir >>> >>>> >>>> /Erik >>>>> >>>>> >>>>> Please, review build changes for AOT. Only Linux/x64 platform is >>>>> supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>>> >>>>> Changes include new 'jaotc' launcher, makefile changes to build >>>>> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>>>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and >>>>> hotspot/src/jdk.vm.compiler. >>>>> 'jaotc' requires installed libelf package on a system to build native >>>>> part of 'jaotc'. It is used to generated AOT shared libraries (.so) as >>>>> result of AOT compilation. >>>>> >>>>> Hotspot makefile changes will be pushed together with Hotspot AOT >>>>> changes. >>>>> >>>>> Thanks, >>>>> Vladimir >>>> >>>> > From volker.simonis at gmail.com Fri Oct 28 09:20:06 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 28 Oct 2016 11:20:06 +0200 Subject: [9] RFR(L) 8166417: Graal-core into JDK for AOT compiler In-Reply-To: <5812295F.4000107@oracle.com> References: <5812295F.4000107@oracle.com> Message-ID: On Thu, Oct 27, 2016 at 6:20 PM, Vladimir Kozlov wrote: > AOT JEP: > https://bugs.openjdk.java.net/browse/JDK-8166089 > Subtask: > https://bugs.openjdk.java.net/browse/JDK-8166417 > Graal-core: > http://hg.openjdk.java.net/graal/graal-core/ > Webrev: > http://cr.openjdk.java.net/~kvn/aot/graal.webrev/ > > This is last part of AOT project. AOT requires Graal-core to be part of JDK. > AOT compiler uses Graal-core as backend compiler - we use its ability to > generate immutable PIC code. > > We are working with Oracle's Labs on this integration. > > Meanwhile, if you want to experiment do next: > > hg clone http://hg.openjdk.java.net/graal/graal-core/ > > apply patch from graal.webrev > > cp graal-core/graal myjdk/hotspot/src/jdk.vm.compiler > The convention is that the directory name under "src/" (i.e. "jdk.vm.ci") corresponds to the package name of the contained sources (i.e. "jdk.vm.ci/share/classes/jdk.vm.ci.code/src/jdk/vm/ci/code"). Following this convention, the graal sources should either be copied to myjdk/hotspot/src/com.oracle.graal/share/classes or the graal packe should be renamed to jdk.vm.compiler. And then there's still the package com.oracle.nfi which would require a separate sub-dir under hotspot/src if it wouldn't be moved into com.oracle.graal... > After that you can build JDK with AOT. > > Thanks, > Vladimir From erik.joelsson at oracle.com Fri Oct 28 10:27:30 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 28 Oct 2016 12:27:30 +0200 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <0acdf13c-88d9-f904-7fcb-1f5d3683e30c@oracle.com> Message-ID: On 2016-10-28 11:09, Volker Simonis wrote: > On Fri, Oct 28, 2016 at 10:57 AM, Erik Joelsson > wrote: >> Hello, >> >> >> On 2016-10-28 09:56, Vladimir Kozlov wrote: >>> Webrevs updated in place (please, reload webpage to get new version). >>> generated-configure.sh changes are included. >>> >>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> >>> libelfshim generation is guarded by ENABLE_AOT as Volker suggested. >>> >>> I did not figure out how switch off ENABLE_AOT if libelf not found. So I >>> changed warning to configuration error: >>> >>> error: libelf not found, cannot build AOT. Disable AOT build: >>> --enable-aot=no. You might be able to fix this by running 'sudo yum install >>> elfutils-libelf-devel'. >>> >>> Which is not correct since AOT build is enabled by default on linux-x64. >>> Need help with this. >>> >> I think it should be fine moving HOTSPOT_SETUP_JVM_FEATURES to after >> LIB_SETUP_LIBRARIES in configure.ac. (Note that many of these calls are just >> ordered as they happened to be, though some really need to be before or >> after others. I cannot find a reason for these two.) Then you can >> enable/disable AOT after looking for libelf. It depends on what kind of >> behavior you want. >> >> If libelf isn't commonly installed, then having configure use it, and enable >> AOT if it is, and semi silently just disabling AOT if it's not is fine. We >> can define our Oracle builds to explicitly set --enable-aot to get an early >> fail on a missing dependency. That's inline with current configure behavior >> so I like that. > I'm fine with that. But then we shouldn't warn if we don't find > libelf. We should only warn, if '--enable-aot' was explicitly > requested by the user and if this request can't be fulfilled because > of the missing libelf dependency. --enable-aot and not finding libelf should result in an error. If on linux-x64 where aot is default enabled, there should be a warning that it was disabled because libelf is missing. This warning would now need to be in the ENABLE_AOT logic. /Erik From adinn at redhat.com Fri Oct 28 11:04:25 2016 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 28 Oct 2016 12:04:25 +0100 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <581304B6.7030804@oracle.com> References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> Message-ID: <9f52669e-75fe-7139-78cb-d28ae1aff0a7@redhat.com> On 28/10/16 08:56, Vladimir Kozlov wrote: > Webrevs updated in place (please, reload webpage to get new version). > generated-configure.sh changes are included. > > http://cr.openjdk.java.net/~kvn/aot/top.webrev/ > http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ I have applied these 3 change sets to the latest hs on AArch64 and it build and runs ok (perhaps not much of a surprise). However, I'm not sure exactly what this proves -- or, rather, whether that is all that you want proved in order to proceed with 8166416. 1) Do I need to use any specific configure options to exercise these specific changes satisfactorily on AArch64? 2) Do I need to include any further changes in order to test more fully? regards, Andrew Dinn ----------- From vladimir.kozlov at oracle.com Fri Oct 28 15:45:06 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 08:45:06 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <0acdf13c-88d9-f904-7fcb-1f5d3683e30c@oracle.com> Message-ID: <58137282.4000700@oracle.com> Volker, Patch for aot sources is tracked by: 8166415: Integrate AOT tool JAOTC http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ JAOTC code is in Hotspot, not in JDK, repo. Vladimir On 10/28/16 2:09 AM, Volker Simonis wrote: > On Fri, Oct 28, 2016 at 10:57 AM, Erik Joelsson > wrote: >> Hello, >> >> >> On 2016-10-28 09:56, Vladimir Kozlov wrote: >>> >>> Webrevs updated in place (please, reload webpage to get new version). >>> generated-configure.sh changes are included. >>> >>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> >>> libelfshim generation is guarded by ENABLE_AOT as Volker suggested. >>> >>> I did not figure out how switch off ENABLE_AOT if libelf not found. So I >>> changed warning to configuration error: >>> >>> error: libelf not found, cannot build AOT. Disable AOT build: >>> --enable-aot=no. You might be able to fix this by running 'sudo yum install >>> elfutils-libelf-devel'. >>> >>> Which is not correct since AOT build is enabled by default on linux-x64. >>> Need help with this. >>> >> I think it should be fine moving HOTSPOT_SETUP_JVM_FEATURES to after >> LIB_SETUP_LIBRARIES in configure.ac. (Note that many of these calls are just >> ordered as they happened to be, though some really need to be before or >> after others. I cannot find a reason for these two.) Then you can >> enable/disable AOT after looking for libelf. It depends on what kind of >> behavior you want. >> >> If libelf isn't commonly installed, then having configure use it, and enable >> AOT if it is, and semi silently just disabling AOT if it's not is fine. We >> can define our Oracle builds to explicitly set --enable-aot to get an early >> fail on a missing dependency. That's inline with current configure behavior >> so I like that. > > I'm fine with that. But then we shouldn't warn if we don't find > libelf. We should only warn, if '--enable-aot' was explicitly > requested by the user and if this request can't be fulfilled because > of the missing libelf dependency. > > I also noticed that currently the configuration for AOT bails out > early, because it can not find the "aot src": > > + AC_MSG_CHECKING([if aot src is present]) > + if test -d "$HOTSPOT_TOPDIR/src/jdk.aot"; then > + AC_MSG_RESULT([yes]) > > But I couldn't find the patch which provides > "$HOTSPOT_TOPDIR/src/jdk.aot". Should this read > "$HOTSPOT_TOPDIR/src/jdk.vm.compiler" which is the location where the > graal sources will be copied to (see RFR for "8166417: Graal-core into > JDK for AOT compiler")? Or is there yet another patch to come which > will provide "$HOTSPOT_TOPDIR/src/jdk.aot"? > >> >> Another note. We usually use the style --disable- rather than >> --enable-=no when referring to configure options. >> >> /Erik >> >>> AOT modules dependencies moved to new module-info.java.extra. >>> >>> And other changes suggested by Erik. >>> >>> Thanks, >>> Vladimir >>> >>> On 10/27/16 11:41 AM, Vladimir Kozlov wrote: >>>> >>>> Thank you, Erik >>>> >>>> On 10/27/16 5:40 AM, Erik Joelsson wrote: >>>>> >>>>> >>>>> On 2016-10-27 02:45, Vladimir Kozlov wrote: >>>>>> >>>>>> AOT JEP: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>>>> Subtask: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8166416 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>>>> >>>>> hotspot.m4: 296: Comment is misleading. Should just be removed. >>>> >>>> >>>> Removed. >>>> >>>>> CompileJavaModules.gmk: Use of -g flag for java compilation is >>>>> controlled globally. Please remove. >>>> >>>> >>>> See Chris comment on this. We want to have JAOTC java debug information >>>> for debugging in product builds. >>>> >>>>> Main.gmk: buildtools-hotspot should be declared inside the >>>>> CREATING_BUILDJDK conditional like all other buildtools targets. >>>> >>>> >>>> Done. >>>> >>>>> >>>>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>>>> >>>>> The extra exports from java.base needs to go in a new >>>>> jdk/src/java.base/share/classes/module-info.java.extra since the module >>>>> jdk.vm.compiler is optional. >>>> >>>> >>>> Done. >>>> >>>>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>>>> >>>>> Lib-jdk.aot.gmk: Please inline LDFLAGS and LIBS and add $(LIBS_JDKLIB) >>>>> to LIBS since that will provide -lc on Solaris automatically. >>>> >>>> >>>> Like this?: >>>> >>>> LDFLAGS := $(LDFLAGS_JDKLIB), \ >>>> LIBS := $(ELF_LIBS) $(LIBS_JDKLIB), \ >>>> >>>>> No need to set DEBUG_SYMBOLS or STRIP_SYMBOLS as the defaults should be >>>>> correct and controlled globally. >>>> >>>> >>>> Done. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>>> >>>>> /Erik >>>>>> >>>>>> >>>>>> Please, review build changes for AOT. Only Linux/x64 platform is >>>>>> supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>>>> >>>>>> Changes include new 'jaotc' launcher, makefile changes to build >>>>>> jdk.vm.compiler (Graal) and jdk.aot modules used by 'jaotc'. >>>>>> Both modules sources are located in Hotspot: hotspot/src/jdk.aot and >>>>>> hotspot/src/jdk.vm.compiler. >>>>>> 'jaotc' requires installed libelf package on a system to build native >>>>>> part of 'jaotc'. It is used to generated AOT shared libraries (.so) as >>>>>> result of AOT compilation. >>>>>> >>>>>> Hotspot makefile changes will be pushed together with Hotspot AOT >>>>>> changes. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>> >>>>> >> From vladimir.kozlov at oracle.com Fri Oct 28 15:52:40 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 08:52:40 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <9f52669e-75fe-7139-78cb-d28ae1aff0a7@redhat.com> References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <9f52669e-75fe-7139-78cb-d28ae1aff0a7@redhat.com> Message-ID: <58137448.6040205@oracle.com> Thank you, Andrew, for verifying that build changes do not break AArch64. But it would be nice if you can also apply Hotspot changes (revert hs.make.webrev changes before that since hs.webrev have them): http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ and jaotc sources (which are located in Hotspot repo): http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ Thanks, Vladimir On 10/28/16 4:04 AM, Andrew Dinn wrote: > On 28/10/16 08:56, Vladimir Kozlov wrote: >> Webrevs updated in place (please, reload webpage to get new version). >> generated-configure.sh changes are included. >> >> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > > I have applied these 3 change sets to the latest hs on AArch64 and it > build and runs ok (perhaps not much of a surprise). > > However, I'm not sure exactly what this proves -- or, rather, whether > that is all that you want proved in order to proceed with 8166416. > > 1) Do I need to use any specific configure options to exercise these > specific changes satisfactorily on AArch64? > > 2) Do I need to include any further changes in order to test more fully? > > regards, > > > Andrew Dinn > ----------- > From vladimir.kozlov at oracle.com Fri Oct 28 16:45:23 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 09:45:23 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> <5812437E.9070404@oracle.com> Message-ID: <581380A3.1010201@oracle.com> I got some data. It is about 10% increase size with -g: product build, no -g for AOT modules 191173 Oct 28 09:15 build/product/images/jmods/jdk.aot.jmod 4893583 Oct 28 09:16 build/product/images/jmods/jdk.vm.compiler.jmod 352073 Oct 28 09:15 build/product/images/jmods/jdk.vm.ci.jmod product build, AOT modules with -g 208913 Oct 28 09:31 build/product/images/jmods/jdk.aot.jmod 5465020 Oct 28 09:32 build/product/images/jmods/jdk.vm.compiler.jmod 387958 Oct 28 09:32 build/product/images/jmods/jdk.vm.ci.jmod Vladimir On 10/28/16 1:43 AM, Alan Bateman wrote: > On 28/10/2016 09:34, Erik Joelsson wrote: > >> >> but maybe jlink is able to strip the debug info from java classes now? In that case we could consider globally enabling -g for product builds. > Yes, it can, `jlink --strip-debug` (or `-G`). A good test would be to build with `javac -g` and then strip the debug information when generating the JRE image to see that it comes out at around the > same size as the current build. > > -Alan. From cthalinger at twitter.com Fri Oct 28 17:11:05 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Fri, 28 Oct 2016 07:11:05 -1000 Subject: [9] RFR(L) 8166417: Graal-core into JDK for AOT compiler In-Reply-To: References: <5812295F.4000107@oracle.com> Message-ID: <28A4F2B3-0820-46EE-899E-C0AA58D5BAEE@twitter.com> > On Oct 27, 2016, at 11:20 PM, Volker Simonis wrote: > > On Thu, Oct 27, 2016 at 6:20 PM, Vladimir Kozlov > > wrote: >> AOT JEP: >> https://bugs.openjdk.java.net/browse/JDK-8166089 >> Subtask: >> https://bugs.openjdk.java.net/browse/JDK-8166417 >> Graal-core: >> http://hg.openjdk.java.net/graal/graal-core/ >> Webrev: >> http://cr.openjdk.java.net/~kvn/aot/graal.webrev/ >> >> This is last part of AOT project. AOT requires Graal-core to be part of JDK. >> AOT compiler uses Graal-core as backend compiler - we use its ability to >> generate immutable PIC code. >> >> We are working with Oracle's Labs on this integration. >> >> Meanwhile, if you want to experiment do next: >> >> hg clone http://hg.openjdk.java.net/graal/graal-core/ >> >> apply patch from graal.webrev >> >> cp graal-core/graal myjdk/hotspot/src/jdk.vm.compiler >> > > The convention is that the directory name under "src/" (i.e. > "jdk.vm.ci") corresponds to the package name of the contained sources > (i.e. "jdk.vm.ci/share/classes/jdk.vm.ci.code/src/jdk/vm/ci/code?). I don?t think that?s a convention. There are plenty of examples that say otherwise. Also, you could drop another compiler in the same module jdk.vm.compiler and things would just work. > > Following this convention, the graal sources should either be copied > to myjdk/hotspot/src/com.oracle.graal/share/classes or the graal packe > should be renamed to jdk.vm.compiler. And then there's still the > package com.oracle.nfi which would require a separate sub-dir under > hotspot/src if it wouldn't be moved into com.oracle.graal... > >> After that you can build JDK with AOT. >> >> Thanks, >> Vladimir From cthalinger at twitter.com Fri Oct 28 17:26:07 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Fri, 28 Oct 2016 07:26:07 -1000 Subject: [9] RFR(L) 8166413: Integrate VM part of AOT changes In-Reply-To: References: <58115536.5080205@oracle.com> <4ab955c6-7a68-622a-7c6a-522875251c84@redhat.com> <58125AF0.2000902@oracle.com> Message-ID: > On Oct 27, 2016, at 10:11 PM, Andrew Haley wrote: > > On 27/10/16 23:02, Christian Thalinger wrote: > >> John and I discussed this before we decided to go with standard >> formats. The biggest issue with the current CDS archive is that >> it?s not relocatable. That means if the address you want to map at >> is not available you?re out of luck. > > Is that really an issue in practice? Yes, I?ve seen this a lot on my MacBook in the past (interestingly not anymore) when first looking into storing interned strings in the CDS archive for AOT. Also with address space layout randomization (ASLR) on Linux it can happen pretty easily. It happens for people: https://www.google.com/#q=java+unable+to+use+shared+archive and usually they don?t know why. > >> So, the proprietary CDS format needs to become relocatable; pretty >> much make it a DSO. And then you?re just reinventing the wheel. >> Also, every operating system had decades to tune loading and linking >> their format while you have to do it yourself. > > They have, but what they have is still not so efficient: think about > GOTs, PLTs, etc. These work, but we do better in JIT-generated code. > In other words, being as good as existing DSO formats is a very low > bar. Maybe you are right. > > Andrew. From cthalinger at twitter.com Fri Oct 28 17:45:20 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Fri, 28 Oct 2016 07:45:20 -1000 Subject: [9] RFR(L) 8166415: Integrate AOT tool JAOTC In-Reply-To: <581304D2.5060407@oracle.com> References: <581151ED.7080904@oracle.com> <58127414.8020907@oracle.com> <581304D2.5060407@oracle.com> Message-ID: > On Oct 27, 2016, at 9:57 PM, Vladimir Kozlov wrote: > > Webrevs updated in place (please, reload webpage to get new version). > > http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ > > jaotc tool package was renamed from com.oracle.graal.aot to jdk.tools.jaotc similar to other new jdk tools (link). That?s good. Thanks. > > Switched to new internal Unsafe instead of old sun.misc Unsafe. > > Thanks, > Vladimir > > On 10/27/16 2:39 PM, Vladimir Kozlov wrote: >> Thanks Paul, >> >> I think it is not difficult to adjust jaotc code to use internal Unsafe instead of sun.misc.Unsafe. >> Give me an example and I will do that. >> >> We have one UnsafeAccess.java class through which we do access: >> >> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/src/jdk.aot/share/classes/com.oracle.jnilibelf/src/com/oracle/jnilibelf/UnsafeAccess.java.html >> >> Thanks, >> Vladimir >> On 10/27/16 1:56 PM, Paul Sandoz wrote: >>> Hi Vladimir, >>> >>> I think we should adjust to depend on the internal Unsafe and update the java.base module to export qualified to jdk.aot (like that for jdk.vm.compiler). It could easily be done later rather than >>> now, if so I can log an issue. >>> >>> Paul. >>> >>>> On 26 Oct 2016, at 18:01, Vladimir Kozlov wrote: >>>> >>>> AOT JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>> Subtask: >>>> https://bugs.openjdk.java.net/browse/JDK-8166415 >>>> Webrev: >>>> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ >>>> >>>> Please, review implementation of jaotc - Java Ahead-Of-Time static compiler. It produces native code for compiled java methods. It uses Graal as the code-generating backend and libelf for >>>> generating shared .so AOT libraries. It is written mostly in java. Code interface to libelf is written in C. >>>> >>>> Only Linux/x64 platform is supported. 'jaotc' and AOT part of Hotspot will be build only on Linux/x64. >>>> >>>> Corresponding build changes are reviewed in a separate thread for https://bugs.openjdk.java.net/browse/JDK-8166416 >>>> >>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>>> >>>> Thanks, >>>> Vladimir >>> From paul.sandoz at oracle.com Fri Oct 28 19:03:54 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 28 Oct 2016 12:03:54 -0700 Subject: [9] RFR(L) 8166415: Integrate AOT tool JAOTC In-Reply-To: <581304D2.5060407@oracle.com> References: <581151ED.7080904@oracle.com> <58127414.8020907@oracle.com> <581304D2.5060407@oracle.com> Message-ID: <8E12182D-383E-4FC3-B6F5-63084932EDF9@oracle.com> > On 28 Oct 2016, at 00:57, Vladimir Kozlov wrote: > > Webrevs updated in place (please, reload webpage to get new version). > > http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ > > jaotc tool package was renamed from com.oracle.graal.aot to jdk.tools.jaotc similar to other new jdk tools (jlink). > > Switched to new internal Unsafe instead of old sun.misc Unsafe. > +1 on the Unsafe stuff. Paul. From mandy.chung at oracle.com Fri Oct 28 20:31:06 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 28 Oct 2016 13:31:06 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <58114E0C.107@oracle.com> References: <58114E0C.107@oracle.com> Message-ID: <4168467B-C9E4-4DEF-9851-0D04C647ECDF@oracle.com> > On Oct 26, 2016, at 5:45 PM, Vladimir Kozlov wrote: > > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ make/gensrc/Gensrc-jdk.vm.compiler.gmk This generates module-info.java.extra at build time to augment module-info.java with `uses` and `provides`. module-info.java.extra is expected to be checked in the source. Do you expect the list of `uses` and `provides` are often changed? The alternative is to declare `uses` and `provides` in module-info.java in the source repo so that a reader can see the module descriptor content without needing to do a build. A test to detect if the module-info.java is out-of-sync with the annotated options. In addition a make target to generate the list of `use` and `provides` can be used to generate module-info.java to be included in any change in the annotated options list. Mandy From vladimir.kozlov at oracle.com Fri Oct 28 20:52:05 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 13:52:05 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <4168467B-C9E4-4DEF-9851-0D04C647ECDF@oracle.com> References: <58114E0C.107@oracle.com> <4168467B-C9E4-4DEF-9851-0D04C647ECDF@oracle.com> Message-ID: <5813BA75.8060604@oracle.com> Thank you, Mandy On 10/28/16 1:31 PM, Mandy Chung wrote: > >> On Oct 26, 2016, at 5:45 PM, Vladimir Kozlov wrote: >> >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ > > make/gensrc/Gensrc-jdk.vm.compiler.gmk > This generates module-info.java.extra at build time to augment module-info.java with `uses` and `provides`. module-info.java.extra is expected to be checked in the source. > > Do you expect the list of `uses` and `provides` are often changed? Yes, Graal code changes frequently and we need to run annotation processor to generate these dependencies for each jdk.vm.compiler (Graal) module build. > The alternative is to declare `uses` and `provides` in module-info.java in the source repo so that a reader can see the module descriptor content without needing to do a build. A test to detect if the module-info.java is out-of-sync with the annotated options. In addition a make target to generate the list of `use` and `provides` can be used to generate module-info.java to be included in any change in the annotated options list. What is "a reader"? And how to check "out-of-sync" without running 'processor'? Sorry, I am not familiar with all this modules build process. Thanks, Vladimir > > Mandy > From forax at univ-mlv.fr Fri Oct 28 21:05:09 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 28 Oct 2016 21:05:09 +0000 Subject: [9] RFR(L) 8166417: Graal-core into JDK for AOT compiler In-Reply-To: <28A4F2B3-0820-46EE-899E-C0AA58D5BAEE@twitter.com> References: <5812295F.4000107@oracle.com> <28A4F2B3-0820-46EE-899E-C0AA58D5BAEE@twitter.com> Message-ID: On October 28, 2016 7:11:05 PM GMT+02:00, Christian Thalinger wrote: > >> On Oct 27, 2016, at 11:20 PM, Volker Simonis > wrote: >> >> On Thu, Oct 27, 2016 at 6:20 PM, Vladimir Kozlov >> > >wrote: >>> AOT JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>> Subtask: >>> https://bugs.openjdk.java.net/browse/JDK-8166417 >>> Graal-core: >>> http://hg.openjdk.java.net/graal/graal-core/ >>> Webrev: >>> http://cr.openjdk.java.net/~kvn/aot/graal.webrev/ >>> >>> This is last part of AOT project. AOT requires Graal-core to be part >of JDK. >>> AOT compiler uses Graal-core as backend compiler - we use its >ability to >>> generate immutable PIC code. >>> >>> We are working with Oracle's Labs on this integration. >>> >>> Meanwhile, if you want to experiment do next: >>> >>> hg clone http://hg.openjdk.java.net/graal/graal-core/ >>> >>> apply patch from graal.webrev >>> >>> cp graal-core/graal myjdk/hotspot/src/jdk.vm.compiler >>> >> >> The convention is that the directory name under "src/" (i.e. >> "jdk.vm.ci") corresponds to the package name of the contained sources >> (i.e. "jdk.vm.ci/share/classes/jdk.vm.ci.code/src/jdk/vm/ci/code?). > >I don?t think that?s a convention. There are plenty of examples that >say otherwise. Also, you could drop another compiler in the same >module jdk.vm.compiler and things would just work. That's the convention but obviously the jdk was not designed when modules were around thus the actual jdk modules do not respect that convention. New modules should try to respect that convention. R?mi > >> >> Following this convention, the graal sources should either be copied >> to myjdk/hotspot/src/com.oracle.graal/share/classes or the graal >packe >> should be renamed to jdk.vm.compiler. And then there's still the >> package com.oracle.nfi which would require a separate sub-dir under >> hotspot/src if it wouldn't be moved into com.oracle.graal... >> >>> After that you can build JDK with AOT. >>> >>> Thanks, >>> Vladimir -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From mandy.chung at oracle.com Fri Oct 28 21:23:12 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 28 Oct 2016 14:23:12 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <5813BA75.8060604@oracle.com> References: <58114E0C.107@oracle.com> <4168467B-C9E4-4DEF-9851-0D04C647ECDF@oracle.com> <5813BA75.8060604@oracle.com> Message-ID: <0C15B5A2-2E2E-4AE6-8DDA-8E8F5D8E3472@oracle.com> > On Oct 28, 2016, at 1:52 PM, Vladimir Kozlov wrote: > > Thank you, Mandy > > On 10/28/16 1:31 PM, Mandy Chung wrote: >> >>> On Oct 26, 2016, at 5:45 PM, Vladimir Kozlov wrote: >>> >>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >> >> make/gensrc/Gensrc-jdk.vm.compiler.gmk >> This generates module-info.java.extra at build time to augment module-info.java with `uses` and `provides`. module-info.java.extra is expected to be checked in the source. >> >> Do you expect the list of `uses` and `provides` are often changed? > > Yes, Graal code changes frequently and we need to run annotation processor to generate these dependencies for each jdk.vm.compiler (Graal) module build. > When the OptionDescriptors list is changed, we can run the tool (you can add a make target if it helps) to generate the list of `uses` and `provides`. Then update module-info.java in the source together with the changes. That would work too, right? >> The alternative is to declare `uses` and `provides` in module-info.java in the source repo so that a reader can see the module descriptor content without needing to do a build. A test to detect if the module-info.java is out-of-sync with the annotated options. In addition a make target to generate the list of `use` and `provides` can be used to generate module-info.java to be included in any change in the annotated options list. > > What is "a reader?? A person reading the code. > And how to check "out-of-sync" without running 'processor?? I meant a regression test that will run the annotation processor and compare with module-info.class in the image (using java.lang.module.ModuleDescriptor API) Mandy > Sorry, I am not familiar with all this modules build process. > > Thanks, > Vladimir > >> >> Mandy >> From cthalinger at twitter.com Fri Oct 28 21:41:07 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Fri, 28 Oct 2016 11:41:07 -1000 Subject: [9] RFR(L) 8166417: Graal-core into JDK for AOT compiler In-Reply-To: References: <5812295F.4000107@oracle.com> <28A4F2B3-0820-46EE-899E-C0AA58D5BAEE@twitter.com> Message-ID: > On Oct 28, 2016, at 11:05 AM, Remi Forax wrote: > > > > On October 28, 2016 7:11:05 PM GMT+02:00, Christian Thalinger wrote: >> >>> On Oct 27, 2016, at 11:20 PM, Volker Simonis >> wrote: >>> >>> On Thu, Oct 27, 2016 at 6:20 PM, Vladimir Kozlov >>> > >> wrote: >>>> AOT JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8166089 >>>> Subtask: >>>> https://bugs.openjdk.java.net/browse/JDK-8166417 >>>> Graal-core: >>>> http://hg.openjdk.java.net/graal/graal-core/ >>>> Webrev: >>>> http://cr.openjdk.java.net/~kvn/aot/graal.webrev/ >>>> >>>> This is last part of AOT project. AOT requires Graal-core to be part >> of JDK. >>>> AOT compiler uses Graal-core as backend compiler - we use its >> ability to >>>> generate immutable PIC code. >>>> >>>> We are working with Oracle's Labs on this integration. >>>> >>>> Meanwhile, if you want to experiment do next: >>>> >>>> hg clone http://hg.openjdk.java.net/graal/graal-core/ >>>> >>>> apply patch from graal.webrev >>>> >>>> cp graal-core/graal myjdk/hotspot/src/jdk.vm.compiler >>>> >>> >>> The convention is that the directory name under "src/" (i.e. >>> "jdk.vm.ci") corresponds to the package name of the contained sources >>> (i.e. "jdk.vm.ci/share/classes/jdk.vm.ci.code/src/jdk/vm/ci/code?). >> >> I don?t think that?s a convention. There are plenty of examples that >> say otherwise. Also, you could drop another compiler in the same >> module jdk.vm.compiler and things would just work. > > That's the convention but obviously the jdk was not designed when modules were around thus the actual jdk modules do not respect that convention. > New modules should try to respect that convention. Since it?s only a convention I wouldn?t do it. Future merges with upstream Graal would be a major PITA. Luckily I don?t have to do it... > > R?mi > >> >>> >>> Following this convention, the graal sources should either be copied >>> to myjdk/hotspot/src/com.oracle.graal/share/classes or the graal >> packe >>> should be renamed to jdk.vm.compiler. And then there's still the >>> package com.oracle.nfi which would require a separate sub-dir under >>> hotspot/src if it wouldn't be moved into com.oracle.graal... >>> >>>> After that you can build JDK with AOT. >>>> >>>> Thanks, >>>> Vladimir > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. From cthalinger at twitter.com Fri Oct 28 22:14:09 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Fri, 28 Oct 2016 12:14:09 -1000 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <0C15B5A2-2E2E-4AE6-8DDA-8E8F5D8E3472@oracle.com> References: <58114E0C.107@oracle.com> <4168467B-C9E4-4DEF-9851-0D04C647ECDF@oracle.com> <5813BA75.8060604@oracle.com> <0C15B5A2-2E2E-4AE6-8DDA-8E8F5D8E3472@oracle.com> Message-ID: <865B0B9F-8A65-4CA3-83D2-4919F4991C1F@twitter.com> > On Oct 28, 2016, at 11:23 AM, Mandy Chung wrote: > >> >> On Oct 28, 2016, at 1:52 PM, Vladimir Kozlov wrote: >> >> Thank you, Mandy >> >> On 10/28/16 1:31 PM, Mandy Chung wrote: >>> >>>> On Oct 26, 2016, at 5:45 PM, Vladimir Kozlov wrote: >>>> >>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> >>> make/gensrc/Gensrc-jdk.vm.compiler.gmk >>> This generates module-info.java.extra at build time to augment module-info.java with `uses` and `provides`. module-info.java.extra is expected to be checked in the source. >>> >>> Do you expect the list of `uses` and `provides` are often changed? >> >> Yes, Graal code changes frequently and we need to run annotation processor to generate these dependencies for each jdk.vm.compiler (Graal) module build. >> > > When the OptionDescriptors list is changed, we can run the tool (you can add a make target if it helps) to generate the list of `uses` and `provides`. Then update module-info.java in the source together with the changes. That would work too, right? Why can?t we have it generated? It?s much more convenient and less maintenance overhead. There needs to be a good reason to give this up. > >>> The alternative is to declare `uses` and `provides` in module-info.java in the source repo so that a reader can see the module descriptor content without needing to do a build. A test to detect if the module-info.java is out-of-sync with the annotated options. In addition a make target to generate the list of `use` and `provides` can be used to generate module-info.java to be included in any change in the annotated options list. >> >> What is "a reader?? > > A person reading the code. > >> And how to check "out-of-sync" without running 'processor?? > > I meant a regression test that will run the annotation processor and compare with module-info.class in the image (using java.lang.module.ModuleDescriptor API) > > Mandy > >> Sorry, I am not familiar with all this modules build process. >> >> Thanks, >> Vladimir >> >>> >>> Mandy From vladimir.kozlov at oracle.com Fri Oct 28 22:19:46 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 15:19:46 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <0C15B5A2-2E2E-4AE6-8DDA-8E8F5D8E3472@oracle.com> References: <58114E0C.107@oracle.com> <4168467B-C9E4-4DEF-9851-0D04C647ECDF@oracle.com> <5813BA75.8060604@oracle.com> <0C15B5A2-2E2E-4AE6-8DDA-8E8F5D8E3472@oracle.com> Message-ID: <5813CF02.1070104@oracle.com> >>> The alternative is to declare `uses` and `provides` in module-info.java in the source repo >>> so that a reader can see the module descriptor content without needing to do a build. >> What is "a reader?? > > A person reading the code. I don't think it will help "reader" to have 174 lines which are hard to read in module-info.java: provides com.oracle.graal.nodes.graphbuilderconf.NodeIntrinsicPluginFactory with com.oracle.graal.hotspot.nodes.PluginFactory_AcquiredCASLockNode; I would agree if it was only 20-30 dependencies but not so many. Regards, Vladimir On 10/28/16 2:23 PM, Mandy Chung wrote: > >> On Oct 28, 2016, at 1:52 PM, Vladimir Kozlov wrote: >> >> Thank you, Mandy >> >> On 10/28/16 1:31 PM, Mandy Chung wrote: >>> >>>> On Oct 26, 2016, at 5:45 PM, Vladimir Kozlov wrote: >>>> >>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>> >>> make/gensrc/Gensrc-jdk.vm.compiler.gmk >>> This generates module-info.java.extra at build time to augment module-info.java with `uses` and `provides`. module-info.java.extra is expected to be checked in the source. >>> >>> Do you expect the list of `uses` and `provides` are often changed? >> >> Yes, Graal code changes frequently and we need to run annotation processor to generate these dependencies for each jdk.vm.compiler (Graal) module build. >> > > When the OptionDescriptors list is changed, we can run the tool (you can add a make target if it helps) to generate the list of `uses` and `provides`. Then update module-info.java in the source together with the changes. That would work too, right? > >>> The alternative is to declare `uses` and `provides` in module-info.java in the source repo so that a reader can see the module descriptor content without needing to do a build. A test to detect if the module-info.java is out-of-sync with the annotated options. In addition a make target to generate the list of `use` and `provides` can be used to generate module-info.java to be included in any change in the annotated options list. >> >> What is "a reader?? > > A person reading the code. > >> And how to check "out-of-sync" without running 'processor?? > > I meant a regression test that will run the annotation processor and compare with module-info.class in the image (using java.lang.module.ModuleDescriptor API) > > Mandy > >> Sorry, I am not familiar with all this modules build process. >> >> Thanks, >> Vladimir >> >>> >>> Mandy >>> > From doug.simon at oracle.com Fri Oct 28 23:22:10 2016 From: doug.simon at oracle.com (Doug Simon) Date: Sat, 29 Oct 2016 01:22:10 +0200 Subject: RFR: 8168915: [JVMCI] use MethodParameters attribute instead of depending on -g option for sanity checks Message-ID: <2F06847B-314B-4D2A-A7F0-5A3ED83821F1@oracle.com> In light of discussion[1] about build changes for 8166416 with respect to -g, this patch enables JVMCI to expose parameter info available in the MethodParameters class file attribute[2] (produced by the -parameters javac option) instead of relying on the LocalVariableTable attribute (produced by -g). I?m assuming the class file expansion caused by -parameters is significantly less than -g although I?m surprised to see that parameter info is missing from the JDK classes as noted in the JBS issue. https://bugs.openjdk.java.net/browse/JDK-8168915 http://cr.openjdk.java.net/~dnsimon/8168915/ -Doug [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/025051.html [2] https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.24 From vladimir.kozlov at oracle.com Sat Oct 29 01:31:31 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 18:31:31 -0700 Subject: RFR: 8168915: [JVMCI] use MethodParameters attribute instead of depending on -g option for sanity checks In-Reply-To: <2F06847B-314B-4D2A-A7F0-5A3ED83821F1@oracle.com> References: <2F06847B-314B-4D2A-A7F0-5A3ED83821F1@oracle.com> Message-ID: <5813FBF3.2050907@oracle.com> Thank you Doug, Looks good to me (not expert). I will test it with AOT. I see only 3% size increase vs 10% with -g. I can replace '-g' with '-parameters' javac option for AOT modules when the fix is integrated. Thanks, Vladimir On 10/28/16 4:22 PM, Doug Simon wrote: > In light of discussion[1] about build changes for 8166416 with respect to -g, this patch enables JVMCI to expose parameter info available in the MethodParameters class file attribute[2] (produced by the -parameters javac option) instead of relying on the LocalVariableTable attribute (produced by -g). > > I?m assuming the class file expansion caused by -parameters is significantly less than -g although I?m surprised to see that parameter info is missing from the JDK classes as noted in the JBS issue. > > https://bugs.openjdk.java.net/browse/JDK-8168915 > http://cr.openjdk.java.net/~dnsimon/8168915/ > > -Doug > > [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/025051.html > [2] https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.24 > From mandy.chung at oracle.com Sat Oct 29 01:35:46 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 28 Oct 2016 18:35:46 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <5813CF02.1070104@oracle.com> References: <58114E0C.107@oracle.com> <4168467B-C9E4-4DEF-9851-0D04C647ECDF@oracle.com> <5813BA75.8060604@oracle.com> <0C15B5A2-2E2E-4AE6-8DDA-8E8F5D8E3472@oracle.com> <5813CF02.1070104@oracle.com> Message-ID: > On Oct 28, 2016, at 3:19 PM, Vladimir Kozlov wrote: > >>>> The alternative is to declare `uses` and `provides` in module-info.java in the source repo >>>> so that a reader can see the module descriptor content without needing to do a build. >>> What is "a reader?? >> >> A person reading the code. > > I don't think it will help "reader" to have 174 lines which are hard to read in module-info.java: > > provides com.oracle.graal.nodes.graphbuilderconf.NodeIntrinsicPluginFactory with com.oracle.graal.hotspot.nodes.PluginFactory_AcquiredCASLockNode; > > I would agree if it was only 20-30 dependencies but not so many. Can you send module-info.java.extra and module-info.java generated in your build? Mandy From vladimir.kozlov at oracle.com Sat Oct 29 02:23:20 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 28 Oct 2016 19:23:20 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <0acdf13c-88d9-f904-7fcb-1f5d3683e30c@oracle.com> Message-ID: <58140818.1090907@oracle.com> Updated build webrevs (webrev.2) http://cr.openjdk.java.net/~kvn/aot/top.webrev.2/ http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev.2/ Use one ENABLE_AOT variable in all config and make files (removed NEEDS_LIB_JELFSHIM). Moved HOTSPOT_SETUP_JVM_FEATURES after LIB_SETUP_LIBRARIES. First, check for linux-x64 presence of jdk.aot and jdk.vm.compiler directories. Set ENABLE_AOT=true if all are present. Throw config errors if --enable-aot is specified but sources are missing or on different platforms. Second, in lib-elf.m4, when ENABLE_AOT==true check if libelf.h is present and usable. Throw config errors if --enable-aot is specified but libelf.h is missing. Otherwise warnings and set ENABLE_AOT==false. Third, check ENABLE_AOT in HOTSPOT_SETUP_JVM_FEATURES to set JVM_FEATURES. Thanks, Vladimir On 10/28/16 3:27 AM, Erik Joelsson wrote: > > > On 2016-10-28 11:09, Volker Simonis wrote: >> On Fri, Oct 28, 2016 at 10:57 AM, Erik Joelsson >> wrote: >>> Hello, >>> >>> >>> On 2016-10-28 09:56, Vladimir Kozlov wrote: >>>> Webrevs updated in place (please, reload webpage to get new version). >>>> generated-configure.sh changes are included. >>>> >>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>>> >>>> libelfshim generation is guarded by ENABLE_AOT as Volker suggested. >>>> >>>> I did not figure out how switch off ENABLE_AOT if libelf not found. So I >>>> changed warning to configuration error: >>>> >>>> error: libelf not found, cannot build AOT. Disable AOT build: >>>> --enable-aot=no. You might be able to fix this by running 'sudo yum install >>>> elfutils-libelf-devel'. >>>> >>>> Which is not correct since AOT build is enabled by default on linux-x64. >>>> Need help with this. >>>> >>> I think it should be fine moving HOTSPOT_SETUP_JVM_FEATURES to after >>> LIB_SETUP_LIBRARIES in configure.ac. (Note that many of these calls are just >>> ordered as they happened to be, though some really need to be before or >>> after others. I cannot find a reason for these two.) Then you can >>> enable/disable AOT after looking for libelf. It depends on what kind of >>> behavior you want. >>> >>> If libelf isn't commonly installed, then having configure use it, and enable >>> AOT if it is, and semi silently just disabling AOT if it's not is fine. We >>> can define our Oracle builds to explicitly set --enable-aot to get an early >>> fail on a missing dependency. That's inline with current configure behavior >>> so I like that. >> I'm fine with that. But then we shouldn't warn if we don't find >> libelf. We should only warn, if '--enable-aot' was explicitly >> requested by the user and if this request can't be fulfilled because >> of the missing libelf dependency. > --enable-aot and not finding libelf should result in an error. If on linux-x64 where aot is default enabled, there should be a warning that it was disabled because libelf is missing. This warning > would now need to be in the ENABLE_AOT logic. > > /Erik From david.holmes at oracle.com Sat Oct 29 15:31:09 2016 From: david.holmes at oracle.com (David Holmes) Date: Sun, 30 Oct 2016 01:31:09 +1000 Subject: [New Bug] [aarch64] [hotspot] src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp In-Reply-To: References: Message-ID: Hi Daniel, On 29/10/2016 12:35 AM, Daniel Stewart wrote: > I'm new to OpenJDK and not sure where to send this, so I thought I'd start > with this list. Hotspot issues should go to hotspot-dev (cc'd for reference), or if you know the sub-area to hotspot-runtime-dev, or hotspot-compiler-dev etc. > There is a bug that occurred 3 days ago in the jdk9/dev branch > in src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp. I am doing > nightly aarch64 builds of jdk9/dev branch and started seeing it 3 days ago. > This bug does not appear in the jdk9/hs branch. > > The actual changeset that appears to have caused the issue is 12188. I've > included a simple fix below, as this is simply a matter of a missing > #endif. This is already fixed in the jdk9/hs repo. It is unfortunate that the breakage was allowed to escape. The fix should appear in jdk9/dev early this week. http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/95c6654fa2ee Cheers, David > Thanks and my apologies if this is the wrong mailing list. > > Daniel Stewart > > -- > > diff -r c30b6e2d2ec4 > src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp > --- a/src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp Thu > Oct 27 21:22:32 2016 +0000 > +++ b/src/cpu/aarch64/vm/templateInterpreterGenerator_aarch64.cpp Fri > Oct 28 14:33:28 2016 +0000 > @@ -476,6 +476,7 @@ > } > #endif > } > +#endif > // handle exceptions > { > Label L; > From kim.barrett at oracle.com Sat Oct 29 23:26:21 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 29 Oct 2016 19:26:21 -0400 Subject: RFR: 8166811: Missing memory fences between memory allocation and refinement In-Reply-To: References: Message-ID: > On Oct 25, 2016, at 7:13 PM, Kim Barrett wrote: > > Please review this change to address missing memory barriers needed to > ensure ordering between allocation and refinement in G1. > [?] > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166811 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166811/webrev.00/ > [Based on http://cr.openjdk.java.net/~kbarrett/8166607/webrev.02/] > ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1RemSet.cpp 581 // The region could be young. Cards for young regions are dirtied, 582 // so the post-barrier will filter them out. However, that dirtying 583 // is performed concurrently. A write to a young object could occur 584 // before the card has been dirtied, slipping past the filter. This is a rewording of the comment that used to be here. However, it was not true even before these changes. As part of JDK-8014555 we mark young region cards with g1_young_card_val(). That's the change set that added the storeload to the post-barrier. I'm not quite sure what to do about this. The comment is currently wrong. However, the storeload is considered a problem, and there have been various ideas discussed for eliminating it that might allow us to go back to dirtying young cards. ------------------------------------------------------------------------------ From erik.joelsson at oracle.com Mon Oct 31 10:11:33 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 31 Oct 2016 11:11:33 +0100 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <58140818.1090907@oracle.com> References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <0acdf13c-88d9-f904-7fcb-1f5d3683e30c@oracle.com> <58140818.1090907@oracle.com> Message-ID: Hello Vladimir, The new organization of the checks looks good. Just found one minor nit, in lib-elf.m4, no need to AC_SUBST(ENABLE_AOT) again. /Erik On 2016-10-29 04:23, Vladimir Kozlov wrote: > Updated build webrevs (webrev.2) > > http://cr.openjdk.java.net/~kvn/aot/top.webrev.2/ > http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev.2/ > > Use one ENABLE_AOT variable in all config and make files (removed > NEEDS_LIB_JELFSHIM). > > Moved HOTSPOT_SETUP_JVM_FEATURES after LIB_SETUP_LIBRARIES. > > First, check for linux-x64 presence of jdk.aot and jdk.vm.compiler > directories. Set ENABLE_AOT=true if all are present. Throw config > errors if --enable-aot is specified but sources are missing or on > different platforms. > > Second, in lib-elf.m4, when ENABLE_AOT==true check if libelf.h is > present and usable. > Throw config errors if --enable-aot is specified but libelf.h is missing. > Otherwise warnings and set ENABLE_AOT==false. > > Third, check ENABLE_AOT in HOTSPOT_SETUP_JVM_FEATURES to set > JVM_FEATURES. > > Thanks, > Vladimir > > On 10/28/16 3:27 AM, Erik Joelsson wrote: >> >> >> On 2016-10-28 11:09, Volker Simonis wrote: >>> On Fri, Oct 28, 2016 at 10:57 AM, Erik Joelsson >>> wrote: >>>> Hello, >>>> >>>> >>>> On 2016-10-28 09:56, Vladimir Kozlov wrote: >>>>> Webrevs updated in place (please, reload webpage to get new version). >>>>> generated-configure.sh changes are included. >>>>> >>>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>>>> >>>>> libelfshim generation is guarded by ENABLE_AOT as Volker suggested. >>>>> >>>>> I did not figure out how switch off ENABLE_AOT if libelf not >>>>> found. So I >>>>> changed warning to configuration error: >>>>> >>>>> error: libelf not found, cannot build AOT. Disable AOT build: >>>>> --enable-aot=no. You might be able to fix this by running 'sudo >>>>> yum install >>>>> elfutils-libelf-devel'. >>>>> >>>>> Which is not correct since AOT build is enabled by default on >>>>> linux-x64. >>>>> Need help with this. >>>>> >>>> I think it should be fine moving HOTSPOT_SETUP_JVM_FEATURES to after >>>> LIB_SETUP_LIBRARIES in configure.ac. (Note that many of these calls >>>> are just >>>> ordered as they happened to be, though some really need to be >>>> before or >>>> after others. I cannot find a reason for these two.) Then you can >>>> enable/disable AOT after looking for libelf. It depends on what >>>> kind of >>>> behavior you want. >>>> >>>> If libelf isn't commonly installed, then having configure use it, >>>> and enable >>>> AOT if it is, and semi silently just disabling AOT if it's not is >>>> fine. We >>>> can define our Oracle builds to explicitly set --enable-aot to get >>>> an early >>>> fail on a missing dependency. That's inline with current configure >>>> behavior >>>> so I like that. >>> I'm fine with that. But then we shouldn't warn if we don't find >>> libelf. We should only warn, if '--enable-aot' was explicitly >>> requested by the user and if this request can't be fulfilled because >>> of the missing libelf dependency. >> --enable-aot and not finding libelf should result in an error. If on >> linux-x64 where aot is default enabled, there should be a warning >> that it was disabled because libelf is missing. This warning >> would now need to be in the ENABLE_AOT logic. >> >> /Erik From doug.simon at oracle.com Mon Oct 31 10:58:20 2016 From: doug.simon at oracle.com (Doug Simon) Date: Mon, 31 Oct 2016 11:58:20 +0100 Subject: RFR: 8168915: [JVMCI] use MethodParameters attribute instead of depending on -g option for sanity checks In-Reply-To: <5813FBF3.2050907@oracle.com> References: <2F06847B-314B-4D2A-A7F0-5A3ED83821F1@oracle.com> <5813FBF3.2050907@oracle.com> Message-ID: <2DFB3C63-2360-4BBF-8408-B129D5BD43E1@oracle.com> > On 29 Oct 2016, at 03:31, Vladimir Kozlov wrote: > > Thank you Doug, > > Looks good to me (not expert). I will test it with AOT. > > I see only 3% size increase vs 10% with -g. I wonder if 3% is still too large for the JDK? If not, it would be great to see JDK jars/modules built with this on by default. > I can replace '-g' with '-parameters' javac option for AOT modules when the fix is integrated. Great. Thanks for the review. -Doug > On 10/28/16 4:22 PM, Doug Simon wrote: >> In light of discussion[1] about build changes for 8166416 with respect to -g, this patch enables JVMCI to expose parameter info available in the MethodParameters class file attribute[2] (produced by the -parameters javac option) instead of relying on the LocalVariableTable attribute (produced by -g). >> >> I?m assuming the class file expansion caused by -parameters is significantly less than -g although I?m surprised to see that parameter info is missing from the JDK classes as noted in the JBS issue. >> >> https://bugs.openjdk.java.net/browse/JDK-8168915 >> http://cr.openjdk.java.net/~dnsimon/8168915/ >> >> -Doug >> >> [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-October/025051.html >> [2] https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.24 >> From adinn at redhat.com Mon Oct 31 11:38:18 2016 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 31 Oct 2016 11:38:18 +0000 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <58137448.6040205@oracle.com> References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <9f52669e-75fe-7139-78cb-d28ae1aff0a7@redhat.com> <58137448.6040205@oracle.com> Message-ID: <23d2851f-027d-f142-e4d1-8c42e4a011f2@redhat.com> Hi Vladimir, On 28/10/16 16:52, Vladimir Kozlov wrote: > Thank you, Andrew, for verifying that build changes do not break AArch64. > But it would be nice if you can also apply Hotspot changes (revert > hs.make.webrev changes before that since hs.webrev have them): > > http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ > > and jaotc sources (which are located in Hotspot repo): > > http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ I tried this and found two missing changes to compiledIC_aarch64.cpp (basically a missing arg in each of two class to find_stub() -- see below for diff). However, I then ran into the problem Volker saw: Compiling 15 files for jdk.attach /home/adinn/openjdk/hs/hotspot/src/jdk.vm.ci/share/classes/module-info.java:40: error: module not found: jdk.vm.compiler jdk.vm.compiler; ^ /home/adinn/openjdk/hs/hotspot/src/jdk.vm.ci/share/classes/module-info.java:43: error: module not found: jdk.vm.compiler jdk.vm.compiler; . . . I assume fixing this second problem requires me to clone the graal-core repo into my tree and the apply the graal.webrev patch then rebuild. I am currently looking into that. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander ----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< --- diff -r 234ce47a2f3f src/cpu/aarch64/vm/compiledIC_aarch64.cpp --- a/src/cpu/aarch64/vm/compiledIC_aarch64.cpp Mon Oct 31 04:54:01 2016 -0400 +++ b/src/cpu/aarch64/vm/compiledIC_aarch64.cpp Mon Oct 31 07:16:47 2016 -0400 @@ -77,7 +77,7 @@ } void CompiledDirectStaticCall::set_to_interpreted(methodHandle callee, address entry) { - address stub = find_stub(); + address stub = find_stub(false /* is_aot */); guarantee(stub != NULL, "stub not found"); if (TraceICs) { @@ -129,7 +129,7 @@ } // Verify stub. - address stub = find_stub(); + address stub = find_stub(false /* is_aot */); assert(stub != NULL, "no stub found for static call"); // Creation also verifies the object. NativeMovConstReg* method_holder = nativeMovConstReg_at(stub); From adinn at redhat.com Mon Oct 31 15:35:08 2016 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 31 Oct 2016 15:35:08 +0000 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: <23d2851f-027d-f142-e4d1-8c42e4a011f2@redhat.com> References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <9f52669e-75fe-7139-78cb-d28ae1aff0a7@redhat.com> <58137448.6040205@oracle.com> <23d2851f-027d-f142-e4d1-8c42e4a011f2@redhat.com> Message-ID: <7bb59402-ecba-692b-9718-16414fe17efd@redhat.com> Hi Vladimir, On 31/10/16 11:38, Andrew Dinn wrote: > On 28/10/16 16:52, Vladimir Kozlov wrote: >> Thank you, Andrew, for verifying that build changes do not break AArch64. >> But it would be nice if you can also apply Hotspot changes (revert >> hs.make.webrev changes before that since hs.webrev have them): >> >> http://cr.openjdk.java.net/~kvn/aot/hs.webrev/ >> >> and jaotc sources (which are located in Hotspot repo): >> >> http://cr.openjdk.java.net/~kvn/aot/jaotc.webrev/ > > I tried this and found two missing changes to compiledIC_aarch64.cpp > (basically a missing arg in each of two class to find_stub() -- see > below for diff). > > However, I then ran into the problem Volker saw: > > Compiling 15 files for jdk.attach > /home/adinn/openjdk/hs/hotspot/src/jdk.vm.ci/share/classes/module-info.java:40: > error: module not found: jdk.vm.compiler > jdk.vm.compiler; > ^ > /home/adinn/openjdk/hs/hotspot/src/jdk.vm.ci/share/classes/module-info.java:43: > error: module not found: jdk.vm.compiler > jdk.vm.compiler; > > . . . > > I assume fixing this second problem requires me to clone the graal-core > repo into my tree and the apply the graal.webrev patch then rebuild. I cloned and patched the graal-core/graal tree and then copied it into my hotspot space as follows $ cp /path/to/graal-core/graal \ /otherpath/to/hs/hotspot/src/share/classes/jdk.vm.compiler With this and the extra tweaks to compiledCI_aarch64.cpp mentioned in the previous reply I managed to build a slowdebug release which successfully ran 'java Hello' and 'javac Hello.java'. Andrew Haley is currently trying to get Graal itself to run on AArch64. So, this is probably good enough for now to confirm the acceptability of the hs and jaotc change sets. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From vladimir.kozlov at oracle.com Mon Oct 31 16:39:00 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 31 Oct 2016 09:39:00 -0700 Subject: [9] RFR(M) 8166416: [AOT] Integrate JDK build changes and launcher 'jaotc' for AOT compiler In-Reply-To: References: <58114E0C.107@oracle.com> <58124A47.5010604@oracle.com> <581304B6.7030804@oracle.com> <0acdf13c-88d9-f904-7fcb-1f5d3683e30c@oracle.com> <58140818.1090907@oracle.com> Message-ID: <29d61039-f81f-2cd7-3240-fa2885e46bdf@oracle.com> Thank you, Erik On 10/31/16 3:11 AM, Erik Joelsson wrote: > Hello Vladimir, > > The new organization of the checks looks good. Just found one minor nit, > in lib-elf.m4, no need to AC_SUBST(ENABLE_AOT) again. Removed. Thanks, Vladimir > > /Erik > > > On 2016-10-29 04:23, Vladimir Kozlov wrote: >> Updated build webrevs (webrev.2) >> >> http://cr.openjdk.java.net/~kvn/aot/top.webrev.2/ >> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev.2/ >> >> Use one ENABLE_AOT variable in all config and make files (removed >> NEEDS_LIB_JELFSHIM). >> >> Moved HOTSPOT_SETUP_JVM_FEATURES after LIB_SETUP_LIBRARIES. >> >> First, check for linux-x64 presence of jdk.aot and jdk.vm.compiler >> directories. Set ENABLE_AOT=true if all are present. Throw config >> errors if --enable-aot is specified but sources are missing or on >> different platforms. >> >> Second, in lib-elf.m4, when ENABLE_AOT==true check if libelf.h is >> present and usable. >> Throw config errors if --enable-aot is specified but libelf.h is missing. >> Otherwise warnings and set ENABLE_AOT==false. >> >> Third, check ENABLE_AOT in HOTSPOT_SETUP_JVM_FEATURES to set >> JVM_FEATURES. >> >> Thanks, >> Vladimir >> >> On 10/28/16 3:27 AM, Erik Joelsson wrote: >>> >>> >>> On 2016-10-28 11:09, Volker Simonis wrote: >>>> On Fri, Oct 28, 2016 at 10:57 AM, Erik Joelsson >>>> wrote: >>>>> Hello, >>>>> >>>>> >>>>> On 2016-10-28 09:56, Vladimir Kozlov wrote: >>>>>> Webrevs updated in place (please, reload webpage to get new version). >>>>>> generated-configure.sh changes are included. >>>>>> >>>>>> http://cr.openjdk.java.net/~kvn/aot/top.webrev/ >>>>>> http://cr.openjdk.java.net/~kvn/aot/jdk.webrev/ >>>>>> http://cr.openjdk.java.net/~kvn/aot/hs.make.webrev/ >>>>>> >>>>>> libelfshim generation is guarded by ENABLE_AOT as Volker suggested. >>>>>> >>>>>> I did not figure out how switch off ENABLE_AOT if libelf not >>>>>> found. So I >>>>>> changed warning to configuration error: >>>>>> >>>>>> error: libelf not found, cannot build AOT. Disable AOT build: >>>>>> --enable-aot=no. You might be able to fix this by running 'sudo >>>>>> yum install >>>>>> elfutils-libelf-devel'. >>>>>> >>>>>> Which is not correct since AOT build is enabled by default on >>>>>> linux-x64. >>>>>> Need help with this. >>>>>> >>>>> I think it should be fine moving HOTSPOT_SETUP_JVM_FEATURES to after >>>>> LIB_SETUP_LIBRARIES in configure.ac. (Note that many of these calls >>>>> are just >>>>> ordered as they happened to be, though some really need to be >>>>> before or >>>>> after others. I cannot find a reason for these two.) Then you can >>>>> enable/disable AOT after looking for libelf. It depends on what >>>>> kind of >>>>> behavior you want. >>>>> >>>>> If libelf isn't commonly installed, then having configure use it, >>>>> and enable >>>>> AOT if it is, and semi silently just disabling AOT if it's not is >>>>> fine. We >>>>> can define our Oracle builds to explicitly set --enable-aot to get >>>>> an early >>>>> fail on a missing dependency. That's inline with current configure >>>>> behavior >>>>> so I like that. >>>> I'm fine with that. But then we shouldn't warn if we don't find >>>> libelf. We should only warn, if '--enable-aot' was explicitly >>>> requested by the user and if this request can't be fulfilled because >>>> of the missing libelf dependency. >>> --enable-aot and not finding libelf should result in an error. If on >>> linux-x64 where aot is default enabled, there should be a warning >>> that it was disabled because libelf is missing. This warning >>> would now need to be in the ENABLE_AOT logic. >>> >>> /Erik >