From kim.barrett at oracle.com Sun Nov 2 04:12:25 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 2 Nov 2014 00:12:25 -0400 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes Message-ID: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> Please review this fix for a nightly test failure: Webrev: http://cr.openjdk.java.net/~kbarrett/8062036/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8062036 I'll also need a sponsor for this. The failing test is run with the -XX:+ScavengeALot option. That leads to collections during VM initialization, and one of them might need the surrogate locking thread before that thread has been created. The proximate cause of the failure, use of -XX:+ScavengeALot (or -XX:+FullGCALot) leading to such problematic collections, is being addressed by suppressing "gc alot" until VM initialization is complete, conditionalizing it on Threads::is_vm_complete(), rather than the previously used is_init_completed(). The latter function was never really the proper predicate for this decision, but happened to work for collectors that don't use a Java thread as part of their implementation; that predicate has never been adequate for G1 or CMS, which (sometimes) involve the SLT (Java) thread. However, this doesn't address the possibility of a collection with some other cause occurring before SLT creation in VM initialization, and failing because it requires the SLT. This might happen if, for example, the initial memory configuration options are overly restrictive. There are limited options to deal with this situation. * In some cases it might be possible to report OOME, but there are no application threads running yet that might do anything useful with it. It's also not clear this should be treated as an OOME; there may be lots of available memory, if only the collector could actually run. * Creating the SLT on demand isn't a reliable solution; such a collection could occur before it is possible to create and run a Java thread. (The SLT is created soon after Java thread creation is possible, but there is a period between when the heap supports allocation (which might trigger GC) and the point where Java thread creation is allowed.) * Instead we're changing the reporting of the SLT being needed before created situation to use vm_exit_during_initialization() with a message about what happened, instead of the previous segfault. From kim.barrett at oracle.com Sun Nov 2 20:14:13 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 2 Nov 2014 15:14:13 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> Message-ID: <113E829A-9E81-44D8-8F91-C280E73CDBF1@oracle.com> On Nov 2, 2014, at 12:12 AM, Kim Barrett wrote: > > Please review this fix for a nightly test failure: > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062036 > > I'll also need a sponsor for this. Forgot to mention testing in my review request: jtreg hotspot/test/[closed/]{gc,runtime}, locally (linux x86_64) refworkload using G1 and CMS, locally (linux x86_64) jprt -testset hotspot From yiyeguhu at gmail.com Sun Nov 2 21:41:47 2014 From: yiyeguhu at gmail.com (Tao Mao) Date: Sun, 2 Nov 2014 13:41:47 -0800 Subject: [8u40] Request for review (xs) : backport of 8034056 - assert(_heap_alignment >= _space_alignment) failed: heap_alignment less than space_alignment In-Reply-To: <5452A52E.3030508@oracle.com> References: <54526B19.3020606@oracle.com> <5452A52E.3030508@oracle.com> Message-ID: Ok, thanks. -Tao On Thu, Oct 30, 2014 at 1:53 PM, Jon Masamitsu wrote: > Tao, > > Thanks for the review. A backport bug will be > created when this is integrated (since it has a > CR corresponding to one pushed to jdk9). > > > Jon > > > On 10/30/2014 10:49 AM, Tao Mao wrote: > > Looks clean. Do we have a backport bug report on this, or it's not > required? > > Thanks. > Tao > > On Thu, Oct 30, 2014 at 9:45 AM, Jon Masamitsu > wrote: > >> This is a backport of 8034056. >> https://bugs.openjdk.java.net/browse/JDK-8034056 >> >> This is the same fix that went into jdk9 but the >> patch did not apply cleanly so I redid it. >> >> http://cr.openjdk.java.net/~jmasa/8034056/webrev.00/ >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcus.larsson at oracle.com Mon Nov 3 09:49:35 2014 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 03 Nov 2014 10:49:35 +0100 Subject: RFR (XS): 8054491: Remove wrong assert and refactor code in G1CollectorPolicy::record_concurrent_mark_end In-Reply-To: <53F1ABB5.2080406@oracle.com> References: <53F1ABB5.2080406@oracle.com> Message-ID: <54574FAF.3000304@oracle.com> Hi, Looking for another review for this. Rebased and resolved merge conflicts, updated webrev: http://cr.openjdk.java.net/~mlarsson/8054491/webrev.01/ Thanks, Marcus On 18/08/14 09:31, Marcus Larsson wrote: > Hi, > > Can I have reviews for this small change cleaning up and refactoring > record_concurrent_mark_end()? > > CR: > http://cr.openjdk.java.net/~mgerdin/mlarsson/webrev-8054491/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8054491 > > Testing: > jprt > > Thanks, > Marcus From mikael.gerdin at oracle.com Mon Nov 3 10:04:01 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 03 Nov 2014 11:04:01 +0100 Subject: RFR (S): 8049341: Parallelize clearing the next mark bitmap In-Reply-To: <543B96FB.6050302@oracle.com> References: <53FF0D4F.3040505@oracle.com> <540423BD.7090608@oracle.com> <1409561622.2755.37.camel@cirrus> <543B96FB.6050302@oracle.com> Message-ID: <54575311.9050705@oracle.com> Hi Marcus, On 2014-10-13 11:10, Marcus Larsson wrote: > Hi, > > > On 01/09/14 10:53, Thomas Schatzl wrote: >> Hi Marcus, >> > > [...] > >> >> I would prefer if the parallel iteration over the heap regions used >> the available G1CollectedHeap::heap_region_par_iterate_chunked() >> mechanism. Using it avoids imbalance of work due to not all regions >> being available. >> > > Fixed! > >> For this, the per-region claim value (HeapRegion::_claim) should >> probably be extracted from the HeapRegion (where it imo does not belong >> in the first place) because the concurrent bitmap clear may be >> interrupted by young GCs that use it too. Please add a separate >> refactoring CR for that. >> >> Then the pattern established by other parallel gc tasks that use >> heap_region_par_iterate_chunked() can be reused. >> > > Fixed in separate issue [1]. > >> With the recent changes I asked this change to be updated with, this >> task unfortunately also got a lot harder - it includes JDK-8038423 that >> allows arbitrary regions to be not committed - including the >> corresponding bitmaps. > > [...] > >> Please use the vm.gc.compact.* tests from UTE. They are also included in >> the vm.gc.* testlists. >> > > Tested without problems. > >> Thanks, >> Thomas >> >> >> > > Thank you for looking into this! > > New webrev: > http://cr.openjdk.java.net/~mlarsson/8049341/webrev.02/ Looks good! /Mikael > > Marcus > > [1]: https://bugs.openjdk.java.net/browse/JDK-8058298 From bengt.rutisson at oracle.com Mon Nov 3 10:07:56 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 03 Nov 2014 11:07:56 +0100 Subject: RFR (XS): 8054491: Remove wrong assert and refactor code in G1CollectorPolicy::record_concurrent_mark_end In-Reply-To: <54574FAF.3000304@oracle.com> References: <53F1ABB5.2080406@oracle.com> <54574FAF.3000304@oracle.com> Message-ID: <545753FC.2070406@oracle.com> Hi Marcus, Sorry for the late reply. On 2014-11-03 10:49, Marcus Larsson wrote: > Hi, > > Looking for another review for this. > > Rebased and resolved merge conflicts, updated webrev: > http://cr.openjdk.java.net/~mlarsson/8054491/webrev.01/ Looks good. Bengt > > Thanks, > Marcus > > On 18/08/14 09:31, Marcus Larsson wrote: >> Hi, >> >> Can I have reviews for this small change cleaning up and refactoring >> record_concurrent_mark_end()? >> >> CR: >> http://cr.openjdk.java.net/~mgerdin/mlarsson/webrev-8054491/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8054491 >> >> Testing: >> jprt >> >> Thanks, >> Marcus > From marcus.larsson at oracle.com Mon Nov 3 10:27:14 2014 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 03 Nov 2014 11:27:14 +0100 Subject: RFR (XS): 8054491: Remove wrong assert and refactor code in G1CollectorPolicy::record_concurrent_mark_end In-Reply-To: <545753FC.2070406@oracle.com> References: <53F1ABB5.2080406@oracle.com> <54574FAF.3000304@oracle.com> <545753FC.2070406@oracle.com> Message-ID: <54575882.6050708@oracle.com> Thanks for reviewing, Thomas and Bengt! Marcus On 03/11/14 11:07, Bengt Rutisson wrote: > > Hi Marcus, > > Sorry for the late reply. > > On 2014-11-03 10:49, Marcus Larsson wrote: >> Hi, >> >> Looking for another review for this. >> >> Rebased and resolved merge conflicts, updated webrev: >> http://cr.openjdk.java.net/~mlarsson/8054491/webrev.01/ > > Looks good. > > Bengt > >> >> Thanks, >> Marcus >> >> On 18/08/14 09:31, Marcus Larsson wrote: >>> Hi, >>> >>> Can I have reviews for this small change cleaning up and refactoring >>> record_concurrent_mark_end()? >>> >>> CR: >>> http://cr.openjdk.java.net/~mgerdin/mlarsson/webrev-8054491/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8054491 >>> >>> Testing: >>> jprt >>> >>> Thanks, >>> Marcus >> > From marcus.larsson at oracle.com Mon Nov 3 10:27:51 2014 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 03 Nov 2014 11:27:51 +0100 Subject: RFR (S): 8049341: Parallelize clearing the next mark bitmap In-Reply-To: <54575311.9050705@oracle.com> References: <53FF0D4F.3040505@oracle.com> <540423BD.7090608@oracle.com> <1409561622.2755.37.camel@cirrus> <543B96FB.6050302@oracle.com> <54575311.9050705@oracle.com> Message-ID: <545758A7.3060506@oracle.com> Thanks for reviewing, Thomas and Mikael! Marcus On 03/11/14 11:04, Mikael Gerdin wrote: > Hi Marcus, > > On 2014-10-13 11:10, Marcus Larsson wrote: >> Hi, >> >> >> On 01/09/14 10:53, Thomas Schatzl wrote: >>> Hi Marcus, >>> >> >> [...] >> >>> >>> I would prefer if the parallel iteration over the heap regions used >>> the available G1CollectedHeap::heap_region_par_iterate_chunked() >>> mechanism. Using it avoids imbalance of work due to not all regions >>> being available. >>> >> >> Fixed! >> >>> For this, the per-region claim value (HeapRegion::_claim) should >>> probably be extracted from the HeapRegion (where it imo does not belong >>> in the first place) because the concurrent bitmap clear may be >>> interrupted by young GCs that use it too. Please add a separate >>> refactoring CR for that. >>> >>> Then the pattern established by other parallel gc tasks that use >>> heap_region_par_iterate_chunked() can be reused. >>> >> >> Fixed in separate issue [1]. >> >>> With the recent changes I asked this change to be updated with, this >>> task unfortunately also got a lot harder - it includes JDK-8038423 that >>> allows arbitrary regions to be not committed - including the >>> corresponding bitmaps. >> >> [...] >> >>> Please use the vm.gc.compact.* tests from UTE. They are also >>> included in >>> the vm.gc.* testlists. >>> >> >> Tested without problems. >> >>> Thanks, >>> Thomas >>> >>> >>> >> >> Thank you for looking into this! >> >> New webrev: >> http://cr.openjdk.java.net/~mlarsson/8049341/webrev.02/ > > Looks good! > /Mikael > >> >> Marcus >> >> [1]: https://bugs.openjdk.java.net/browse/JDK-8058298 From bengt.rutisson at oracle.com Mon Nov 3 13:31:13 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 03 Nov 2014 14:31:13 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <545386E1.2050402@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> Message-ID: <545783A1.3050300@oracle.com> Hi Dima, Answers inline. On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: > Hi Bengt, > > Thanks a lot for your detailed feedback, we appreciate it very much! > > See comments inline. > > On 31.10.2014 1:09, Bengt Rutisson wrote: >> >> Hi Evgeniya, >> >> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>> Hi, >>> >>> Please review changes for 8062537, the OpenJDK/hotspot part of the >>> JDK-8019361 >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>> >>> Problem: Some tests explicitly set GC and fail when jtreg set >>> another GC. >>> Solution: Such tests marked with the jtreg tag "requires" to skip >>> test if there is a conflict >> >> Thanks for fixing this! It is really great that we finally start >> sorting this out. >> >> First a general comment. The @requires tag has been developed without >> much cooperation with the GC team. We did have a lot of feedback when >> it was first presented a year ago, but it does not seem like this >> feedback was incorporated into the @requires that was eventually built. > > We tried to implement as much developer's wishes as possible. But not > everything is possible, sorry for that. Yes, I'm sure you have done your best. It's just that we have been requesting this feature for 3 years and I was expecting us to be able to influence the feature much more than was the case now. > >> >> I think this change that gets proposed now is a big step forward and >> I won't object to it. But I am pretty convinced that we will soon run >> in to the limitations of the current @requires implementation and we >> will have to redo this work. >> >> Some of the points I don't really like about the @requires tag are: >> >> - the "vm.gc" abstraction is more limiting than helping. It would >> have been better to just "require" any command line flag. > "vm.gc" is an alias to a very popular flag. It's also possible to use: > vm.opt.UseG1GC == true instead. > > The table with all vars available in jtreg: > http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names The problem with having this matching built in to JTreg is that it makes it very hard to change. When we discussed this a year ago I think we said that JTreg should only provide a means to test against the command line and a hook for running some java code in the @requires tag. That way we could put logic like this in a test library that is under our control. This would make it easy for us to change and also enables us to use different logic for different versions. > >> - the requirement should be per @run tag. Right now we have to do >> what you did in this change and use vm.gc=null even when some tests >> could actually have been run when a GC was specified. > it would be great, but it will unlikely happen in jtreg, as well as > test case support. what do you mean with test case support? > >> - there are many tests that require more than just a specific GC. >> Often there are other flags that can't be changed either for the test >> to work properly. > > yes. conflicting GC is just the most popular problem caused by > conflicting options. > If we address this issue and we are satisfied with solution, we could > move further. Yes, I agree that taking one step at the time is good. Personally I would have preferred that the first step was a "just run the command line as specified in the @run tag" step. > >> >> Maybe this is not the right place to discuss the current >> implementation of the @requires tag. I just want to say that I'm not >> too happy about how the @requires tag turned out. But assuming we >> have to use it the way it is now I guess the proposed changeset looks >> good. > > yes, this thread is about change made by Evgeniya, not about jtreg :) > And thanks for reviewing it! Agreed. And as I said, I think the patch looks ok. I have not looked at all tests. But if they now pass with the combinations that we test with I guess they should be ok. > >> >>> Tested locally with different GC flags (-XX:+UseG1GC, >>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>> without any GC flag). Tests are being excluded as expected. No tests >>> failed because of the conflict. >> Have you tested with -Xconcgc too? It's an alias for >> -XX:+UseConcMarkSweepGC. > > '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) Ok. Thanks. > >> >> I think some of the test, like >> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >> with -XX:+UseParNewGC. Others, like >> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >> with -XX:-UseParNewGC. Could you test these two cases too? > > These two tests ignore vm flags. > Add @requires here is not necessary, but it will allow not execute the > tests when not needed. > So, if we run HS tests with 4 GC, we don't need to run these tests 4 > times, 1 should be enough. Do we really want to use the @requires functionality for this purpose? It seems like a way of misusing @requires. If we just want the tests to be run once I think Leonid's approach with tests lists seems more suitable. But are you sure that this is the reason for the @requires in this case? TestDefNewCMS does sound like a test that is DefNew specific. I don't see a reason to run it with ParNew. If it doesn't fail today it should probably be changed so that it does fail if it is run with the wrong GC. > >> Similarly it looks to me like there are tests that will fail if you >> run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >> >> Just a heads up. These two tests will soon be removed. I'm about to >> push a changeset that removes them: >> >> test/gc/startup_warnings/TestCMSIncrementalMode.java >> test/gc/startup_warnings/TestCMSNoIncrementalMode.java > okay, thank for letting us know. > >> >> Is there some way of making sure that all tests are run at one time >> or another. With this change there is a risk that some tests are >> never run and always skipped. Will we somehow be tracking what gets >> skipped and make sure that all tests are at least run once with the >> correct GC so that it is not skipped all the time? > > This is a very good question! > jtreg now doesn't report skipped tests, hopefully it will do soon, > after getting fix of: > https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 > > And yes, tracking tests which are not run is important thing. > @requires - is not the only to exclude test from execution. > > Other examples: > > /* > *@ignore > *@test > */ > ... > > /*@bug 4445555 > *@test > */ > ... > Such tests will never be run, because jtreg treats as test only files > with @test on the first place... > > So, making sure that tests do not disappear is important SQE task, we > know about that, we're thinking on solution (may be very actively). > But this subject for another discussion, not within RFR :) Right. Glad to hear that you are actively working on this! Bengt > > Thanks, > Dima > > > >> >> Thanks, >> Bengt >> >>> >>> Thanks, >>> Evgeniya Stepanova >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwha at google.com Mon Nov 3 17:16:01 2014 From: jwha at google.com (Jungwoo Ha) Date: Mon, 03 Nov 2014 17:16:01 +0000 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <5452A080.5090904@oracle.com> Message-ID: Nice catch! PTAL http://cr.openjdk.java.net/~rasbold/8061259/webrev.03/ On Thu Oct 30 2014 at 1:33:04 PM Bengt Rutisson wrote: > > Hi Jungwoo, > > > On 10/30/14 6:24 PM, Jungwoo Ha wrote: > > PTAL > http://cr.openjdk.java.net/~rasbold/8061259/webrev.02/ > > > Thanks! Looks good except for one detail. > > 1125 bool set_promotion_failed() { _has_promotion_failed = 1; } > 1126 bool reset_promotion_failed() { _has_promotion_failed = 0; } > > Since _has_promotion_failed is now a bool I don't think we should be > assigning 1 and 0 to it. We should be using true and false. > > Other than that it looks good to me. > > Thanks! > > Bengt > > > > > On Thu Oct 30 2014 at 12:28:19 AM Bengt Rutisson < > bengt.rutisson at oracle.com> wrote: > >> >> Hi again, >> >> One more minor thing. >> >> The methods has_promotion_failed(), set_promotion_failed() and >> reset_promotion_failed() are protected but they could be made private >> instead. >> >> Thanks, >> Bengt >> >> >> On 2014-10-30 08:09, Bengt Rutisson wrote: >> >> >> >> Hi Jungwoo, >> >> On 2014-10-29 23:51, Jungwoo Ha wrote: >> >> PTAL >> http://cr.openjdk.java.net/~rasbold/8061259/webrev.01/ >> >> I've looked a bit at the webrev. A couple of comments: >>> >>> Why do you use OrderAccess methods for writing and reading the >>> _has_promo_failed flag in has_promo_failed() and set_promot_failed() ? >>> >> >> I think that has no effect on x86, but I assumed that processors with >> weak memory model may want ordering of set/reset/has call. >> >> >> You don't need the OrderAccess methods for the weak memory models here >> either. You just race on reading the variable and if you see the "wrong" >> value you eventually take a lock (which will order all memory accesses) to >> read the variable properly. >> >> By removing the use of OrderAccess you can make >> ConcurrentMarkSweepGeneration::_has_promotion_failed a bool instead of a >> juint which simplifies the code a bit. >> >> >> Can we write out the full word "promotion" instead of just "promo" in >>> the variables and methods? >>> >> >> Done. >> >> Can we change the name of the flag from UseCMSFastPromotionFailure to >>> CMSFastPromotionFailure? Most CMS flags start with CMS and I don't think we >>> need the "Use" prefix. >>> >> >> Done. >> >> What do you think about making the flag true by default? At least for >>> JDK 9. If we decide to backport to JDK 8 or 7 it might be a good idea to >>> keep the default value as false. >>> >> >> Done. >> Let me know if there is anything for me to do to backport to JDK8 and 7. >> >> >> I think this fix would be worth backporting to JDK 8. I don't think there >> is much action required on your side. I created a backport bug for JDK 8 >> just to make sure that we don't forget it. It will be a little while before >> the 8 update repos are in a state to accept enhancements again. So, it is >> nice to have the backport bug to keep track of this. >> >> Backporting to JDK 7 requires some more work. Unless you have good >> arguments for why it is important to backport this to JDK 7 I don't think >> it is worth doing. >> >> >> Did you find the information provided by _fast_promo_failure_hitcount >>> useful for debugging? If it not too useful I would consider removing it >>> since it is cluttering up the code a bit. >>> >> >> I removed it. >> It was useful to for development, but I think it is no longer needed. >> >> >> Great. Thanks. >> >> One more comment. This code comment appears in two places just after we >> have taken the lock. >> >> 3365 if (CMSFastPromotionFailure && has_promotion_failed()) { >> 3366 // Caller must have checked already without synchronization. >> 3367 // Check again here while holding the lock. >> 3368 return NULL; >> 3369 } >> >> There is actually really no requirement that the caller must have checked >> has_promotion_failed() before calling the method. That's just an >> optimization. I think the first comment can be skipped and we just leave >> the second comment "// Check again here while holding the lock.". I would >> also suggest moving that comment up to the line just before the if >> statement. >> >> Thanks, >> Bengt >> >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.masamitsu at oracle.com Mon Nov 3 17:56:13 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 03 Nov 2014 09:56:13 -0800 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <545365C2.5040609@oracle.com> References: <545245C5.4050504@oracle.com> <545305C4.5070901@oracle.com> <545365C2.5040609@oracle.com> Message-ID: <5457C1BD.1010004@oracle.com> Tests look good. Reviewed. Jon On 10/31/2014 03:34 AM, Evgeniya Stepanova wrote: > Hi Jon! > > Please see comments inline > On 31.10.2014 7:45, Jon Masamitsu wrote: >> Evgeniya, >> >> I've only look at a few of the files. I'm not sure >> I'm interpreting the @requires correctly. If you >> reply to these initial comments, that will help >> me understand. >> >> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/test/gc/arguments/TestMaxNewSize.java.udiff.html >> >> + * @requires vm.gc=="null" >> Test seems to expect Serial GC. Should the requires >> be Serial or null? > The diff info is not full: > 25 * @test TestMaxNewSize > 26 * @key gc > 27 * @bug 7057939 > 28 * @requires vm.gc=="null" > 29 * @summary Make sure that MaxNewSize always has a useful value > after argument > 30 * processing. > 31 * @library /testlibrary > 32 * @build TestMaxNewSize > 33 * @run main TestMaxNewSize -XX:+UseSerialGC > 34 * @run main TestMaxNewSize -XX:+UseParallelGC > 35 * @run main TestMaxNewSize -XX:+UseConcMarkSweepGC > 36 * @run main TestMaxNewSize -XX:+UseG1GC > > So test run with all GCs. > >> >> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/test/gc/arguments/TestSerialHeapSizeFlags.java.udiff.html >> >> Comments say this is a test of Serial but null >> allowed. >> >> + * @requires vm.gc=="Serial" | vm.gc=="null" >> > Test itself set "-XX:+UseSerialGC" flag and launch separate java process. > Setting @requires in such cases aiming 2 targets: > 1)It is skipps if GC is for example ParallelGC. In concrete case you > could guess it only for serial GC, but in some tests you couldn't. > So you see test passed and think it passed with your options and > actually it passed with other GC. With requires you are totally sure > test run with option you set. I believe such tests should be changed > to pass options to the java processes, but now they doesn't. > 2)From the other hand, when run without options, tests always run (ALL > tests will run, no tests skipped at all) > >> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/test/gc/class_unloading/TestCMSClassUnloadingEnabledHWM.java.udiff.html >> >> If you're going to use this on all the collectors (allow vm.gc == null), >> then class loading should be turned off on all the GC's, not >> just CMS (-XX:-CMSClassUnloadingEnabled only affects CMS). > See previous comment. All tests you mentioned were not failed, and > "requires" added for them to make order in test run. >> >> Jon >> >> On 10/30/2014 07:05 AM, Evgeniya Stepanova wrote: >>> Hi, >>> >>> Please review changes for 8062537, the OpenJDK/hotspot part of the >>> JDK-8019361 >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>> >>> Problem: Some tests explicitly set GC and fail when jtreg set >>> another GC. >>> Solution: Such tests marked with the jtreg tag "requires" to skip >>> test if there is a conflict >>> >>> Tested locally with different GC flags (-XX:+UseG1GC, >>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>> without any GC flag). Tests are being excluded as expected. No tests >>> failed because of the conflict. >>> >>> Thanks, >>> Evgeniya Stepanova >>> >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.masamitsu at oracle.com Mon Nov 3 18:14:53 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 03 Nov 2014 10:14:53 -0800 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <545245C5.4050504@oracle.com> References: <545245C5.4050504@oracle.com> Message-ID: <5457C61D.6040903@oracle.com> Evgeniya, One suggestion on this review. http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/test/gc/defnew/HeapChangeLogging.java.frames.html This part of comment should be deleted. 33 * Test the output of G1SummarizeRSetStats in conjunction with G1SummarizeRSetStatsPeriod. The test doesn't use G1. Jon On 10/30/2014 07:05 AM, Evgeniya Stepanova wrote: > Hi, > > Please review changes for 8062537, the OpenJDK/hotspot part of the > JDK-8019361 > > bug: https://bugs.openjdk.java.net/browse/JDK-8062537 > fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ > > Problem: Some tests explicitly set GC and fail when jtreg set another GC. > Solution: Such tests marked with the jtreg tag "requires" to skip test > if there is a conflict > > Tested locally with different GC flags (-XX:+UseG1GC, > -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and > without any GC flag). Tests are being excluded as expected. No tests > failed because of the conflict. > > Thanks, > Evgeniya Stepanova > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Nov 3 18:41:03 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Nov 2014 13:41:03 -0500 Subject: RFR: 8062206: Remove unusable G1RSLogCheckCardTable command line argument Message-ID: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> Please review this removal of conditional code that appears to have never worked and doesn't have an apparent repair. Webrev: http://cr.openjdk.java.net/~kbarrett/8062206/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8062206 Testing: jtreg hotspot/test/[closed/]{gc,runtime}, locally (linux x86_64) refworkload using G1, locally (linux x86_64) jprt -testset hotspot I'll also need a sponsor for this. Removed from G1RemSet::updateRS() the assertion that all cards are clean after remembered set updating. This "invariant" hasn't held during the entire life of G1 in the hotspot repository. Presently, cards for young generation are colored "young_card". And there are a variety of reasons why cards colored "dirty_card" may remain so (see G1RemSet::refine_card()), so changing the assertion to test for no such cards doesn't work either. Removed the G1RSLogCheckCardTable develop option that controlled the bogus assertion, since that's all that option does. Removed CountNonCleanMemRegionClosure class, since it is no longer used. From jesper.wilhelmsson at oracle.com Mon Nov 3 18:48:46 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 03 Nov 2014 19:48:46 +0100 Subject: RFR: 8062206: Remove unusable G1RSLogCheckCardTable command line argument In-Reply-To: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> References: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> Message-ID: <5457CE0E.70705@oracle.com> Looks good! I can sponsor this. /Jesper Kim Barrett skrev 3/11/14 19:41: > Please review this removal of conditional code that appears to have never > worked and doesn't have an apparent repair. > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8062206/webrev/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062206 > > Testing: > jtreg hotspot/test/[closed/]{gc,runtime}, locally (linux x86_64) > refworkload using G1, locally (linux x86_64) > jprt -testset hotspot > > I'll also need a sponsor for this. > > Removed from G1RemSet::updateRS() the assertion that all cards are clean after > remembered set updating. This "invariant" hasn't held during the entire life > of G1 in the hotspot repository. Presently, cards for young generation are > colored "young_card". And there are a variety of reasons why cards colored > "dirty_card" may remain so (see G1RemSet::refine_card()), so changing the > assertion to test for no such cards doesn't work either. > > Removed the G1RSLogCheckCardTable develop option that controlled the bogus > assertion, since that's all that option does. > > Removed CountNonCleanMemRegionClosure class, since it is no longer used. > From Dmitry.Fazunenko at oracle.com Mon Nov 3 21:12:06 2014 From: Dmitry.Fazunenko at oracle.com (Dmitry Fazunenko) Date: Tue, 04 Nov 2014 00:12:06 +0300 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <545783A1.3050300@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> Message-ID: <5457EFA6.7050404@oracle.com> Hi Bengt, That's great that we have very closed visions! The general comment: currently, jtreg doesn't support any sort of plugins, so you can't provide a VM specific handler of the @requires or another tag. This is very annoying limitation and we have to live with it. A few more comments inline. On 03.11.2014 16:31, Bengt Rutisson wrote: > > > Hi Dima, > > Answers inline. > > On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >> Hi Bengt, >> >> Thanks a lot for your detailed feedback, we appreciate it very much! >> >> See comments inline. >> >> On 31.10.2014 1:09, Bengt Rutisson wrote: >>> >>> Hi Evgeniya, >>> >>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>> Hi, >>>> >>>> Please review changes for 8062537, the OpenJDK/hotspot part of the >>>> JDK-8019361 >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>> >>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>> another GC. >>>> Solution: Such tests marked with the jtreg tag "requires" to skip >>>> test if there is a conflict >>> >>> Thanks for fixing this! It is really great that we finally start >>> sorting this out. >>> >>> First a general comment. The @requires tag has been developed >>> without much cooperation with the GC team. We did have a lot of >>> feedback when it was first presented a year ago, but it does not >>> seem like this feedback was incorporated into the @requires that was >>> eventually built. >> >> We tried to implement as much developer's wishes as possible. But not >> everything is possible, sorry for that. > > Yes, I'm sure you have done your best. It's just that we have been > requesting this feature for 3 years and I was expecting us to be able > to influence the feature much more than was the case now. My personal hope: @requires will address ~90% of existing issues. > >> >>> >>> I think this change that gets proposed now is a big step forward and >>> I won't object to it. But I am pretty convinced that we will soon >>> run in to the limitations of the current @requires implementation >>> and we will have to redo this work. >>> >>> Some of the points I don't really like about the @requires tag are: >>> >>> - the "vm.gc" abstraction is more limiting than helping. It would >>> have been better to just "require" any command line flag. >> "vm.gc" is an alias to a very popular flag. It's also possible to use: >> vm.opt.UseG1GC == true instead. >> >> The table with all vars available in jtreg: >> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names > > The problem with having this matching built in to JTreg is that it > makes it very hard to change. When we discussed this a year ago I > think we said that JTreg should only provide a means to test against > the command line and a hook for running some java code in the > @requires tag. That way we could put logic like this in a test library > that is under our control. This would make it easy for us to change > and also enables us to use different logic for different versions. I would be glad to have own harness... > >> >>> - the requirement should be per @run tag. Right now we have to do >>> what you did in this change and use vm.gc=null even when some tests >>> could actually have been run when a GC was specified. >> it would be great, but it will unlikely happen in jtreg, as well as >> test case support. > > what do you mean with test case support? Hi Evgeniya, Under test case support I mean ability to treat each @run as a separate test. Now @test @run -XX:g1RegSize=1m MyTest @run -XX:g1RegSize=2m MyTest @run -XX:g1RegSize=4m MyTest class MyTest { } is always a single test. You can't exclude, or re-run a part of it. > >> >>> - there are many tests that require more than just a specific GC. >>> Often there are other flags that can't be changed either for the >>> test to work properly. >> >> yes. conflicting GC is just the most popular problem caused by >> conflicting options. >> If we address this issue and we are satisfied with solution, we could >> move further. > > Yes, I agree that taking one step at the time is good. Personally I > would have preferred that the first step was a "just run the command > line as specified in the @run tag" step. > >> >>> >>> Maybe this is not the right place to discuss the current >>> implementation of the @requires tag. I just want to say that I'm not >>> too happy about how the @requires tag turned out. But assuming we >>> have to use it the way it is now I guess the proposed changeset >>> looks good. >> >> yes, this thread is about change made by Evgeniya, not about jtreg :) >> And thanks for reviewing it! > > Agreed. And as I said, I think the patch looks ok. I have not looked > at all tests. But if they now pass with the combinations that we test > with I guess they should be ok. Excellent! Thanks a lot! > >> >>> >>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>> without any GC flag). Tests are being excluded as expected. No >>>> tests failed because of the conflict. >>> Have you tested with -Xconcgc too? It's an alias for >>> -XX:+UseConcMarkSweepGC. >> >> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) > > Ok. Thanks. > >> >>> >>> I think some of the test, like >>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >>> with -XX:+UseParNewGC. Others, like >>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >>> with -XX:-UseParNewGC. Could you test these two cases too? >> >> These two tests ignore vm flags. >> Add @requires here is not necessary, but it will allow not execute >> the tests when not needed. >> So, if we run HS tests with 4 GC, we don't need to run these tests 4 >> times, 1 should be enough. > > Do we really want to use the @requires functionality for this purpose? > It seems like a way of misusing @requires. If we just want the tests > to be run once I think Leonid's approach with tests lists seems more > suitable. No, it's not a purpose of course, it's just side effect :) > But are you sure that this is the reason for the @requires in this > case? TestDefNewCMS does sound like a test that is DefNew specific. I > don't see a reason to run it with ParNew. If it doesn't fail today it > should probably be changed so that it does fail if it is run with the > wrong GC. @requires - is not the silver bullet, but it's quite easy way to solve a lot of issues. I hope, @requires will allow to reduce the number of "selfish" tests, which produce a new java process to ignore vm flags coming from outside. No @requires, no other mechanism could 100% protect a test from running with conflicting options, but this is not the goal. If one runs tests with an exotic option, like a new G2 collector, there shouldn't mass failures caused by options conflicts. But a few failures could be handled manually. > >> >>> Similarly it looks to me like there are tests that will fail if you >>> run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>> >>> Just a heads up. These two tests will soon be removed. I'm about to >>> push a changeset that removes them: >>> >>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >> okay, thank for letting us know. >> >>> >>> Is there some way of making sure that all tests are run at one time >>> or another. With this change there is a risk that some tests are >>> never run and always skipped. Will we somehow be tracking what gets >>> skipped and make sure that all tests are at least run once with the >>> correct GC so that it is not skipped all the time? >> >> This is a very good question! >> jtreg now doesn't report skipped tests, hopefully it will do soon, >> after getting fix of: >> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >> >> And yes, tracking tests which are not run is important thing. >> @requires - is not the only to exclude test from execution. >> >> Other examples: >> >> /* >> *@ignore >> *@test >> */ >> ... >> >> /*@bug 4445555 >> *@test >> */ >> ... >> Such tests will never be run, because jtreg treats as test only files >> with @test on the first place... >> >> So, making sure that tests do not disappear is important SQE task, >> we know about that, we're thinking on solution (may be very >> actively). But this subject for another discussion, not within RFR :) > > Right. Glad to hear that you are actively working on this! I was going to say "not very actively", but never mind, we know about this problem. With introducing @requires mechanism it will become more important! Thanks for your comments! -- Dima > > Bengt > >> >> Thanks, >> Dima >> >> >> >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Nov 3 21:33:03 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Nov 2014 16:33:03 -0500 Subject: RFR: 8062206: Remove unusable G1RSLogCheckCardTable command line argument In-Reply-To: <5457CE0E.70705@oracle.com> References: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> <5457CE0E.70705@oracle.com> Message-ID: On Nov 3, 2014, at 1:48 PM, Jesper Wilhelmsson wrote: > > Looks good! > I can sponsor this. > /Jesper Thanks. From jon.masamitsu at oracle.com Mon Nov 3 21:31:27 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 03 Nov 2014 13:31:27 -0800 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> Message-ID: <5457F42F.2070509@oracle.com> Kim, Change looks good. I can sponsor. Let me know when the reviews are complete. Some suggestions. http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/g1/vm_operations_g1.cpp.frames.html > 216 if (SurrogateLockerThread* slt = ConcurrentMarkThread::slt()) { > 217 slt->manipulatePLL(SurrogateLockerThread::acquirePLL); > 218 } else { > 219 SurrogateLockerThread::report_missing_slt(); > 220 } I have a preference for SurrogateLockerThread* slt = ConcurrentMarkThread::slt(); if (slt != NULL) { slt->manipulatePLL(SurrogateLockerThread::acquirePLL); } else { SurrogateLockerThread::report_missing_slt(); } Similarly with http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/concurrentMarkSweep/vmCMSOperations.cpp.frames.html > 45 if (SurrogateLockerThread* slt = ConcurrentMarkSweepThread::slt()) { > 46 slt->manipulatePLL(SurrogateLockerThread::acquirePLL); > 47 } else { > 48 SurrogateLockerThread::report_missing_slt(); > 49 } http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/shared/concurrentGCThread.hpp.frames.html 96 // Terminate VM with error message that SLT needed but not yet created. I think it would read better as "SLT needed but not yet created." => "SLT is needed but has not yet been created." Jon On 11/01/2014 09:12 PM, Kim Barrett wrote: > Please review this fix for a nightly test failure: > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062036 > > I'll also need a sponsor for this. > > The failing test is run with the -XX:+ScavengeALot option. That leads > to collections during VM initialization, and one of them might need > the surrogate locking thread before that thread has been created. > > The proximate cause of the failure, use of -XX:+ScavengeALot (or > -XX:+FullGCALot) leading to such problematic collections, is being > addressed by suppressing "gc alot" until VM initialization is > complete, conditionalizing it on Threads::is_vm_complete(), rather > than the previously used is_init_completed(). The latter function was > never really the proper predicate for this decision, but happened to > work for collectors that don't use a Java thread as part of their > implementation; that predicate has never been adequate for G1 or CMS, > which (sometimes) involve the SLT (Java) thread. > > However, this doesn't address the possibility of a collection with > some other cause occurring before SLT creation in VM initialization, > and failing because it requires the SLT. This might happen if, for > example, the initial memory configuration options are overly > restrictive. There are limited options to deal with this situation. > > * In some cases it might be possible to report OOME, but there are no > application threads running yet that might do anything useful with it. > It's also not clear this should be treated as an OOME; there may be > lots of available memory, if only the collector could actually run. > > * Creating the SLT on demand isn't a reliable solution; such a > collection could occur before it is possible to create and run a Java > thread. (The SLT is created soon after Java thread creation is > possible, but there is a period between when the heap supports > allocation (which might trigger GC) and the point where Java thread > creation is allowed.) > > * Instead we're changing the reporting of the SLT being needed before > created situation to use vm_exit_during_initialization() with a > message about what happened, instead of the previous segfault. > From kim.barrett at oracle.com Mon Nov 3 22:13:00 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Nov 2014 17:13:00 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <5457F42F.2070509@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> Message-ID: On Nov 3, 2014, at 4:31 PM, Jon Masamitsu wrote: > > Kim, > > Change looks good. I can sponsor. Let me know when the reviews > are complete. Thanks. > Some suggestions. > > http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/g1/vm_operations_g1.cpp.frames.html > >> 216 if (SurrogateLockerThread* slt = ConcurrentMarkThread::slt()) { >> 217 slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >> 218 } else { >> 219 SurrogateLockerThread::report_missing_slt(); >> 220 } > > I have a preference for > > SurrogateLockerThread* slt = ConcurrentMarkThread::slt(); > if (slt != NULL) { > slt->manipulatePLL(SurrogateLockerThread::acquirePLL); > } else { > SurrogateLockerThread::report_missing_slt(); > } I like declarations in conditions (also applicable to switch, for, and while statements) because it limits the scope of the declared variable. Though I tend to not use that form when the true case is large, because I dislike having an else clause that is a long way from the associated condition. So I would typically write either if (T v = init()) { /* small amount of code using v */ } else { /* code for !v */ } or T v = init() if (!v) { /* small block of code for !v */ } else { /* code using v */ } or (pretty commonly) T v = init(); if (!v) { /* small amount of code for !v, with break/return avoiding code using v */ } /* larger block of code using v */ And if I can?t do one of those I?d probably factor out into helpers to allow one of those. I?m willing to change my ways and drop this idiom though if folks dislike or find it too odd or difficult to read. > http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/shared/concurrentGCThread.hpp.frames.html > > 96 // Terminate VM with error message that SLT needed but not yet created. > > I think it would read better as > > "SLT needed but not yet created." => "SLT is needed but has not yet been created." Sure. I guess this text *is* a bit more user-facing than an assert message, so grammar is more important. From kim.barrett at oracle.com Mon Nov 3 22:13:50 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Nov 2014 17:13:50 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <5457F42F.2070509@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> Message-ID: <536C7B03-F842-489D-B9B8-E6939B847CB4@oracle.com> On Nov 3, 2014, at 4:31 PM, Jon Masamitsu wrote: > > Kim, > > Change looks good. I can sponsor. Let me know when the reviews > are complete. Thanks. > Some suggestions. > > http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/g1/vm_operations_g1.cpp.frames.html > >> 216 if (SurrogateLockerThread* slt = ConcurrentMarkThread::slt()) { >> 217 slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >> 218 } else { >> 219 SurrogateLockerThread::report_missing_slt(); >> 220 } > > I have a preference for > > SurrogateLockerThread* slt = ConcurrentMarkThread::slt(); > if (slt != NULL) { > slt->manipulatePLL(SurrogateLockerThread::acquirePLL); > } else { > SurrogateLockerThread::report_missing_slt(); > } I like declarations in conditions (also applicable to switch, for, and while statements) because it limits the scope of the declared variable. Though I tend to not use that form when the true case is large, because I dislike having an else clause that is a long way from the associated condition. So I would typically write either if (T v = init()) { /* small amount of code using v */ } else { /* code for !v */ } or T v = init() if (!v) { /* small block of code for !v */ } else { /* code using v */ } or (pretty commonly) T v = init(); if (!v) { /* small amount of code for !v, with break/return avoiding code using v */ } /* larger block of code using v */ And if I can?t do one of those I?d probably factor out into helpers to allow one of those. I?m willing to change my ways and drop this idiom though if folks dislike or find it too odd or difficult to read. > http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/shared/concurrentGCThread.hpp.frames.html > > 96 // Terminate VM with error message that SLT needed but not yet created. > > I think it would read better as > > "SLT needed but not yet created." => "SLT is needed but has not yet been created." Sure. I guess this text *is* a bit more user-facing than an assert message, so grammar is more important. From John.Coomes at oracle.com Mon Nov 3 23:20:00 2014 From: John.Coomes at oracle.com (John Coomes) Date: Mon, 3 Nov 2014 15:20:00 -0800 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert Message-ID: <21592.3488.92171.340837@mykonos.us.oracle.com> Hi all, Please review this small fix for 8060467: CMS: small OldPLABSize and -XX:-ResizePLAB cause assert(ResizePLAB || n_blks == OldPLABSize) failed: Error http://cr.openjdk.java.net/~jcoomes/9/8060467-cms-plab-assert/ -John From kim.barrett at oracle.com Mon Nov 3 23:43:28 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Nov 2014 18:43:28 -0500 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert In-Reply-To: <21592.3488.92171.340837@mykonos.us.oracle.com> References: <21592.3488.92171.340837@mykonos.us.oracle.com> Message-ID: On Nov 3, 2014, at 6:20 PM, John Coomes wrote: > > Please review this small fix for > > 8060467: CMS: small OldPLABSize and -XX:-ResizePLAB cause > assert(ResizePLAB || n_blks == OldPLABSize) failed: Error > > http://cr.openjdk.java.net/~jcoomes/9/8060467-cms-plab-assert/ Looks good. From bengt.rutisson at oracle.com Tue Nov 4 08:40:43 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 04 Nov 2014 09:40:43 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5457EFA6.7050404@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> Message-ID: <5458910B.2070100@oracle.com> Hi Dima, Thanks for the answers. I think the currently proposed patch is a good start. We will have to evolve the @requires tag in the future, but let's have that discussion separate from this review. And we can start that discussion later when we have more experience with the current version of @requires. Thanks for doing this! Bengt On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: > Hi Bengt, > > That's great that we have very closed visions! > > The general comment: currently, jtreg doesn't support any sort of > plugins, so you can't provide a VM specific handler of the @requires > or another tag. This is very annoying limitation and we have to live > with it. > > A few more comments inline. > > > On 03.11.2014 16:31, Bengt Rutisson wrote: >> >> >> Hi Dima, >> >> Answers inline. >> >> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>> Hi Bengt, >>> >>> Thanks a lot for your detailed feedback, we appreciate it very much! >>> >>> See comments inline. >>> >>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>> >>>> Hi Evgeniya, >>>> >>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>> Hi, >>>>> >>>>> Please review changes for 8062537, the OpenJDK/hotspot part of the >>>>> JDK-8019361 >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>> >>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>> another GC. >>>>> Solution: Such tests marked with the jtreg tag "requires" to skip >>>>> test if there is a conflict >>>> >>>> Thanks for fixing this! It is really great that we finally start >>>> sorting this out. >>>> >>>> First a general comment. The @requires tag has been developed >>>> without much cooperation with the GC team. We did have a lot of >>>> feedback when it was first presented a year ago, but it does not >>>> seem like this feedback was incorporated into the @requires that >>>> was eventually built. >>> >>> We tried to implement as much developer's wishes as possible. But >>> not everything is possible, sorry for that. >> >> Yes, I'm sure you have done your best. It's just that we have been >> requesting this feature for 3 years and I was expecting us to be able >> to influence the feature much more than was the case now. > > My personal hope: @requires will address ~90% of existing issues. > >> >>> >>>> >>>> I think this change that gets proposed now is a big step forward >>>> and I won't object to it. But I am pretty convinced that we will >>>> soon run in to the limitations of the current @requires >>>> implementation and we will have to redo this work. >>>> >>>> Some of the points I don't really like about the @requires tag are: >>>> >>>> - the "vm.gc" abstraction is more limiting than helping. It would >>>> have been better to just "require" any command line flag. >>> "vm.gc" is an alias to a very popular flag. It's also possible to use: >>> vm.opt.UseG1GC == true instead. >>> >>> The table with all vars available in jtreg: >>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >> >> The problem with having this matching built in to JTreg is that it >> makes it very hard to change. When we discussed this a year ago I >> think we said that JTreg should only provide a means to test against >> the command line and a hook for running some java code in the >> @requires tag. That way we could put logic like this in a test >> library that is under our control. This would make it easy for us to >> change and also enables us to use different logic for different versions. > > I would be glad to have own harness... > >> >>> >>>> - the requirement should be per @run tag. Right now we have to do >>>> what you did in this change and use vm.gc=null even when some tests >>>> could actually have been run when a GC was specified. >>> it would be great, but it will unlikely happen in jtreg, as well as >>> test case support. >> >> what do you mean with test case support? Hi Evgeniya, > > Under test case support I mean ability to treat each @run as a > separate test. Now > > @test > @run -XX:g1RegSize=1m MyTest > @run -XX:g1RegSize=2m MyTest > @run -XX:g1RegSize=4m MyTest > class MyTest { > } > > is always a single test. You can't exclude, or re-run a part of it. > > >> >>> >>>> - there are many tests that require more than just a specific GC. >>>> Often there are other flags that can't be changed either for the >>>> test to work properly. >>> >>> yes. conflicting GC is just the most popular problem caused by >>> conflicting options. >>> If we address this issue and we are satisfied with solution, we >>> could move further. >> >> Yes, I agree that taking one step at the time is good. Personally I >> would have preferred that the first step was a "just run the command >> line as specified in the @run tag" step. >> >>> >>>> >>>> Maybe this is not the right place to discuss the current >>>> implementation of the @requires tag. I just want to say that I'm >>>> not too happy about how the @requires tag turned out. But assuming >>>> we have to use it the way it is now I guess the proposed changeset >>>> looks good. >>> >>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>> And thanks for reviewing it! >> >> Agreed. And as I said, I think the patch looks ok. I have not looked >> at all tests. But if they now pass with the combinations that we test >> with I guess they should be ok. > > Excellent! Thanks a lot! > >> >>> >>>> >>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>> without any GC flag). Tests are being excluded as expected. No >>>>> tests failed because of the conflict. >>>> Have you tested with -Xconcgc too? It's an alias for >>>> -XX:+UseConcMarkSweepGC. >>> >>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >> >> Ok. Thanks. >> >>> >>>> >>>> I think some of the test, like >>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >>>> with -XX:+UseParNewGC. Others, like >>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >>>> with -XX:-UseParNewGC. Could you test these two cases too? >>> >>> These two tests ignore vm flags. >>> Add @requires here is not necessary, but it will allow not execute >>> the tests when not needed. >>> So, if we run HS tests with 4 GC, we don't need to run these tests 4 >>> times, 1 should be enough. >> >> Do we really want to use the @requires functionality for this >> purpose? It seems like a way of misusing @requires. If we just want >> the tests to be run once I think Leonid's approach with tests lists >> seems more suitable. > > No, it's not a purpose of course, it's just side effect :) > > >> But are you sure that this is the reason for the @requires in this >> case? TestDefNewCMS does sound like a test that is DefNew specific. I >> don't see a reason to run it with ParNew. If it doesn't fail today it >> should probably be changed so that it does fail if it is run with the >> wrong GC. > > @requires - is not the silver bullet, but it's quite easy way to solve > a lot of issues. > > I hope, @requires will allow to reduce the number of "selfish" tests, > which produce a new java process to ignore vm flags coming from > outside. No @requires, no other mechanism could 100% protect a test > from running with conflicting options, but this is not the goal. > > If one runs tests with an exotic option, like a new G2 collector, > there shouldn't mass failures caused by options conflicts. But a few > failures could be handled manually. > > >> >>> >>>> Similarly it looks to me like there are tests that will fail if you >>>> run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>> >>>> Just a heads up. These two tests will soon be removed. I'm about to >>>> push a changeset that removes them: >>>> >>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>> okay, thank for letting us know. >>> >>>> >>>> Is there some way of making sure that all tests are run at one time >>>> or another. With this change there is a risk that some tests are >>>> never run and always skipped. Will we somehow be tracking what gets >>>> skipped and make sure that all tests are at least run once with the >>>> correct GC so that it is not skipped all the time? >>> >>> This is a very good question! >>> jtreg now doesn't report skipped tests, hopefully it will do soon, >>> after getting fix of: >>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>> >>> And yes, tracking tests which are not run is important thing. >>> @requires - is not the only to exclude test from execution. >>> >>> Other examples: >>> >>> /* >>> *@ignore >>> *@test >>> */ >>> ... >>> >>> /*@bug 4445555 >>> *@test >>> */ >>> ... >>> Such tests will never be run, because jtreg treats as test only >>> files with @test on the first place... >>> >>> So, making sure that tests do not disappear is important SQE task, >>> we know about that, we're thinking on solution (may be very >>> actively). But this subject for another discussion, not within RFR :) >> >> Right. Glad to hear that you are actively working on this! > > I was going to say "not very actively", but never mind, we know about > this problem. With introducing @requires mechanism it will become more > important! > > > Thanks for your comments! > > -- Dima > > >> >> Bengt >> >>> >>> Thanks, >>> Dima >>> >>> >>> >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Tue Nov 4 09:10:02 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Nov 2014 10:10:02 +0100 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert In-Reply-To: <21592.3488.92171.340837@mykonos.us.oracle.com> References: <21592.3488.92171.340837@mykonos.us.oracle.com> Message-ID: <1415092202.3101.2.camel@oracle.com> Hi, On Mon, 2014-11-03 at 15:20 -0800, John Coomes wrote: > Hi all, > > Please review this small fix for > > 8060467: CMS: small OldPLABSize and -XX:-ResizePLAB cause > assert(ResizePLAB || n_blks == OldPLABSize) failed: Error > > http://cr.openjdk.java.net/~jcoomes/9/8060467-cms-plab-assert/ looks good. Could you please add a regression test? Thanks, Thomas From mikael.gerdin at oracle.com Tue Nov 4 09:15:36 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 04 Nov 2014 10:15:36 +0100 Subject: RFR: 8062206: Remove unusable G1RSLogCheckCardTable command line argument In-Reply-To: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> References: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> Message-ID: <54589938.1090602@oracle.com> Hi Kim, On 2014-11-03 19:41, Kim Barrett wrote: > Please review this removal of conditional code that appears to have never > worked and doesn't have an apparent repair. > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8062206/webrev/ Looks good. /Mikael > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062206 > > Testing: > jtreg hotspot/test/[closed/]{gc,runtime}, locally (linux x86_64) > refworkload using G1, locally (linux x86_64) > jprt -testset hotspot > > I'll also need a sponsor for this. > > Removed from G1RemSet::updateRS() the assertion that all cards are clean after > remembered set updating. This "invariant" hasn't held during the entire life > of G1 in the hotspot repository. Presently, cards for young generation are > colored "young_card". And there are a variety of reasons why cards colored > "dirty_card" may remain so (see G1RemSet::refine_card()), so changing the > assertion to test for no such cards doesn't work either. > > Removed the G1RSLogCheckCardTable develop option that controlled the bogus > assertion, since that's all that option does. > > Removed CountNonCleanMemRegionClosure class, since it is no longer used. > From bengt.rutisson at oracle.com Tue Nov 4 09:13:25 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 04 Nov 2014 10:13:25 +0100 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> Message-ID: <545898B5.4080602@oracle.com> Hi Kim, The change looks good. Thanks for fixing this. Regarding assignments in if statements; I thought we had a style where we use booleans for comparisons. This means that I would prefer Jon's suggestion since the comparison will be on a bool value rather than relying on that NULL is interpreted as false. The hotspot style guidelines does not cover a lot and I guess it is open for interpretation whether or not this case is covered. But there is a short sentence about booleans: https://wiki.openjdk.java.net/display/HotSpot/StyleGuide#StyleGuide-Booleans I interpret "use true & false (not 1 & 0)" as that NULL should also not be used as a bool value. Thanks, Bengt On 2014-11-03 23:13, Kim Barrett wrote: > On Nov 3, 2014, at 4:31 PM, Jon Masamitsu wrote: >> Kim, >> >> Change looks good. I can sponsor. Let me know when the reviews >> are complete. > Thanks. > >> Some suggestions. >> >> http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/g1/vm_operations_g1.cpp.frames.html >> >>> 216 if (SurrogateLockerThread* slt = ConcurrentMarkThread::slt()) { >>> 217 slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >>> 218 } else { >>> 219 SurrogateLockerThread::report_missing_slt(); >>> 220 } >> I have a preference for >> >> SurrogateLockerThread* slt = ConcurrentMarkThread::slt(); >> if (slt != NULL) { >> slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >> } else { >> SurrogateLockerThread::report_missing_slt(); >> } > I like declarations in conditions (also applicable to switch, for, and while statements) because it limits the scope of the declared variable. Though I tend to not use that form when the true case is large, because I dislike having an else clause that is a long way from the associated condition. So I would typically write either > > if (T v = init()) { > /* small amount of code using v */ > } else { > /* code for !v */ > } > > or > > T v = init() > if (!v) { > /* small block of code for !v */ > } else { > /* code using v */ > } > > or (pretty commonly) > > T v = init(); > if (!v) { > /* small amount of code for !v, with break/return avoiding code using v */ > } > /* larger block of code using v */ > > And if I can?t do one of those I?d probably factor out into helpers to allow one of those. > > I?m willing to change my ways and drop this idiom though if folks dislike or find it too odd or difficult to read. > >> http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/shared/concurrentGCThread.hpp.frames.html >> >> 96 // Terminate VM with error message that SLT needed but not yet created. >> >> I think it would read better as >> >> "SLT needed but not yet created." => "SLT is needed but has not yet been created." > Sure. I guess this text *is* a bit more user-facing than an assert message, so grammar is more important. > From bengt.rutisson at oracle.com Tue Nov 4 09:19:10 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 04 Nov 2014 10:19:10 +0100 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <5452A080.5090904@oracle.com> Message-ID: <54589A0E.3000400@oracle.com> Hi Jungwoo, One final nit. The two setters don't need a return value. They could be void. 1125 bool set_promotion_failed() { _has_promotion_failed = true; } 1126 bool reset_promotion_failed() { _has_promotion_failed = false; } Bengt On 2014-11-03 18:16, Jungwoo Ha wrote: > Nice catch! > PTAL > http://cr.openjdk.java.net/~rasbold/8061259/webrev.03/ > > > On Thu Oct 30 2014 at 1:33:04 PM Bengt Rutisson > > wrote: > > > Hi Jungwoo, > > > On 10/30/14 6:24 PM, Jungwoo Ha wrote: >> PTAL >> http://cr.openjdk.java.net/~rasbold/8061259/webrev.02/ >> > > Thanks! Looks good except for one detail. > > 1125 bool set_promotion_failed() { _has_promotion_failed = 1; } > 1126 bool reset_promotion_failed() { _has_promotion_failed = 0; } > > Since _has_promotion_failed is now a bool I don't think we should > be assigning 1 and 0 to it. We should be using true and false. > > Other than that it looks good to me. > > Thanks! > > Bengt > > >> >> >> On Thu Oct 30 2014 at 12:28:19 AM Bengt Rutisson >> > wrote: >> >> >> Hi again, >> >> One more minor thing. >> >> The methods has_promotion_failed(), set_promotion_failed() >> and reset_promotion_failed() are protected but they could be >> made private instead. >> >> Thanks, >> Bengt >> >> >> On 2014-10-30 08:09, Bengt Rutisson wrote: >>> >>> >>> Hi Jungwoo, >>> >>> On 2014-10-29 23:51, Jungwoo Ha wrote: >>>> PTAL >>>> http://cr.openjdk.java.net/~rasbold/8061259/webrev.01/ >>>> >>>> >>>> I've looked a bit at the webrev. A couple of comments: >>>> >>>> Why do you use OrderAccess methods for writing and >>>> reading the _has_promo_failed flag in >>>> has_promo_failed() and set_promot_failed() ? >>>> >>>> >>>> I think that has no effect on x86, but I assumed that >>>> processors with weak memory model may want ordering of >>>> set/reset/has call. >>> >>> You don't need the OrderAccess methods for the weak memory >>> models here either. You just race on reading the variable >>> and if you see the "wrong" value you eventually take a lock >>> (which will order all memory accesses) to read the variable >>> properly. >>> >>> By removing the use of OrderAccess you can make >>> ConcurrentMarkSweepGeneration::_has_promotion_failed a bool >>> instead of a juint which simplifies the code a bit. >>> >>>> >>>> Can we write out the full word "promotion" instead of >>>> just "promo" in the variables and methods? >>>> >>>> >>>> Done. >>>> >>>> Can we change the name of the flag from >>>> UseCMSFastPromotionFailure to CMSFastPromotionFailure? >>>> Most CMS flags start with CMS and I don't think we need >>>> the "Use" prefix. >>>> >>>> >>>> Done. >>>> >>>> What do you think about making the flag true by >>>> default? At least for JDK 9. If we decide to backport >>>> to JDK 8 or 7 it might be a good idea to keep the >>>> default value as false. >>>> >>>> >>>> Done. >>>> Let me know if there is anything for me to do to backport >>>> to JDK8 and 7. >>> >>> I think this fix would be worth backporting to JDK 8. I >>> don't think there is much action required on your side. I >>> created a backport bug for JDK 8 just to make sure that we >>> don't forget it. It will be a little while before the 8 >>> update repos are in a state to accept enhancements again. >>> So, it is nice to have the backport bug to keep track of this. >>> >>> Backporting to JDK 7 requires some more work. Unless you >>> have good arguments for why it is important to backport this >>> to JDK 7 I don't think it is worth doing. >>> >>>> >>>> Did you find the information provided by >>>> _fast_promo_failure_hitcount useful for debugging? If >>>> it not too useful I would consider removing it since it >>>> is cluttering up the code a bit. >>>> >>>> >>>> I removed it. >>>> It was useful to for development, but I think it is no >>>> longer needed. >>> >>> Great. Thanks. >>> >>> One more comment. This code comment appears in two places >>> just after we have taken the lock. >>> >>> 3365 if (CMSFastPromotionFailure && has_promotion_failed()) { >>> 3366 // Caller must have checked already without >>> synchronization. >>> 3367 // Check again here while holding the lock. >>> 3368 return NULL; >>> 3369 } >>> >>> There is actually really no requirement that the caller must >>> have checked has_promotion_failed() before calling the >>> method. That's just an optimization. I think the first >>> comment can be skipped and we just leave the second comment >>> "// Check again here while holding the lock.". I would also >>> suggest moving that comment up to the line just before the >>> if statement. >>> >>> Thanks, >>> Bengt >>> >>> >>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Tue Nov 4 09:29:50 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Nov 2014 10:29:50 +0100 Subject: RFR: 8062206: Remove unusable G1RSLogCheckCardTable command line argument In-Reply-To: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> References: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> Message-ID: <1415093390.5218.1.camel@oracle.com> Hi Kim, On Mon, 2014-11-03 at 13:41 -0500, Kim Barrett wrote: > Please review this removal of conditional code that appears to have never > worked and doesn't have an apparent repair. > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8062206/webrev/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062206 [...] > > Removed from G1RemSet::updateRS() the assertion that all cards are clean after > remembered set updating. This "invariant" hasn't held during the entire life > of G1 in the hotspot repository. Presently, cards for young generation are > colored "young_card". And there are a variety of reasons why cards colored > "dirty_card" may remain so (see G1RemSet::refine_card()), so changing the > assertion to test for no such cards doesn't work either. > > Removed the G1RSLogCheckCardTable develop option that controlled the bogus > assertion, since that's all that option does. > > Removed CountNonCleanMemRegionClosure class, since it is no longer used. Quick grepping showed that also CardTableModRefBS::mod_card_iterate() is not used any more. Imo it should be removed too. Thanks, Thomas From thomas.schatzl at oracle.com Tue Nov 4 09:55:01 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Nov 2014 10:55:01 +0100 Subject: [RFR] 8016752: Test case for 8005857/8016740 (assert in GC_locker from PSOldGen::expand with -XX:+PrintGCDetails and Verbose) In-Reply-To: <54476AD3.3050907@oracle.com> References: <54476AD3.3050907@oracle.com> Message-ID: <1415094901.5218.5.camel@oracle.com> Hi Andrey, On Wed, 2014-10-22 at 12:29 +0400, Andrey Zakharov wrote: > Hi, team. > Here is simple test for JVM don't crash when its with PSOldGen, low > memory and with -XX:+PrintGCDetails and -XX:+Verbose flags > (https://bugs.openjdk.java.net/browse/JDK-8016740) based on Crasher > attached to original incident bug > > webrev: > http://cr.openjdk.java.net/~azakharov/8016752/webrev/ The test looks okay, but did you check that if you revert JDK-8016740, the test actually fails? The original reproducer was stressing the GC a lot more as it tried to cause crashes for 60 seconds. If that is the case, it looks okay to me. Thanks, Thomas From thomas.schatzl at oracle.com Tue Nov 4 09:56:03 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Nov 2014 10:56:03 +0100 Subject: RFR: 8059661: Need test for SoftReference and OOM In-Reply-To: <5440F4B1.2060304@oracle.com> References: <542E871D.4070000@oracle.com> <542E8843.8010301@oracle.com> <1412767715.2690.24.camel@cirrus> <543E8636.4030909@oracle.com> <5440F4B1.2060304@oracle.com> Message-ID: <1415094963.5218.6.camel@oracle.com> Hi Andrey, On Fri, 2014-10-17 at 14:51 +0400, Andrey Zakharov wrote: > Hi, Thomas, Jesper, could you please a review this simple test as GC > experts. > Thanks. > looks okay. Thanks, Thomas From Dmitry.Fazunenko at oracle.com Tue Nov 4 11:32:48 2014 From: Dmitry.Fazunenko at oracle.com (Dmitry Fazunenko) Date: Tue, 04 Nov 2014 14:32:48 +0300 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5458910B.2070100@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> Message-ID: <5458B960.8000305@oracle.com> Nice plan! Please feel free to send me any feedback/questions regarding @requires Thanks, Dima On 04.11.2014 11:40, Bengt Rutisson wrote: > > Hi Dima, > > Thanks for the answers. I think the currently proposed patch is a good > start. We will have to evolve the @requires tag in the future, but > let's have that discussion separate from this review. And we can start > that discussion later when we have more experience with the current > version of @requires. > > Thanks for doing this! > Bengt > > > > On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >> Hi Bengt, >> >> That's great that we have very closed visions! >> >> The general comment: currently, jtreg doesn't support any sort of >> plugins, so you can't provide a VM specific handler of the @requires >> or another tag. This is very annoying limitation and we have to live >> with it. >> >> A few more comments inline. >> >> >> On 03.11.2014 16:31, Bengt Rutisson wrote: >>> >>> >>> Hi Dima, >>> >>> Answers inline. >>> >>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>> Hi Bengt, >>>> >>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>> >>>> See comments inline. >>>> >>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>> >>>>> Hi Evgeniya, >>>>> >>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>> Hi, >>>>>> >>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of >>>>>> the JDK-8019361 >>>>>> >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>> >>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>> another GC. >>>>>> Solution: Such tests marked with the jtreg tag "requires" to skip >>>>>> test if there is a conflict >>>>> >>>>> Thanks for fixing this! It is really great that we finally start >>>>> sorting this out. >>>>> >>>>> First a general comment. The @requires tag has been developed >>>>> without much cooperation with the GC team. We did have a lot of >>>>> feedback when it was first presented a year ago, but it does not >>>>> seem like this feedback was incorporated into the @requires that >>>>> was eventually built. >>>> >>>> We tried to implement as much developer's wishes as possible. But >>>> not everything is possible, sorry for that. >>> >>> Yes, I'm sure you have done your best. It's just that we have been >>> requesting this feature for 3 years and I was expecting us to be >>> able to influence the feature much more than was the case now. >> >> My personal hope: @requires will address ~90% of existing issues. >> >>> >>>> >>>>> >>>>> I think this change that gets proposed now is a big step forward >>>>> and I won't object to it. But I am pretty convinced that we will >>>>> soon run in to the limitations of the current @requires >>>>> implementation and we will have to redo this work. >>>>> >>>>> Some of the points I don't really like about the @requires tag are: >>>>> >>>>> - the "vm.gc" abstraction is more limiting than helping. It would >>>>> have been better to just "require" any command line flag. >>>> "vm.gc" is an alias to a very popular flag. It's also possible to use: >>>> vm.opt.UseG1GC == true instead. >>>> >>>> The table with all vars available in jtreg: >>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>> >>> The problem with having this matching built in to JTreg is that it >>> makes it very hard to change. When we discussed this a year ago I >>> think we said that JTreg should only provide a means to test against >>> the command line and a hook for running some java code in the >>> @requires tag. That way we could put logic like this in a test >>> library that is under our control. This would make it easy for us to >>> change and also enables us to use different logic for different >>> versions. >> >> I would be glad to have own harness... >> >>> >>>> >>>>> - the requirement should be per @run tag. Right now we have to do >>>>> what you did in this change and use vm.gc=null even when some >>>>> tests could actually have been run when a GC was specified. >>>> it would be great, but it will unlikely happen in jtreg, as well as >>>> test case support. >>> >>> what do you mean with test case support? Hi Evgeniya, >> >> Under test case support I mean ability to treat each @run as a >> separate test. Now >> >> @test >> @run -XX:g1RegSize=1m MyTest >> @run -XX:g1RegSize=2m MyTest >> @run -XX:g1RegSize=4m MyTest >> class MyTest { >> } >> >> is always a single test. You can't exclude, or re-run a part of it. >> >> >>> >>>> >>>>> - there are many tests that require more than just a specific GC. >>>>> Often there are other flags that can't be changed either for the >>>>> test to work properly. >>>> >>>> yes. conflicting GC is just the most popular problem caused by >>>> conflicting options. >>>> If we address this issue and we are satisfied with solution, we >>>> could move further. >>> >>> Yes, I agree that taking one step at the time is good. Personally I >>> would have preferred that the first step was a "just run the command >>> line as specified in the @run tag" step. >>> >>>> >>>>> >>>>> Maybe this is not the right place to discuss the current >>>>> implementation of the @requires tag. I just want to say that I'm >>>>> not too happy about how the @requires tag turned out. But assuming >>>>> we have to use it the way it is now I guess the proposed changeset >>>>> looks good. >>>> >>>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>>> And thanks for reviewing it! >>> >>> Agreed. And as I said, I think the patch looks ok. I have not looked >>> at all tests. But if they now pass with the combinations that we >>> test with I guess they should be ok. >> >> Excellent! Thanks a lot! >> >>> >>>> >>>>> >>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>> without any GC flag). Tests are being excluded as expected. No >>>>>> tests failed because of the conflict. >>>>> Have you tested with -Xconcgc too? It's an alias for >>>>> -XX:+UseConcMarkSweepGC. >>>> >>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>> >>> Ok. Thanks. >>> >>>> >>>>> >>>>> I think some of the test, like >>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >>>>> with -XX:+UseParNewGC. Others, like >>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >>>>> with -XX:-UseParNewGC. Could you test these two cases too? >>>> >>>> These two tests ignore vm flags. >>>> Add @requires here is not necessary, but it will allow not execute >>>> the tests when not needed. >>>> So, if we run HS tests with 4 GC, we don't need to run these tests >>>> 4 times, 1 should be enough. >>> >>> Do we really want to use the @requires functionality for this >>> purpose? It seems like a way of misusing @requires. If we just want >>> the tests to be run once I think Leonid's approach with tests lists >>> seems more suitable. >> >> No, it's not a purpose of course, it's just side effect :) >> >> >>> But are you sure that this is the reason for the @requires in this >>> case? TestDefNewCMS does sound like a test that is DefNew specific. >>> I don't see a reason to run it with ParNew. If it doesn't fail today >>> it should probably be changed so that it does fail if it is run with >>> the wrong GC. >> >> @requires - is not the silver bullet, but it's quite easy way to >> solve a lot of issues. >> >> I hope, @requires will allow to reduce the number of "selfish" tests, >> which produce a new java process to ignore vm flags coming from >> outside. No @requires, no other mechanism could 100% protect a test >> from running with conflicting options, but this is not the goal. >> >> If one runs tests with an exotic option, like a new G2 collector, >> there shouldn't mass failures caused by options conflicts. But a few >> failures could be handled manually. >> >> >>> >>>> >>>>> Similarly it looks to me like there are tests that will fail if >>>>> you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>> >>>>> Just a heads up. These two tests will soon be removed. I'm about >>>>> to push a changeset that removes them: >>>>> >>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>> okay, thank for letting us know. >>>> >>>>> >>>>> Is there some way of making sure that all tests are run at one >>>>> time or another. With this change there is a risk that some tests >>>>> are never run and always skipped. Will we somehow be tracking what >>>>> gets skipped and make sure that all tests are at least run once >>>>> with the correct GC so that it is not skipped all the time? >>>> >>>> This is a very good question! >>>> jtreg now doesn't report skipped tests, hopefully it will do soon, >>>> after getting fix of: >>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>> >>>> And yes, tracking tests which are not run is important thing. >>>> @requires - is not the only to exclude test from execution. >>>> >>>> Other examples: >>>> >>>> /* >>>> *@ignore >>>> *@test >>>> */ >>>> ... >>>> >>>> /*@bug 4445555 >>>> *@test >>>> */ >>>> ... >>>> Such tests will never be run, because jtreg treats as test only >>>> files with @test on the first place... >>>> >>>> So, making sure that tests do not disappear is important SQE task, >>>> we know about that, we're thinking on solution (may be very >>>> actively). But this subject for another discussion, not within RFR :) >>> >>> Right. Glad to hear that you are actively working on this! >> >> I was going to say "not very actively", but never mind, we know about >> this problem. With introducing @requires mechanism it will become >> more important! >> >> >> Thanks for your comments! >> >> -- Dima >> >> >>> >>> Bengt >>> >>>> >>>> Thanks, >>>> Dima >>>> >>>> >>>> >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Tue Nov 4 12:09:37 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 04 Nov 2014 13:09:37 +0100 Subject: RFR: 8061467: Bad page size passed to setup_large_pages() on Solaris In-Reply-To: <544FDC0F.2040805@oracle.com> References: <544FDC0F.2040805@oracle.com> Message-ID: <1415102977.5218.8.camel@oracle.com> Hi Erik, On Tue, 2014-10-28 at 19:10 +0100, Erik Helin wrote: > Hi all, > > this small patch fixes a call to Solaris::setup_large_pages that does > not check if the `alignment_hint` parameter is a valid page size (the > only such call left). > > Webrev: > http://cr.openjdk.java.net/~ehelin/8061467/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8061467 > > Testing: > - Verified that the fix solves the issue on a Solaris machine that > supported NUMA. > - JPRT looks good. Could you add a simple regression test like the one suggested in the CR? Thanks, Thomas From kim.barrett at oracle.com Tue Nov 4 16:09:07 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 4 Nov 2014 11:09:07 -0500 Subject: RFR: 8062206: Remove unusable G1RSLogCheckCardTable command line argument In-Reply-To: <1415093390.5218.1.camel@oracle.com> References: <845B3B7C-5A13-47D0-8C57-4B721999E6BC@oracle.com> <1415093390.5218.1.camel@oracle.com> Message-ID: <8FA3C0B9-B4E1-4112-B87D-7120D54469BE@oracle.com> On Nov 4, 2014, at 4:29 AM, Thomas Schatzl wrote: > > Quick grepping showed that also CardTableModRefBS::mod_card_iterate() is > not used any more. Imo it should be removed too. Yes, I know about that. Also CardTableModRefBS::non_clean_card_iterate(). I?m saving those for later, because there?s an important comment attached to mod_card_iterate() that needs to be reworked and moved to some new location that I haven?t decided upon, and might not exist yet. From kim.barrett at oracle.com Tue Nov 4 16:13:17 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 4 Nov 2014 11:13:17 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <545898B5.4080602@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <545898B5.4080602@oracle.com> Message-ID: On Nov 4, 2014, at 4:13 AM, Bengt Rutisson wrote: > > Regarding assignments in if statements; Technically, it?s not an assignment, it?s a declaration. > I thought we had a style where we use booleans for comparisons. This means that I would prefer Jon's suggestion since the comparison will be on a bool value rather than relying on that NULL is interpreted as false. > > The hotspot style guidelines does not cover a lot and I guess it is open for interpretation whether or not this case is covered. But there is a short sentence about booleans: > https://wiki.openjdk.java.net/display/HotSpot/StyleGuide#StyleGuide-Booleans > > I interpret "use true & false (not 1 & 0)" as that NULL should also not be used as a bool value. That?s two out of two reviewers voting against this idiom; I?ll change the code and send an updated webrev later today. From jwha at google.com Tue Nov 4 16:16:29 2014 From: jwha at google.com (Jungwoo Ha) Date: Tue, 04 Nov 2014 16:16:29 +0000 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <5452A080.5090904@oracle.com> <54589A0E.3000400@oracle.com> Message-ID: http://cr.openjdk.java.net/~rasbold/8061259/webrev.04/ Thanks for the review. On Tue Nov 04 2014 at 1:27:44 AM Bengt Rutisson wrote: > > Hi Jungwoo, > > One final nit. > > The two setters don't need a return value. They could be void. > > 1125 bool set_promotion_failed() { _has_promotion_failed = true; } > 1126 bool reset_promotion_failed() { _has_promotion_failed = false; } > > > Bengt > > > On 2014-11-03 18:16, Jungwoo Ha wrote: > > Nice catch! > PTAL > http://cr.openjdk.java.net/~rasbold/8061259/webrev.03/ > > On Thu Oct 30 2014 at 1:33:04 PM Bengt Rutisson > wrote: > >> >> Hi Jungwoo, >> >> >> On 10/30/14 6:24 PM, Jungwoo Ha wrote: >> >> PTAL >> http://cr.openjdk.java.net/~rasbold/8061259/webrev.02/ >> >> >> Thanks! Looks good except for one detail. >> >> 1125 bool set_promotion_failed() { _has_promotion_failed = 1; } >> 1126 bool reset_promotion_failed() { _has_promotion_failed = 0; } >> >> Since _has_promotion_failed is now a bool I don't think we should be >> assigning 1 and 0 to it. We should be using true and false. >> >> Other than that it looks good to me. >> >> Thanks! >> >> Bengt >> >> >> >> >> On Thu Oct 30 2014 at 12:28:19 AM Bengt Rutisson < >> bengt.rutisson at oracle.com> wrote: >> >>> >>> Hi again, >>> >>> One more minor thing. >>> >>> The methods has_promotion_failed(), set_promotion_failed() and >>> reset_promotion_failed() are protected but they could be made private >>> instead. >>> >>> Thanks, >>> Bengt >>> >>> >>> On 2014-10-30 08:09, Bengt Rutisson wrote: >>> >>> >>> >>> Hi Jungwoo, >>> >>> On 2014-10-29 23:51, Jungwoo Ha wrote: >>> >>> PTAL >>> http://cr.openjdk.java.net/~rasbold/8061259/webrev.01/ >>> >>> I've looked a bit at the webrev. A couple of comments: >>>> >>>> Why do you use OrderAccess methods for writing and reading the >>>> _has_promo_failed flag in has_promo_failed() and set_promot_failed() ? >>>> >>> >>> I think that has no effect on x86, but I assumed that processors with >>> weak memory model may want ordering of set/reset/has call. >>> >>> >>> You don't need the OrderAccess methods for the weak memory models here >>> either. You just race on reading the variable and if you see the "wrong" >>> value you eventually take a lock (which will order all memory accesses) to >>> read the variable properly. >>> >>> By removing the use of OrderAccess you can make >>> ConcurrentMarkSweepGeneration::_has_promotion_failed a bool instead of a >>> juint which simplifies the code a bit. >>> >>> >>> Can we write out the full word "promotion" instead of just "promo" in >>>> the variables and methods? >>>> >>> >>> Done. >>> >>> Can we change the name of the flag from UseCMSFastPromotionFailure to >>>> CMSFastPromotionFailure? Most CMS flags start with CMS and I don't think we >>>> need the "Use" prefix. >>>> >>> >>> Done. >>> >>> What do you think about making the flag true by default? At least for >>>> JDK 9. If we decide to backport to JDK 8 or 7 it might be a good idea to >>>> keep the default value as false. >>>> >>> >>> Done. >>> Let me know if there is anything for me to do to backport to JDK8 and 7. >>> >>> >>> I think this fix would be worth backporting to JDK 8. I don't think >>> there is much action required on your side. I created a backport bug for >>> JDK 8 just to make sure that we don't forget it. It will be a little while >>> before the 8 update repos are in a state to accept enhancements again. So, >>> it is nice to have the backport bug to keep track of this. >>> >>> Backporting to JDK 7 requires some more work. Unless you have good >>> arguments for why it is important to backport this to JDK 7 I don't think >>> it is worth doing. >>> >>> >>> Did you find the information provided by _fast_promo_failure_hitcount >>>> useful for debugging? If it not too useful I would consider removing it >>>> since it is cluttering up the code a bit. >>>> >>> >>> I removed it. >>> It was useful to for development, but I think it is no longer needed. >>> >>> >>> Great. Thanks. >>> >>> One more comment. This code comment appears in two places just after we >>> have taken the lock. >>> >>> 3365 if (CMSFastPromotionFailure && has_promotion_failed()) { >>> 3366 // Caller must have checked already without synchronization. >>> 3367 // Check again here while holding the lock. >>> 3368 return NULL; >>> 3369 } >>> >>> There is actually really no requirement that the caller must have >>> checked has_promotion_failed() before calling the method. That's just an >>> optimization. I think the first comment can be skipped and we just leave >>> the second comment "// Check again here while holding the lock.". I would >>> also suggest moving that comment up to the line just before the if >>> statement. >>> >>> Thanks, >>> Bengt >>> >>> >>> >>> >>> >>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.masamitsu at oracle.com Tue Nov 4 17:56:20 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 04 Nov 2014 09:56:20 -0800 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert In-Reply-To: <21592.3488.92171.340837@mykonos.us.oracle.com> References: <21592.3488.92171.340837@mykonos.us.oracle.com> Message-ID: <54591344.4020900@oracle.com> Change looks good. Reviewed. Jon On 11/3/14 3:20 PM, John Coomes wrote: > Hi all, > > Please review this small fix for > > 8060467: CMS: small OldPLABSize and -XX:-ResizePLAB cause > assert(ResizePLAB || n_blks == OldPLABSize) failed: Error > > http://cr.openjdk.java.net/~jcoomes/9/8060467-cms-plab-assert/ > > -John From Dmitry.Fazunenko at oracle.com Tue Nov 4 19:25:28 2014 From: Dmitry.Fazunenko at oracle.com (Dmitry Fazunenko) Date: Tue, 04 Nov 2014 22:25:28 +0300 Subject: PIT results: jdk9/hs for jdk9/dev: NO GO! In-Reply-To: <544FA9E8.5060101@oracle.com> References: <544FA9E8.5060101@oracle.com> Message-ID: <54592828.1030005@oracle.com> Component : VM Status : Go for integration Date : 11/04/2014 at 22:00 MSK Tested By : VM SQE &dmitry.fazunenko at oracle.com Bundles : 2014-10-31-144335.amurillo.jdk9-hs-2014-10-31-snapshot Testing: http://surl.us.oracle.com/pit_jdk9_b38_summary 199 new failures, 4967 known failures, 412056 passed. Issues and Notes: One serious issue has been detected:/https://bugs.openjdk.java.net/browse/JDK-8062735 /It caused 1350+ failures and considered as a stopper./ / NO GO -- Dmitry Fazunenko JVM SQE -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Tue Nov 4 21:18:31 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 4 Nov 2014 16:18:31 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <5457F42F.2070509@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> Message-ID: On Nov 3, 2014, at 4:31 PM, Jon Masamitsu wrote: > > Some suggestions. > > http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/g1/vm_operations_g1.cpp.frames.html > >> 216 if (SurrogateLockerThread* slt = ConcurrentMarkThread::slt()) { >> 217 slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >> 218 } else { >> 219 SurrogateLockerThread::report_missing_slt(); >> 220 } > > I have a preference for > > SurrogateLockerThread* slt = ConcurrentMarkThread::slt(); > if (slt != NULL) { > slt->manipulatePLL(SurrogateLockerThread::acquirePLL); > } else { > SurrogateLockerThread::report_missing_slt(); > } > Done > http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/shared/concurrentGCThread.hpp.frames.html > > 96 // Terminate VM with error message that SLT needed but not yet created. > > I think it would read better as > > "SLT needed but not yet created." => "SLT is needed but has not yet been created." Done Bug: https://bugs.openjdk.java.net/browse/JDK-8062036 Updated webrev: http://cr.openjdk.java.net/~kbarrett/8062036/webrev01/ Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8062036/webrev01-inc/ From John.Coomes at oracle.com Wed Nov 5 00:17:28 2014 From: John.Coomes at oracle.com (John Coomes) Date: Tue, 4 Nov 2014 16:17:28 -0800 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert In-Reply-To: References: <21592.3488.92171.340837@mykonos.us.oracle.com> Message-ID: <21593.27800.476006.256@mykonos.us.oracle.com> Kim Barrett (kim.barrett at oracle.com) wrote: > On Nov 3, 2014, at 6:20 PM, John Coomes wrote: > > > > Please review this small fix for > > > > 8060467: CMS: small OldPLABSize and -XX:-ResizePLAB cause > > assert(ResizePLAB || n_blks == OldPLABSize) failed: Error > > > > http://cr.openjdk.java.net/~jcoomes/9/8060467-cms-plab-assert/ > > Looks good. Thanks! -John From John.Coomes at oracle.com Wed Nov 5 00:18:24 2014 From: John.Coomes at oracle.com (John Coomes) Date: Tue, 4 Nov 2014 16:18:24 -0800 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert In-Reply-To: <1415092202.3101.2.camel@oracle.com> References: <21592.3488.92171.340837@mykonos.us.oracle.com> <1415092202.3101.2.camel@oracle.com> Message-ID: <21593.27856.476087.536448@mykonos.us.oracle.com> Thomas Schatzl (thomas.schatzl at oracle.com) wrote: > Hi, > > On Mon, 2014-11-03 at 15:20 -0800, John Coomes wrote: > > Hi all, > > > > Please review this small fix for > > > > 8060467: CMS: small OldPLABSize and -XX:-ResizePLAB cause > > assert(ResizePLAB || n_blks == OldPLABSize) failed: Error > > > > http://cr.openjdk.java.net/~jcoomes/9/8060467-cms-plab-assert/ > > looks good. Could you please add a regression test? Thanks. An updated webrev that adds a test is at the same location. -John From John.Coomes at oracle.com Wed Nov 5 00:19:04 2014 From: John.Coomes at oracle.com (John Coomes) Date: Tue, 4 Nov 2014 16:19:04 -0800 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert In-Reply-To: <54591344.4020900@oracle.com> References: <21592.3488.92171.340837@mykonos.us.oracle.com> <54591344.4020900@oracle.com> Message-ID: <21593.27896.923751.792781@mykonos.us.oracle.com> Jon Masamitsu (jon.masamitsu at oracle.com) wrote: > Change looks good. > > Reviewed. Thanks for the review. -John > On 11/3/14 3:20 PM, John Coomes wrote: > > Hi all, > > > > Please review this small fix for > > > > 8060467: CMS: small OldPLABSize and -XX:-ResizePLAB cause > > assert(ResizePLAB || n_blks == OldPLABSize) failed: Error > > > > http://cr.openjdk.java.net/~jcoomes/9/8060467-cms-plab-assert/ > > > > -John > From kim.barrett at oracle.com Wed Nov 5 07:15:25 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Nov 2014 02:15:25 -0500 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert In-Reply-To: <21593.27856.476087.536448@mykonos.us.oracle.com> References: <21592.3488.92171.340837@mykonos.us.oracle.com> <1415092202.3101.2.camel@oracle.com> <21593.27856.476087.536448@mykonos.us.oracle.com> Message-ID: <6DA971F7-DF17-4720-BF47-51D3007FA24A@oracle.com> On Nov 4, 2014, at 7:18 PM, John Coomes wrote: > > Thanks. An updated webrev that adds a test is at the same location. Test looks ok too. From thomas.schatzl at oracle.com Wed Nov 5 08:51:08 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Nov 2014 09:51:08 +0100 Subject: review request (XXS) - 8060467: small OldPLABSize causes assert In-Reply-To: <21593.27856.476087.536448@mykonos.us.oracle.com> References: <21592.3488.92171.340837@mykonos.us.oracle.com> <1415092202.3101.2.camel@oracle.com> <21593.27856.476087.536448@mykonos.us.oracle.com> Message-ID: <1415177468.2912.2.camel@oracle.com> Hi, On Tue, 2014-11-04 at 16:18 -0800, John Coomes wrote: > Thomas Schatzl (thomas.schatzl at oracle.com) wrote: > > Hi, > > > > On Mon, 2014-11-03 at 15:20 -0800, John Coomes wrote: > > > Hi all, > > > > > > Please review this small fix for > > > > > > 8060467: CMS: small OldPLABSize and -XX:-ResizePLAB cause > > > assert(ResizePLAB || n_blks == OldPLABSize) failed: Error > > > > > > http://cr.openjdk.java.net/~jcoomes/9/8060467-cms-plab-assert/ > > > > looks good. Could you please add a regression test? > > Thanks. An updated webrev that adds a test is at the same location. looks good. Thanks. Thomas From bengt.rutisson at oracle.com Wed Nov 5 08:59:45 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 05 Nov 2014 09:59:45 +0100 Subject: RFR (XS): JDK-8061449: G1: FreeRegionList_test() fails with G1 after the JDK-8058534 fix to HeapRegion::orig_end() Message-ID: <5459E701.7080203@oracle.com> Hi all, Can I have a couple for reviews of a fix for this internal VM test? http://cr.openjdk.java.net/~brutisso/8061449/webrev.00/ Bug link: https://bugs.openjdk.java.net/browse/JDK-8061449 Background: When this fix was pushed: Remove HeapRegion::_orig_end https://bugs.openjdk.java.net/browse/JDK-8058534 The heap region constructor actually examines the MemRegion that is passed to it. In the internal VM tests there was a test, FreeRegionList_test(), that passed a random MemRegion to the heap region constructor. This worked before, but now stopped working. The original fix, JDK-8058534, is correct so the fix for the failing test is to set up proper memory regions and pass those to the heap region constructors. Before my suggested patch this fails: $ java -XX:+ExecuteInternalVMTests -XX:+UseG1GC -version The test fails on this assert in HeapRegion::hr_clear(): assert(_end == orig_end(), With my suggested patch all tests pass with the above command line. Thanks, Bengt From mikael.gerdin at oracle.com Wed Nov 5 09:17:38 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 05 Nov 2014 10:17:38 +0100 Subject: RFR (XS): JDK-8061449: G1: FreeRegionList_test() fails with G1 after the JDK-8058534 fix to HeapRegion::orig_end() In-Reply-To: <5459E701.7080203@oracle.com> References: <5459E701.7080203@oracle.com> Message-ID: <5459EB32.1090406@oracle.com> Bengt, On 2014-11-05 09:59, Bengt Rutisson wrote: > > Hi all, > > Can I have a couple for reviews of a fix for this internal VM test? > > http://cr.openjdk.java.net/~brutisso/8061449/webrev.00/ Looks good, thanks for fixing this. /Mikael > > Bug link: > https://bugs.openjdk.java.net/browse/JDK-8061449 > > Background: > > When this fix was pushed: > > Remove HeapRegion::_orig_end > https://bugs.openjdk.java.net/browse/JDK-8058534 > > The heap region constructor actually examines the MemRegion that is > passed to it. In the internal VM tests there was a test, > FreeRegionList_test(), that passed a random MemRegion to the heap region > constructor. This worked before, but now stopped working. The original > fix, JDK-8058534, is correct so the fix for the failing test is to set > up proper memory regions and pass those to the heap region constructors. > > Before my suggested patch this fails: > > $ java -XX:+ExecuteInternalVMTests -XX:+UseG1GC -version > > The test fails on this assert in HeapRegion::hr_clear(): > > assert(_end == orig_end(), > > With my suggested patch all tests pass with the above command line. > > Thanks, > Bengt From thomas.schatzl at oracle.com Wed Nov 5 09:18:38 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Nov 2014 10:18:38 +0100 Subject: RFR (XS): JDK-8061449: G1: FreeRegionList_test() fails with G1 after the JDK-8058534 fix to HeapRegion::orig_end() In-Reply-To: <5459E701.7080203@oracle.com> References: <5459E701.7080203@oracle.com> Message-ID: <1415179118.2912.4.camel@oracle.com> Hi, On Wed, 2014-11-05 at 09:59 +0100, Bengt Rutisson wrote: > Hi all, > > Can I have a couple for reviews of a fix for this internal VM test? > > http://cr.openjdk.java.net/~brutisso/8061449/webrev.00/ > > Bug link: > https://bugs.openjdk.java.net/browse/JDK-8061449 Looks good. Thanks, Thomas From bengt.rutisson at oracle.com Wed Nov 5 09:10:13 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 05 Nov 2014 10:10:13 +0100 Subject: RFR (XS): JDK-8061449: G1: FreeRegionList_test() fails with G1 after the JDK-8058534 fix to HeapRegion::orig_end() In-Reply-To: <5459E701.7080203@oracle.com> References: <5459E701.7080203@oracle.com> Message-ID: <5459E975.4030506@oracle.com> Hi again, StefanK noticed that I my patch created the same MemRegion for all heap regions. This still made the test pass, but was not my intention. Here's an updated webrev: http://cr.openjdk.java.net/~brutisso/8061449/webrev.01/ Bengt On 2014-11-05 09:59, Bengt Rutisson wrote: > > Hi all, > > Can I have a couple for reviews of a fix for this internal VM test? > > http://cr.openjdk.java.net/~brutisso/8061449/webrev.00/ > > Bug link: > https://bugs.openjdk.java.net/browse/JDK-8061449 > > Background: > > When this fix was pushed: > > Remove HeapRegion::_orig_end > https://bugs.openjdk.java.net/browse/JDK-8058534 > > The heap region constructor actually examines the MemRegion that is > passed to it. In the internal VM tests there was a test, > FreeRegionList_test(), that passed a random MemRegion to the heap > region constructor. This worked before, but now stopped working. The > original fix, JDK-8058534, is correct so the fix for the failing test > is to set up proper memory regions and pass those to the heap region > constructors. > > Before my suggested patch this fails: > > $ java -XX:+ExecuteInternalVMTests -XX:+UseG1GC -version > > The test fails on this assert in HeapRegion::hr_clear(): > > assert(_end == orig_end(), > > With my suggested patch all tests pass with the above command line. > > Thanks, > Bengt From stefan.karlsson at oracle.com Wed Nov 5 09:09:58 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 05 Nov 2014 10:09:58 +0100 Subject: RFR (XS): JDK-8061449: G1: FreeRegionList_test() fails with G1 after the JDK-8058534 fix to HeapRegion::orig_end() In-Reply-To: <5459E975.4030506@oracle.com> References: <5459E701.7080203@oracle.com> <5459E975.4030506@oracle.com> Message-ID: <5459E966.7010204@oracle.com> On 2014-11-05 10:10, Bengt Rutisson wrote: > > Hi again, > > StefanK noticed that I my patch created the same MemRegion for all > heap regions. This still made the test pass, but was not my intention. > Here's an updated webrev: > > http://cr.openjdk.java.net/~brutisso/8061449/webrev.01/ Looks good. StefanK > > Bengt > > > On 2014-11-05 09:59, Bengt Rutisson wrote: >> >> Hi all, >> >> Can I have a couple for reviews of a fix for this internal VM test? >> >> http://cr.openjdk.java.net/~brutisso/8061449/webrev.00/ >> >> Bug link: >> https://bugs.openjdk.java.net/browse/JDK-8061449 >> >> Background: >> >> When this fix was pushed: >> >> Remove HeapRegion::_orig_end >> https://bugs.openjdk.java.net/browse/JDK-8058534 >> >> The heap region constructor actually examines the MemRegion that is >> passed to it. In the internal VM tests there was a test, >> FreeRegionList_test(), that passed a random MemRegion to the heap >> region constructor. This worked before, but now stopped working. The >> original fix, JDK-8058534, is correct so the fix for the failing test >> is to set up proper memory regions and pass those to the heap region >> constructors. >> >> Before my suggested patch this fails: >> >> $ java -XX:+ExecuteInternalVMTests -XX:+UseG1GC -version >> >> The test fails on this assert in HeapRegion::hr_clear(): >> >> assert(_end == orig_end(), >> >> With my suggested patch all tests pass with the above command line. >> >> Thanks, >> Bengt > From bengt.rutisson at oracle.com Wed Nov 5 09:14:49 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 05 Nov 2014 10:14:49 +0100 Subject: RFR (XS): JDK-8061449: G1: FreeRegionList_test() fails with G1 after the JDK-8058534 fix to HeapRegion::orig_end() In-Reply-To: <5459E966.7010204@oracle.com> References: <5459E701.7080203@oracle.com> <5459E975.4030506@oracle.com> <5459E966.7010204@oracle.com> Message-ID: <5459EA89.3000103@oracle.com> Thanks Mikael, Thomas and StefanK for the quick reviews! Bengt On 2014-11-05 10:09, Stefan Karlsson wrote: > > On 2014-11-05 10:10, Bengt Rutisson wrote: >> >> Hi again, >> >> StefanK noticed that I my patch created the same MemRegion for all >> heap regions. This still made the test pass, but was not my >> intention. Here's an updated webrev: >> >> http://cr.openjdk.java.net/~brutisso/8061449/webrev.01/ > > Looks good. > > StefanK > >> >> Bengt >> >> >> On 2014-11-05 09:59, Bengt Rutisson wrote: >>> >>> Hi all, >>> >>> Can I have a couple for reviews of a fix for this internal VM test? >>> >>> http://cr.openjdk.java.net/~brutisso/8061449/webrev.00/ >>> >>> Bug link: >>> https://bugs.openjdk.java.net/browse/JDK-8061449 >>> >>> Background: >>> >>> When this fix was pushed: >>> >>> Remove HeapRegion::_orig_end >>> https://bugs.openjdk.java.net/browse/JDK-8058534 >>> >>> The heap region constructor actually examines the MemRegion that is >>> passed to it. In the internal VM tests there was a test, >>> FreeRegionList_test(), that passed a random MemRegion to the heap >>> region constructor. This worked before, but now stopped working. The >>> original fix, JDK-8058534, is correct so the fix for the failing >>> test is to set up proper memory regions and pass those to the heap >>> region constructors. >>> >>> Before my suggested patch this fails: >>> >>> $ java -XX:+ExecuteInternalVMTests -XX:+UseG1GC -version >>> >>> The test fails on this assert in HeapRegion::hr_clear(): >>> >>> assert(_end == orig_end(), >>> >>> With my suggested patch all tests pass with the above command line. >>> >>> Thanks, >>> Bengt >> > From thomas.schatzl at oracle.com Wed Nov 5 09:24:18 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Nov 2014 10:24:18 +0100 Subject: RFR (XS): JDK-8061449: G1: FreeRegionList_test() fails with G1 after the JDK-8058534 fix to HeapRegion::orig_end() In-Reply-To: <5459E975.4030506@oracle.com> References: <5459E701.7080203@oracle.com> <5459E975.4030506@oracle.com> Message-ID: <1415179458.2912.6.camel@oracle.com> Hi Bengt, On Wed, 2014-11-05 at 10:10 +0100, Bengt Rutisson wrote: > Hi again, > > StefanK noticed that I my patch created the same MemRegion for all heap > regions. This still made the test pass, but was not my intention. Here's > an updated webrev: Noticed that as well, but it does not matter for the test. It is nicer to fix that though to avoid confusion. > http://cr.openjdk.java.net/~brutisso/8061449/webrev.01/ Still okay. Thanks, Thomas From bengt.rutisson at oracle.com Wed Nov 5 09:17:14 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 05 Nov 2014 10:17:14 +0100 Subject: RFR (XS): JDK-8061449: G1: FreeRegionList_test() fails with G1 after the JDK-8058534 fix to HeapRegion::orig_end() In-Reply-To: <1415179458.2912.6.camel@oracle.com> References: <5459E701.7080203@oracle.com> <5459E975.4030506@oracle.com> <1415179458.2912.6.camel@oracle.com> Message-ID: <5459EB1A.1030009@oracle.com> On 2014-11-05 10:24, Thomas Schatzl wrote: > Hi Bengt, > > On Wed, 2014-11-05 at 10:10 +0100, Bengt Rutisson wrote: >> Hi again, >> >> StefanK noticed that I my patch created the same MemRegion for all heap >> regions. This still made the test pass, but was not my intention. Here's >> an updated webrev: > Noticed that as well, but it does not matter for the test. It is nicer > to fix that though to avoid confusion. Agreed. :) > >> http://cr.openjdk.java.net/~brutisso/8061449/webrev.01/ > Still okay. Thanks! Bengt > > Thanks, > Thomas > > From bengt.rutisson at oracle.com Wed Nov 5 09:41:24 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 05 Nov 2014 10:41:24 +0100 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> Message-ID: <5459F0C4.2040208@oracle.com> Hi Kim, Latest webrev (8062036/webrev01/) looks good to me. Bengt On 2014-11-04 22:18, Kim Barrett wrote: > On Nov 3, 2014, at 4:31 PM, Jon Masamitsu wrote: >> Some suggestions. >> >> http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/g1/vm_operations_g1.cpp.frames.html >> >>> 216 if (SurrogateLockerThread* slt = ConcurrentMarkThread::slt()) { >>> 217 slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >>> 218 } else { >>> 219 SurrogateLockerThread::report_missing_slt(); >>> 220 } >> I have a preference for >> >> SurrogateLockerThread* slt = ConcurrentMarkThread::slt(); >> if (slt != NULL) { >> slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >> } else { >> SurrogateLockerThread::report_missing_slt(); >> } >> > Done > >> http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/shared/concurrentGCThread.hpp.frames.html >> >> 96 // Terminate VM with error message that SLT needed but not yet created. >> >> I think it would read better as >> >> "SLT needed but not yet created." => "SLT is needed but has not yet been created." > Done > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062036 > > Updated webrev: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev01/ > > Incremental webrev: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev01-inc/ > From bengt.rutisson at oracle.com Wed Nov 5 09:45:07 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 05 Nov 2014 10:45:07 +0100 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <5452A080.5090904@oracle.com> <54589A0E.3000400@oracle.com> Message-ID: <5459F1A3.2010402@oracle.com> On 2014-11-04 17:16, Jungwoo Ha wrote: > http://cr.openjdk.java.net/~rasbold/8061259/webrev.04/ > Looks good. I can sponsor this change. We need one more reviewer to look at it before I can push. Bengt > Thanks for the review. > On Tue Nov 04 2014 at 1:27:44 AM Bengt Rutisson > > wrote: > > > Hi Jungwoo, > > One final nit. > > The two setters don't need a return value. They could be void. > > 1125 bool set_promotion_failed() { _has_promotion_failed = true; } > 1126 bool reset_promotion_failed() { _has_promotion_failed = > false; } > > > Bengt > > > On 2014-11-03 18:16, Jungwoo Ha wrote: >> Nice catch! >> PTAL >> http://cr.openjdk.java.net/~rasbold/8061259/webrev.03/ >> >> >> On Thu Oct 30 2014 at 1:33:04 PM Bengt Rutisson >> > wrote: >> >> >> Hi Jungwoo, >> >> >> On 10/30/14 6:24 PM, Jungwoo Ha wrote: >>> PTAL >>> http://cr.openjdk.java.net/~rasbold/8061259/webrev.02/ >>> >> >> Thanks! Looks good except for one detail. >> >> 1125 bool set_promotion_failed() { _has_promotion_failed >> = 1; } >> 1126 bool reset_promotion_failed() { _has_promotion_failed >> = 0; } >> >> Since _has_promotion_failed is now a bool I don't think we >> should be assigning 1 and 0 to it. We should be using true >> and false. >> >> Other than that it looks good to me. >> >> Thanks! >> >> Bengt >> >> >>> >>> >>> On Thu Oct 30 2014 at 12:28:19 AM Bengt Rutisson >>> >> > wrote: >>> >>> >>> Hi again, >>> >>> One more minor thing. >>> >>> The methods has_promotion_failed(), >>> set_promotion_failed() and reset_promotion_failed() are >>> protected but they could be made private instead. >>> >>> Thanks, >>> Bengt >>> >>> >>> On 2014-10-30 08:09, Bengt Rutisson wrote: >>>> >>>> >>>> Hi Jungwoo, >>>> >>>> On 2014-10-29 23:51, Jungwoo Ha wrote: >>>>> PTAL >>>>> http://cr.openjdk.java.net/~rasbold/8061259/webrev.01/ >>>>> >>>>> >>>>> I've looked a bit at the webrev. A couple of comments: >>>>> >>>>> Why do you use OrderAccess methods for writing and >>>>> reading the _has_promo_failed flag in >>>>> has_promo_failed() and set_promot_failed() ? >>>>> >>>>> >>>>> I think that has no effect on x86, but I assumed that >>>>> processors with weak memory model may want ordering of >>>>> set/reset/has call. >>>> >>>> You don't need the OrderAccess methods for the weak >>>> memory models here either. You just race on reading the >>>> variable and if you see the "wrong" value you >>>> eventually take a lock (which will order all memory >>>> accesses) to read the variable properly. >>>> >>>> By removing the use of OrderAccess you can make >>>> ConcurrentMarkSweepGeneration::_has_promotion_failed a >>>> bool instead of a juint which simplifies the code a bit. >>>> >>>>> >>>>> Can we write out the full word "promotion" instead >>>>> of just "promo" in the variables and methods? >>>>> >>>>> >>>>> Done. >>>>> >>>>> Can we change the name of the flag from >>>>> UseCMSFastPromotionFailure to >>>>> CMSFastPromotionFailure? Most CMS flags start with >>>>> CMS and I don't think we need the "Use" prefix. >>>>> >>>>> >>>>> Done. >>>>> >>>>> What do you think about making the flag true by >>>>> default? At least for JDK 9. If we decide to >>>>> backport to JDK 8 or 7 it might be a good idea to >>>>> keep the default value as false. >>>>> >>>>> >>>>> Done. >>>>> Let me know if there is anything for me to do to >>>>> backport to JDK8 and 7. >>>> >>>> I think this fix would be worth backporting to JDK 8. I >>>> don't think there is much action required on your side. >>>> I created a backport bug for JDK 8 just to make sure >>>> that we don't forget it. It will be a little while >>>> before the 8 update repos are in a state to accept >>>> enhancements again. So, it is nice to have the backport >>>> bug to keep track of this. >>>> >>>> Backporting to JDK 7 requires some more work. Unless >>>> you have good arguments for why it is important to >>>> backport this to JDK 7 I don't think it is worth doing. >>>> >>>>> >>>>> Did you find the information provided by >>>>> _fast_promo_failure_hitcount useful for debugging? >>>>> If it not too useful I would consider removing it >>>>> since it is cluttering up the code a bit. >>>>> >>>>> >>>>> I removed it. >>>>> It was useful to for development, but I think it is no >>>>> longer needed. >>>> >>>> Great. Thanks. >>>> >>>> One more comment. This code comment appears in two >>>> places just after we have taken the lock. >>>> >>>> 3365 if (CMSFastPromotionFailure && >>>> has_promotion_failed()) { >>>> 3366 // Caller must have checked already without >>>> synchronization. >>>> 3367 // Check again here while holding the lock. >>>> 3368 return NULL; >>>> 3369 } >>>> >>>> There is actually really no requirement that the caller >>>> must have checked has_promotion_failed() before calling >>>> the method. That's just an optimization. I think the >>>> first comment can be skipped and we just leave the >>>> second comment "// Check again here while holding the >>>> lock.". I would also suggest moving that comment up to >>>> the line just before the if statement. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> >>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Wed Nov 5 15:14:25 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Nov 2014 16:14:25 +0100 Subject: RFR: 8061467: Bad page size passed to setup_large_pages() on Solaris In-Reply-To: <1415102977.5218.8.camel@oracle.com> References: <544FDC0F.2040805@oracle.com> <1415102977.5218.8.camel@oracle.com> Message-ID: <1415200465.2912.11.camel@oracle.com> Hi all, since Erik is on vacation and this bug prevents a lot of tests from running, I took over and added a (simple) regression test case. Bug: https://bugs.openjdk.java.net/browse/JDK-8061467 Webrev: http://cr.openjdk.java.net/~tschatzl/8061467/webrev/ Testing: jprt, Solaris numa machine showing the crash before, and no crash after the change, local testing, jprt test case runs Since the test is extremely simple (class with empty main method), I am going to keep Erik as the original author of this change as his change is the one that required the hard work. Thanks, Thomas On Tue, 2014-11-04 at 13:09 +0100, Thomas Schatzl wrote: > Hi Erik, > > On Tue, 2014-10-28 at 19:10 +0100, Erik Helin wrote: > > Hi all, > > > > this small patch fixes a call to Solaris::setup_large_pages that does > > not check if the `alignment_hint` parameter is a valid page size (the > > only such call left). > > > > Webrev: > > http://cr.openjdk.java.net/~ehelin/8061467/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8061467 > > > > Testing: > > - Verified that the fix solves the issue on a Solaris machine that > > supported NUMA. > > - JPRT > > looks good. Could you add a simple regression test like the one > suggested in the CR? > > Thanks, > Thomas > > From jesper.wilhelmsson at oracle.com Wed Nov 5 15:18:39 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 05 Nov 2014 16:18:39 +0100 Subject: RFR: 8062836 - BACKOUT - Parallelize clearing the next mark bitmap Message-ID: <545A3FCF.7000106@oracle.com> Hi, The fix for JDK-8062836 - Parallelize clearing the next mark bitmap caused problems in the nightly testing and the fix was found to be non-trivial. Therefore the change is backed out. Bug: https://bugs.openjdk.java.net/browse/JDK-8062836 Webrev: http://cr.openjdk.java.net/~jwilhelm/8062836/webrev.00/ Thanks! /Jesper -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikael.gerdin at oracle.com Wed Nov 5 15:19:38 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 05 Nov 2014 16:19:38 +0100 Subject: RFR: 8061467: Bad page size passed to setup_large_pages() on Solaris In-Reply-To: <1415200465.2912.11.camel@oracle.com> References: <544FDC0F.2040805@oracle.com> <1415102977.5218.8.camel@oracle.com> <1415200465.2912.11.camel@oracle.com> Message-ID: <545A400A.1010003@oracle.com> Hi Thomas, On 2014-11-05 16:14, Thomas Schatzl wrote: > Hi all, > > since Erik is on vacation and this bug prevents a lot of tests from > running, I took over and added a (simple) regression test case. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8061467 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8061467/webrev/ Looks good to me. /Mikael > > Testing: > jprt, Solaris numa machine showing the crash before, and no crash after > the change, local testing, jprt test case runs > > Since the test is extremely simple (class with empty main method), I am > going to keep Erik as the original author of this change as his change > is the one that required the hard work. > > Thanks, > Thomas > > On Tue, 2014-11-04 at 13:09 +0100, Thomas Schatzl wrote: >> Hi Erik, >> >> On Tue, 2014-10-28 at 19:10 +0100, Erik Helin wrote: >>> Hi all, >>> >>> this small patch fixes a call to Solaris::setup_large_pages that does >>> not check if the `alignment_hint` parameter is a valid page size (the >>> only such call left). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8061467/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8061467 >>> >>> Testing: >>> - Verified that the fix solves the issue on a Solaris machine that >>> supported NUMA. >>> - JPRT >> >> looks good. Could you add a simple regression test like the one >> suggested in the CR? >> >> Thanks, >> Thomas >> >> > > From mikael.gerdin at oracle.com Wed Nov 5 15:20:39 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 05 Nov 2014 16:20:39 +0100 Subject: RFR: 8062836 - BACKOUT - Parallelize clearing the next mark bitmap In-Reply-To: <545A3FCF.7000106@oracle.com> References: <545A3FCF.7000106@oracle.com> Message-ID: <545A4047.50104@oracle.com> Hi Jesper, On 2014-11-05 16:18, Jesper Wilhelmsson wrote: > Hi, > > The fix for JDK-8062836 - Parallelize clearing the next mark bitmap > caused problems in the nightly testing and the fix was found to be > non-trivial. Therefore the change is backed out. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062836 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8062836/webrev.00/ > The backout looks correct. /Mikael > Thanks! > /Jesper > From marcus.larsson at oracle.com Wed Nov 5 15:23:35 2014 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 05 Nov 2014 16:23:35 +0100 Subject: RFR: 8062836 - BACKOUT - Parallelize clearing the next mark bitmap In-Reply-To: <545A3FCF.7000106@oracle.com> References: <545A3FCF.7000106@oracle.com> Message-ID: <545A40F7.1000505@oracle.com> Hi Jesper, On 05/11/14 16:18, Jesper Wilhelmsson wrote: > Hi, > > The fix for JDK-8062836 - Parallelize clearing the next mark bitmap > caused problems in the nightly testing and the fix was found to be > non-trivial. Therefore the change is backed out. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062836 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8062836/webrev.00/ Looks good. Marcus > > Thanks! > /Jesper > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Wed Nov 5 15:24:37 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Nov 2014 16:24:37 +0100 Subject: RFR: 8061467: Bad page size passed to setup_large_pages() on Solaris In-Reply-To: <545A400A.1010003@oracle.com> References: <544FDC0F.2040805@oracle.com> <1415102977.5218.8.camel@oracle.com> <1415200465.2912.11.camel@oracle.com> <545A400A.1010003@oracle.com> Message-ID: <1415201077.2912.13.camel@oracle.com> Hi, On Wed, 2014-11-05 at 16:19 +0100, Mikael Gerdin wrote: > Hi Thomas, > > On 2014-11-05 16:14, Thomas Schatzl wrote: > > Hi all, > > > > since Erik is on vacation and this bug prevents a lot of tests from > > running, I took over and added a (simple) regression test case. > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8061467 > > > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8061467/webrev/ > > Looks good to me. Thanks for the review Mikael. Thomas From evgeniya.stepanova at oracle.com Wed Nov 5 15:33:59 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 05 Nov 2014 19:33:59 +0400 Subject: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests Message-ID: <545A4367.1070407@oracle.com> Hi, Please review changes for 8062536, the OpenJDK/jdk part of the JDK-8019361 bug: https://bugs.openjdk.java.net/browse/JDK-8062536 fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ Problem: Some tests explicitly set GC and fail when another GC is set outside Solution: Such tests marked with the jtreg tag "requires" to skip test if there is a conflict Tested locally with different GC flags (-XX:+UseG1GC, -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and without any GC flag). Tests are being excluded as expected. No tests failed because of the conflict. Thanks, Evgeniya Stepanova // -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Wed Nov 5 16:19:05 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Nov 2014 11:19:05 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <5459F0C4.2040208@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <5459F0C4.2040208@oracle.com> Message-ID: <17C9EE9F-4543-4ABB-9B8B-9E608CA62B73@oracle.com> On Nov 5, 2014, at 4:41 AM, Bengt Rutisson wrote: > > Latest webrev (8062036/webrev01/) looks good to me. Thanks for the review. From thomas.schatzl at oracle.com Wed Nov 5 16:24:23 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 05 Nov 2014 17:24:23 +0100 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> Message-ID: <1415204663.2912.18.camel@oracle.com> Hi Kim, On Tue, 2014-11-04 at 16:18 -0500, Kim Barrett wrote: > On Nov 3, 2014, at 4:31 PM, Jon Masamitsu wrote: > > > > Some suggestions. > > [...] > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062036 > > Updated webrev: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev01/ > > Incremental webrev: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev01-inc/ > Looks good. Could you add a regression test case? Maybe one that tries to force the error message (i.e. checks it), and one that just runs the given example command line. Thanks, Thomas From kim.barrett at oracle.com Wed Nov 5 18:07:59 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Nov 2014 13:07:59 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <1415204663.2912.18.camel@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> Message-ID: <2BE17EDD-4144-4DBC-82AE-591F608EE13F@oracle.com> On Nov 5, 2014, at 11:24 AM, Thomas Schatzl wrote: > > Looks good. Could you add a regression test case? > > Maybe one that tries to force the error message (i.e. checks it), That requires causing a GC (that needs the SLT) during VM initialization. I?m not sure how one would write such a test. It certainly involves producing the desired effect using only command line arguments, since by the time any application Java code is run the initialization is necessarily complete. And I?d like to avoid introducing a command line option to disable the init-time suppression of ScavengeALot so that we can test that error message. > and > one that just runs the given example command line. That I can do, and I?ll verify it fails without the changes, and produces the desired error message with just the changes for that. From jon.masamitsu at oracle.com Wed Nov 5 22:11:43 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 05 Nov 2014 14:11:43 -0800 Subject: RFR: 8061467: Bad page size passed to setup_large_pages() on Solaris In-Reply-To: <1415200465.2912.11.camel@oracle.com> References: <544FDC0F.2040805@oracle.com> <1415102977.5218.8.camel@oracle.com> <1415200465.2912.11.camel@oracle.com> Message-ID: <545AA09F.4040504@oracle.com> Thomas, Change look good but in the test, did you consider adding -XX:+UseLargePages to the command line in the test? On Solaris large pages are on by default but not so with all of the platforms. Jon On 11/5/2014 7:14 AM, Thomas Schatzl wrote: > Hi all, > > since Erik is on vacation and this bug prevents a lot of tests from > running, I took over and added a (simple) regression test case. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8061467 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8061467/webrev/ > > Testing: > jprt, Solaris numa machine showing the crash before, and no crash after > the change, local testing, jprt test case runs > > Since the test is extremely simple (class with empty main method), I am > going to keep Erik as the original author of this change as his change > is the one that required the hard work. > > Thanks, > Thomas > > On Tue, 2014-11-04 at 13:09 +0100, Thomas Schatzl wrote: >> Hi Erik, >> >> On Tue, 2014-10-28 at 19:10 +0100, Erik Helin wrote: >>> Hi all, >>> >>> this small patch fixes a call to Solaris::setup_large_pages that does >>> not check if the `alignment_hint` parameter is a valid page size (the >>> only such call left). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8061467/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8061467 >>> >>> Testing: >>> - Verified that the fix solves the issue on a Solaris machine that >>> supported NUMA. >>> - JPRT >> looks good. Could you add a simple regression test like the one >> suggested in the CR? >> >> Thanks, >> Thomas >> >> > From jon.masamitsu at oracle.com Wed Nov 5 22:16:44 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 05 Nov 2014 14:16:44 -0800 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> Message-ID: <545AA1CC.1030100@oracle.com> Kim, Thanks for the changes. They look good. Reviewed (pending finishing your discussion with Thomas). Jon On 11/4/2014 1:18 PM, Kim Barrett wrote: > On Nov 3, 2014, at 4:31 PM, Jon Masamitsu wrote: >> Some suggestions. >> >> http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/g1/vm_operations_g1.cpp.frames.html >> >>> 216 if (SurrogateLockerThread* slt = ConcurrentMarkThread::slt()) { >>> 217 slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >>> 218 } else { >>> 219 SurrogateLockerThread::report_missing_slt(); >>> 220 } >> I have a preference for >> >> SurrogateLockerThread* slt = ConcurrentMarkThread::slt(); >> if (slt != NULL) { >> slt->manipulatePLL(SurrogateLockerThread::acquirePLL); >> } else { >> SurrogateLockerThread::report_missing_slt(); >> } >> > Done > >> http://cr.openjdk.java.net/~kbarrett/8062036/webrev/src/share/vm/gc_implementation/shared/concurrentGCThread.hpp.frames.html >> >> 96 // Terminate VM with error message that SLT needed but not yet created. >> >> I think it would read better as >> >> "SLT needed but not yet created." => "SLT is needed but has not yet been created." > Done > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062036 > > Updated webrev: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev01/ > > Incremental webrev: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev01-inc/ > From jon.masamitsu at oracle.com Wed Nov 5 22:33:14 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 05 Nov 2014 14:33:14 -0800 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <5452A080.5090904@oracle.com> <54589A0E.3000400@oracle.com> Message-ID: <545AA5AA.1000402@oracle.com> Changes look good. Reviewed. Thanks for the fix. Jon On 11/4/2014 8:16 AM, Jungwoo Ha wrote: > http://cr.openjdk.java.net/~rasbold/8061259/webrev.04/ > > Thanks for the review. > On Tue Nov 04 2014 at 1:27:44 AM Bengt Rutisson > > wrote: > > > Hi Jungwoo, > > One final nit. > > The two setters don't need a return value. They could be void. > > 1125 bool set_promotion_failed() { _has_promotion_failed = true; } > 1126 bool reset_promotion_failed() { _has_promotion_failed = > false; } > > > Bengt > > > On 2014-11-03 18:16, Jungwoo Ha wrote: >> Nice catch! >> PTAL >> http://cr.openjdk.java.net/~rasbold/8061259/webrev.03/ >> >> >> On Thu Oct 30 2014 at 1:33:04 PM Bengt Rutisson >> > wrote: >> >> >> Hi Jungwoo, >> >> >> On 10/30/14 6:24 PM, Jungwoo Ha wrote: >>> PTAL >>> http://cr.openjdk.java.net/~rasbold/8061259/webrev.02/ >>> >> >> Thanks! Looks good except for one detail. >> >> 1125 bool set_promotion_failed() { _has_promotion_failed >> = 1; } >> 1126 bool reset_promotion_failed() { _has_promotion_failed >> = 0; } >> >> Since _has_promotion_failed is now a bool I don't think we >> should be assigning 1 and 0 to it. We should be using true >> and false. >> >> Other than that it looks good to me. >> >> Thanks! >> >> Bengt >> >> >>> >>> >>> On Thu Oct 30 2014 at 12:28:19 AM Bengt Rutisson >>> >> > wrote: >>> >>> >>> Hi again, >>> >>> One more minor thing. >>> >>> The methods has_promotion_failed(), >>> set_promotion_failed() and reset_promotion_failed() are >>> protected but they could be made private instead. >>> >>> Thanks, >>> Bengt >>> >>> >>> On 2014-10-30 08:09, Bengt Rutisson wrote: >>>> >>>> >>>> Hi Jungwoo, >>>> >>>> On 2014-10-29 23:51, Jungwoo Ha wrote: >>>>> PTAL >>>>> http://cr.openjdk.java.net/~rasbold/8061259/webrev.01/ >>>>> >>>>> >>>>> I've looked a bit at the webrev. A couple of comments: >>>>> >>>>> Why do you use OrderAccess methods for writing and >>>>> reading the _has_promo_failed flag in >>>>> has_promo_failed() and set_promot_failed() ? >>>>> >>>>> >>>>> I think that has no effect on x86, but I assumed that >>>>> processors with weak memory model may want ordering of >>>>> set/reset/has call. >>>> >>>> You don't need the OrderAccess methods for the weak >>>> memory models here either. You just race on reading the >>>> variable and if you see the "wrong" value you >>>> eventually take a lock (which will order all memory >>>> accesses) to read the variable properly. >>>> >>>> By removing the use of OrderAccess you can make >>>> ConcurrentMarkSweepGeneration::_has_promotion_failed a >>>> bool instead of a juint which simplifies the code a bit. >>>> >>>>> >>>>> Can we write out the full word "promotion" instead >>>>> of just "promo" in the variables and methods? >>>>> >>>>> >>>>> Done. >>>>> >>>>> Can we change the name of the flag from >>>>> UseCMSFastPromotionFailure to >>>>> CMSFastPromotionFailure? Most CMS flags start with >>>>> CMS and I don't think we need the "Use" prefix. >>>>> >>>>> >>>>> Done. >>>>> >>>>> What do you think about making the flag true by >>>>> default? At least for JDK 9. If we decide to >>>>> backport to JDK 8 or 7 it might be a good idea to >>>>> keep the default value as false. >>>>> >>>>> >>>>> Done. >>>>> Let me know if there is anything for me to do to >>>>> backport to JDK8 and 7. >>>> >>>> I think this fix would be worth backporting to JDK 8. I >>>> don't think there is much action required on your side. >>>> I created a backport bug for JDK 8 just to make sure >>>> that we don't forget it. It will be a little while >>>> before the 8 update repos are in a state to accept >>>> enhancements again. So, it is nice to have the backport >>>> bug to keep track of this. >>>> >>>> Backporting to JDK 7 requires some more work. Unless >>>> you have good arguments for why it is important to >>>> backport this to JDK 7 I don't think it is worth doing. >>>> >>>>> >>>>> Did you find the information provided by >>>>> _fast_promo_failure_hitcount useful for debugging? >>>>> If it not too useful I would consider removing it >>>>> since it is cluttering up the code a bit. >>>>> >>>>> >>>>> I removed it. >>>>> It was useful to for development, but I think it is no >>>>> longer needed. >>>> >>>> Great. Thanks. >>>> >>>> One more comment. This code comment appears in two >>>> places just after we have taken the lock. >>>> >>>> 3365 if (CMSFastPromotionFailure && >>>> has_promotion_failed()) { >>>> 3366 // Caller must have checked already without >>>> synchronization. >>>> 3367 // Check again here while holding the lock. >>>> 3368 return NULL; >>>> 3369 } >>>> >>>> There is actually really no requirement that the caller >>>> must have checked has_promotion_failed() before calling >>>> the method. That's just an optimization. I think the >>>> first comment can be skipped and we just leave the >>>> second comment "// Check again here while holding the >>>> lock.". I would also suggest moving that comment up to >>>> the line just before the if statement. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> >>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Thu Nov 6 01:49:44 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Nov 2014 20:49:44 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> Message-ID: On Oct 30, 2014, at 1:24 PM, Jungwoo Ha wrote: > > PTAL > http://cr.openjdk.java.net/~rasbold/8061259/webrev.02/ ------------------------------------------------------------------------------ src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp 271 // Support for CMSFastPromotionFailure 272 reset_promotion_failed(); Why this and not just initialize the data member in the constructor initializer list? ------------------------------------------------------------------------------ [All of code discussed here is in par_promote().] The tests of CMSFastPromotionFailure && has_promotion_failed() that are outside the lock have a sort of DCLP (double checked locking pattern) feel to them. The goal here is to avoid the locked region and go do something else, instead of waiting for the current lock holder to complete the work the checker would be doing if only they could get the lock. I think I would like it better if the outside the lock tests were moved closer to the lock entries, to emphasize that relationship, and to reduce the code affected by the introduction of this set of changes. That might result in the thread that gives up doing a little more work before giving up. I'm particularly thinking of 1358 if (CMSFastPromotionFailure && has_promotion_failed()) { 1359 return NULL; 1360 } removal of which would let execution proceed to 1373 if (!expand_and_ensure_spooling_space(promoInfo)) { 1374 return NULL; and give up because the moved test caused expand_xxx to return false. I don't know if the placement of the test at line 1358 rather than inside expand_and_ensure_spooling_space() would be significant. I suspect not, but without measurements I don't have a lot of confidence in that supposition. But I think that change would improve the understandability of the code. In the other case, 1381 if (CMSFastPromotionFailure && has_promotion_failed()) { 1382 return NULL; 1383 } 1384 obj_ptr = expand_and_par_lab_allocate(ps, alloc_sz); I would definitely prefer that test be moved into expand_and_par_lab_allocate(). This is the only call to that function, and the first thing it does is lock the mutex and re-perform the test. ------------------------------------------------------------------------------ I'm suspicious of the during-review change to make _has_promotion_failed non-atomic. I see the argument for the interaction between set and test; the set occurs in lock context so gets released on the way out of the lock, while the first time each core gets a stale false value it will acquire the lock and acquire the updated set value, so that future tests on the same core will avoid the code in question, until the reset finally happens. However, I think the present interaction between reset and test is not safe. A test seeing a stale true value leads to an inappropriate skip of the protected code, with nothing obvious to prevent that from occurring "forever". ------------------------------------------------------------------------------ From kim.barrett at oracle.com Thu Nov 6 01:50:30 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Nov 2014 20:50:30 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> Message-ID: On Oct 30, 2014, at 1:24 PM, Jungwoo Ha wrote: > > PTAL > http://cr.openjdk.java.net/~rasbold/8061259/webrev.02/ ------------------------------------------------------------------------------ src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp 271 // Support for CMSFastPromotionFailure 272 reset_promotion_failed(); Why this and not just initialize the data member in the constructor initializer list? ------------------------------------------------------------------------------ [All of code discussed here is in par_promote().] The tests of CMSFastPromotionFailure && has_promotion_failed() that are outside the lock have a sort of DCLP (double checked locking pattern) feel to them. The goal here is to avoid the locked region and go do something else, instead of waiting for the current lock holder to complete the work the checker would be doing if only they could get the lock. I think I would like it better if the outside the lock tests were moved closer to the lock entries, to emphasize that relationship, and to reduce the code affected by the introduction of this set of changes. That might result in the thread that gives up doing a little more work before giving up. I'm particularly thinking of 1358 if (CMSFastPromotionFailure && has_promotion_failed()) { 1359 return NULL; 1360 } removal of which would let execution proceed to 1373 if (!expand_and_ensure_spooling_space(promoInfo)) { 1374 return NULL; and give up because the moved test caused expand_xxx to return false. I don't know if the placement of the test at line 1358 rather than inside expand_and_ensure_spooling_space() would be significant. I suspect not, but without measurements I don't have a lot of confidence in that supposition. But I think that change would improve the understandability of the code. In the other case, 1381 if (CMSFastPromotionFailure && has_promotion_failed()) { 1382 return NULL; 1383 } 1384 obj_ptr = expand_and_par_lab_allocate(ps, alloc_sz); I would definitely prefer that test be moved into expand_and_par_lab_allocate(). This is the only call to that function, and the first thing it does is lock the mutex and re-perform the test. ------------------------------------------------------------------------------ I'm suspicious of the during-review change to make _has_promotion_failed non-atomic. I see the argument for the interaction between set and test; the set occurs in lock context so gets released on the way out of the lock, while the first time each core gets a stale false value it will acquire the lock and acquire the updated set value, so that future tests on the same core will avoid the code in question, until the reset finally happens. However, I think the present interaction between reset and test is not safe. A test seeing a stale true value leads to an inappropriate skip of the protected code, with nothing obvious to prevent that from occurring "forever". ------------------------------------------------------------------------------ From david.holmes at oracle.com Thu Nov 6 02:20:07 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 06 Nov 2014 12:20:07 +1000 Subject: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545A4367.1070407@oracle.com> References: <545A4367.1070407@oracle.com> Message-ID: <545ADAD7.7040902@oracle.com> Hi Evgeniya, On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: > Hi, > > Please review changes for 8062536, the OpenJDK/jdk part of the JDK-8019361 > > bug: https://bugs.openjdk.java.net/browse/JDK-8062536 > fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ > > Problem: Some tests explicitly set GC and fail when another GC is set > outside I don't see why you have done this for the test/sun/tools/jps/TestJps*.java tests. They don't set any GC flags. > Solution: Such tests marked with the jtreg tag "requires" to skip test > if there is a conflict Just wondering: Does a skipped test get a .jtr file showing it was skipped; or does it only appear in the higher-level jtreg log? Thanks, David > Tested locally with different GC flags (-XX:+UseG1GC, > -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and without > any GC flag). Tests are being excluded as expected. No tests failed > because of the conflict. > > Thanks, > Evgeniya Stepanova > > // From kim.barrett at oracle.com Thu Nov 6 02:48:24 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 5 Nov 2014 21:48:24 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <1415204663.2912.18.camel@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> Message-ID: On Nov 5, 2014, at 11:24 AM, Thomas Schatzl wrote: > > Looks good. Could you add a regression test case? New webrev with test: http://cr.openjdk.java.net/~kbarrett/8062036/webrev02 This is the same as the previous one, except for the addition of the test program. From bengt.rutisson at oracle.com Thu Nov 6 09:01:38 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 06 Nov 2014 10:01:38 +0100 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> Message-ID: <545B38F2.9040806@oracle.com> On 2014-11-06 03:48, Kim Barrett wrote: > On Nov 5, 2014, at 11:24 AM, Thomas Schatzl wrote: >> Looks good. Could you add a regression test case? > New webrev with test: > http://cr.openjdk.java.net/~kbarrett/8062036/webrev02 > > This is the same as the previous one, except for the addition of the test program. What does the test verify? When I run the test on a build that does not have your fix it still passes: $ java -showversion -jar /localhome/tests/jtreg/lib/jtreg.jar -noreport test/gc/TestScavengeALot.java java version "1.9.0-ea" Java(TM) SE Runtime Environment (build 1.9.0-ea-b35) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-ea-b35, mixed mode) Test results: passed: 1 Also, since this is a CMS specific test I think it should be located in the "test/gc/concurrentMarkSweep". Bengt > > From evgeniya.stepanova at oracle.com Thu Nov 6 10:05:48 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Thu, 06 Nov 2014 14:05:48 +0400 Subject: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545ADAD7.7040902@oracle.com> References: <545A4367.1070407@oracle.com> <545ADAD7.7040902@oracle.com> Message-ID: <545B47FC.1070305@oracle.com> Hi David, tag added because tests contain string cmd.addAll(JpsHelper.getVmArgs()); and JpsHelper defines ... public static final String[] VM_ARGS = {"-Xmx512m", "-XX:+UseParallelGC"}; ... public static List getVmArgs() throws IOException { if (testVmArgs == null) { testVmArgs = new ArrayList<>(); testVmArgs.addAll(Arrays.asList(VM_ARGS)); testVmArgs.add("-XX:Flags=" + getVmFlagsFile().getAbsolutePath()); } return testVmArgs; } Tests itself wouldn't fail if we use another GC, tag added for cleanup-if we use for example SerialGC we must be sure that tests passed with this GC, not with another one. Now you will assume that concrete test passed with Serial GC, but it run only with Parallel GC. Plus there is no any sense to run test twice in TC (with different GC, since it use only Parallel) Thanks, Evgeniya Stepanova On 06.11.2014 6:20, David Holmes wrote: > Hi Evgeniya, > > On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >> Hi, >> >> Please review changes for 8062536, the OpenJDK/jdk part of the >> JDK-8019361 >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >> >> Problem: Some tests explicitly set GC and fail when another GC is set >> outside > > I don't see why you have done this for the > > test/sun/tools/jps/TestJps*.java > > tests. They don't set any GC flags. > >> Solution: Such tests marked with the jtreg tag "requires" to skip test >> if there is a conflict > > Just wondering: Does a skipped test get a .jtr file showing it was > skipped; or does it only appear in the higher-level jtreg log? > > Thanks, > David > >> Tested locally with different GC flags (-XX:+UseG1GC, >> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and without >> any GC flag). Tests are being excluded as expected. No tests failed >> because of the conflict. >> >> Thanks, >> Evgeniya Stepanova >> >> // -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.friberg at oracle.com Thu Nov 6 10:12:22 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Thu, 06 Nov 2014 11:12:22 +0100 Subject: RFR: JDK-8055845 - Add trace event for promoted objects In-Reply-To: <5416D64C.9020207@oracle.com> References: <53FB6432.1010501@oracle.com> <53FB6A86.1080908@oracle.com> <53FB7338.9030208@oracle.com> <53FC4A57.3030100@oracle.com> <53FDEB15.2070102@oracle.com> <53FEF5F4.5060509@oracle.com> <53FF9C00.5040306@oracle.com> <54001EFF.3030802@oracle.com> <5400FF6F.5040203@oracle.com> <540A3730.1060603@oracle.com> <5416D64C.9020207@oracle.com> Message-ID: <545B4986.1050700@oracle.com> Hi, After further off list discussion it was decided to keep the gc_tracer in par_promote as is. I have uploaded a new webrev, http://cr.openjdk.java.net/~sfriberg/8055845/webrev.05 The main change here is a rewrite of the G1 code which is cleaner and also reuses the read age. By sending the markOop down through the call we can now trust the read age and do not need to reread it when incrementing which improves the YC performance slightly as it avoids the rather complex bit extraction. Cheers, Staffan On 09/15/2014 02:06 PM, Bengt Rutisson wrote: > > Hi Staffan, > > psPromotionManager.inline.hpp > > I think the PSPromotionManager::copy_to_survivor_space() might send > multiple events. If the allocation to the young gen fails we will fall > through to do an old gen allocation. But we send the events before we > realize that the allocation has failed, i.e. new_obj is NULL. > > I talked to Erik a bit about how to handle the gc_tracer in > par_promote. He'll get back to you with some thoughts on that. > > Thanks, > Bengt > > > On 2014-09-06 00:20, Staffan Friberg wrote: >> Hi, >> >> I have uploaded a new webrev here, >> cr.openjdk.java.net/~sfriberg/8055845/webrev.03 >> >> It contains several changes >> >> - Split event into two events (PromoteObjectInNewPLAB, >> PromoteObjectOutsidePLAB) >> - Moved events to "vm/gc/detailed/PromoteObject..." >> - Supporting ParNew+CMS and ParNew+SerialOld tenuring >> - Not sure if the way I do it with passing the ParNewTracer >> is the best solution, please let me know if you have an idea how to >> improve it >> - Simplified the G1 code to avoid sending age and having a single >> call site >> - Fixed so that the generated event has size information in bytes >> rather than words >> >> Thanks for offline comments and suggestions from Dmitry and Thomas. >> >> Cheers, >> Staffan >> >> On 08/29/2014 03:32 PM, Staffan Friberg wrote: >>> Hi Erik, >>> >>> On 08/28/2014 11:34 PM, Erik Helin wrote: >>>> (it seems like we lost hotspot-gc-dev at openjdk.java.net somewhere in >>>> this thread, I'm adding it back) >>>> >>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>> Hi Erik, >>>>> >>>>> Thanks for the comments. >>>>>> - Aren't the events for promotion to the tenured generation >>>>>> (SerialOld) >>>>>> and the CMS generation missing? >>>>> The reason for leaving out these two, Serial completely and CMS >>>>> promotion, was due to that neither as far as I understand make use of >>>>> PLABs. >>>> >>>> I might be wrong here, but looking at the function >>>> TenuredGeneration::par_promote (in tenuredGeneration.cpp) it looks >>>> to me like SerialOld is using PLABs when ParNew is promoting >>>> objects from young to old. >>>> >>>> As for CMS, looking at ConcurrentMarkSweepGeneration::par_promote >>>> (in concurrentMarkSweepGeneration.cpp) it seems like each >>>> CMSParGCThreadState has a CFLS_LAB (CompactibleFreeListSpace Local >>>> Allocation Buffer) that is a thread-local allocation buffer. See >>>> compactibleFreeListSpace.{hpp,cpp} for more details. >>>> >>>> Given this, I think we should add events for Serial and CMS as well. >>>> >>> >>> Ok I see what you mean with CMS, basically the equivalent to getting >>> a PLAB would be when we refill the CFLS_LAB in the alloc function. >>> It still does the allocation to a small memory (~ size of object) >>> area from the freelist, but at least we did extra work to refill the >>> local CFLS_LAB. Need to do some analysis to see how often we refill >>> so the overhead doesn't get too high. >>> The only issue I run into is how I can in a nice way get access to >>> the ParNewTracer from ParNewGeneration to call the report function. >>> Let's sync up next week and see how it can be solved. >>> >>> The tenured GC requires something similar to be supported, however >>> since ParNewGC is deprecated for usage without CMS in JDK 8 we might >>> want to skip that combination. >>> >>> >>>> >>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>> - Would it make sense to differentiate, either by separate events >>>>>> or by >>>>>> a field in a event, between promotions to to-space and to the old >>>>>> generation? >>>>>> - The are two events for TLAB allocations, >>>>>> java/object_alloc_in_new_TLAB and java/object_alloc_outside_TLAB. >>>>>> What do you think about using two events for PLAB allocations >>>>>> as well: >>>>>> - java/object_alloc_in_new_PLAB >>>>>> - java/object_alloc_outside_PLAB >>>>> I think this is a matter of taste and probably how similar we want >>>>> the >>>>> event to be to the existing allocation event. I personally prefer a >>>>> single event but if the GC team and serviceability team feel it >>>>> would be >>>>> better to have two I can certainly rewrite. The reason for me >>>>> preferring >>>>> a single event is just ease of analysis, you can easily filter a >>>>> list of >>>>> events on a field, it is harder to merge two different events with >>>>> different fields and work with them in an straight forward fashion. >>>>> >>>>> Any general preference for having a single or multiple events? >>>> >>>> I would prefer to have two events in this case and try to follow >>>> the existing allocation events as much as possible (both in naming >>>> and in style). Keeping the tenured field (I missed it the first >>>> time I read the patch) is good. >>>> >>> Yes, tenured would be independent of having two events, only PLAB >>> size and directAllocation would be affected when having two event types. >>> >>> *Erik Gahlin*, any preference from your end? >>> >>> >>> >>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>> - In PSPromotionManager, instead of utilizing the C++ friendship >>>>>> with >>>>>> PSScavenge, should we add a getter function for the gc_tracer? >>>>> Created a getter function. >>>> >>>> Thanks! If you make report_promotion_sample const, then the getter >>>> can return a const ParallelScavengeTracer*, right? >>>> >>> Done >>> >>> //Staffan >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Nov 6 10:13:23 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 06 Nov 2014 20:13:23 +1000 Subject: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B47FC.1070305@oracle.com> References: <545A4367.1070407@oracle.com> <545ADAD7.7040902@oracle.com> <545B47FC.1070305@oracle.com> Message-ID: <545B49C3.2040007@oracle.com> On 6/11/2014 8:05 PM, Evgeniya Stepanova wrote: > Hi David, > > tag added because tests contain string > cmd.addAll(JpsHelper.getVmArgs()); > > and JpsHelper defines > ... > public static final String[] VM_ARGS = {"-Xmx512m", "-XX:+UseParallelGC"}; Ah I see! Strange. Thanks for clarifying. Looks okay then. David ----- > ... > public static List getVmArgs() throws IOException { > if (testVmArgs == null) { > testVmArgs = new ArrayList<>(); > testVmArgs.addAll(Arrays.asList(VM_ARGS)); > testVmArgs.add("-XX:Flags=" + > getVmFlagsFile().getAbsolutePath()); > } > return testVmArgs; > } > > Tests itself wouldn't fail if we use another GC, tag added for > cleanup-if we use for example SerialGC we must be sure that tests passed > with this GC, not with another one. Now you will assume that concrete > test passed with Serial GC, but it run only with Parallel GC. Plus there > is no any sense to run test twice in TC (with different GC, since it use > only Parallel) > > Thanks, > Evgeniya Stepanova > On 06.11.2014 6:20, David Holmes wrote: >> Hi Evgeniya, >> >> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>> Hi, >>> >>> Please review changes for 8062536, the OpenJDK/jdk part of the >>> JDK-8019361 >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>> >>> Problem: Some tests explicitly set GC and fail when another GC is set >>> outside >> >> I don't see why you have done this for the >> >> test/sun/tools/jps/TestJps*.java >> >> tests. They don't set any GC flags. >> >>> Solution: Such tests marked with the jtreg tag "requires" to skip test >>> if there is a conflict >> >> Just wondering: Does a skipped test get a .jtr file showing it was >> skipped; or does it only appear in the higher-level jtreg log? >> >> Thanks, >> David >> >>> Tested locally with different GC flags (-XX:+UseG1GC, >>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and without >>> any GC flag). Tests are being excluded as expected. No tests failed >>> because of the conflict. >>> >>> Thanks, >>> Evgeniya Stepanova >>> >>> // > > -- > /Evgeniya Stepanova/ From bengt.rutisson at oracle.com Thu Nov 6 11:01:52 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 06 Nov 2014 12:01:52 +0100 Subject: RFR: JDK-8055845 - Add trace event for promoted objects In-Reply-To: <545B4986.1050700@oracle.com> References: <53FB6432.1010501@oracle.com> <53FB6A86.1080908@oracle.com> <53FB7338.9030208@oracle.com> <53FC4A57.3030100@oracle.com> <53FDEB15.2070102@oracle.com> <53FEF5F4.5060509@oracle.com> <53FF9C00.5040306@oracle.com> <54001EFF.3030802@oracle.com> <5400FF6F.5040203@oracle.com> <540A3730.1060603@oracle.com> <5416D64C.9020207@oracle.com> <545B4986.1050700@oracle.com> Message-ID: <545B5520.7010407@oracle.com> Hi Staffan, On 2014-11-06 11:12, Staffan Friberg wrote: > Hi, > > After further off list discussion it was decided to keep the gc_tracer > in par_promote as is. > > I have uploaded a new webrev, > http://cr.openjdk.java.net/~sfriberg/8055845/webrev.05 > > The main change here is a rewrite of the G1 code which is cleaner and > also reuses the read age. By sending the markOop down through the call > we can now trust the read age and do not need to reread it when > incrementing which improves the YC performance slightly as it avoids > the rather complex bit extraction. Looks good to me. One detail in parNewGeneration.cpp: 274 } else { 275 // Too large; allocate the object individually. 276 gc_tracer()->report_promotion_outside_plab_event(old, word_sz, age, false); 277 obj = sp->par_allocate(word_sz); 278 } Seems like par_allocate() return NULL. Maybe we should check that before reporting the event. Similarly to what you do in the other GCs, for example G1: g1Allocator.cpp 141 obj = _g1h->par_allocate_during_gc(purpose, word_sz, context); 142 if (obj != NULL 143 && _g1h->_gc_tracer_stw->should_report_promotion_outside_plab_event()) { 144 bool tenured = _g1h->heap_region_containing_raw(obj)->is_old(); 145 _g1h->_gc_tracer_stw->report_promotion_outside_plab_event(old, word_sz, 146 age, tenured); 147 } Thanks, Bengt > > Cheers, > Staffan > > On 09/15/2014 02:06 PM, Bengt Rutisson wrote: >> >> Hi Staffan, >> >> psPromotionManager.inline.hpp >> >> I think the PSPromotionManager::copy_to_survivor_space() might send >> multiple events. If the allocation to the young gen fails we will >> fall through to do an old gen allocation. But we send the events >> before we realize that the allocation has failed, i.e. new_obj is NULL. >> >> I talked to Erik a bit about how to handle the gc_tracer in >> par_promote. He'll get back to you with some thoughts on that. >> >> Thanks, >> Bengt >> >> >> On 2014-09-06 00:20, Staffan Friberg wrote: >>> Hi, >>> >>> I have uploaded a new webrev here, >>> cr.openjdk.java.net/~sfriberg/8055845/webrev.03 >>> >>> It contains several changes >>> >>> - Split event into two events (PromoteObjectInNewPLAB, >>> PromoteObjectOutsidePLAB) >>> - Moved events to "vm/gc/detailed/PromoteObject..." >>> - Supporting ParNew+CMS and ParNew+SerialOld tenuring >>> - Not sure if the way I do it with passing the ParNewTracer >>> is the best solution, please let me know if you have an idea how to >>> improve it >>> - Simplified the G1 code to avoid sending age and having a >>> single call site >>> - Fixed so that the generated event has size information in >>> bytes rather than words >>> >>> Thanks for offline comments and suggestions from Dmitry and Thomas. >>> >>> Cheers, >>> Staffan >>> >>> On 08/29/2014 03:32 PM, Staffan Friberg wrote: >>>> Hi Erik, >>>> >>>> On 08/28/2014 11:34 PM, Erik Helin wrote: >>>>> (it seems like we lost hotspot-gc-dev at openjdk.java.net somewhere >>>>> in this thread, I'm adding it back) >>>>> >>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>> Hi Erik, >>>>>> >>>>>> Thanks for the comments. >>>>>>> - Aren't the events for promotion to the tenured generation >>>>>>> (SerialOld) >>>>>>> and the CMS generation missing? >>>>>> The reason for leaving out these two, Serial completely and CMS >>>>>> promotion, was due to that neither as far as I understand make >>>>>> use of >>>>>> PLABs. >>>>> >>>>> I might be wrong here, but looking at the function >>>>> TenuredGeneration::par_promote (in tenuredGeneration.cpp) it looks >>>>> to me like SerialOld is using PLABs when ParNew is promoting >>>>> objects from young to old. >>>>> >>>>> As for CMS, looking at ConcurrentMarkSweepGeneration::par_promote >>>>> (in concurrentMarkSweepGeneration.cpp) it seems like each >>>>> CMSParGCThreadState has a CFLS_LAB (CompactibleFreeListSpace Local >>>>> Allocation Buffer) that is a thread-local allocation buffer. See >>>>> compactibleFreeListSpace.{hpp,cpp} for more details. >>>>> >>>>> Given this, I think we should add events for Serial and CMS as well. >>>>> >>>> >>>> Ok I see what you mean with CMS, basically the equivalent to >>>> getting a PLAB would be when we refill the CFLS_LAB in the alloc >>>> function. It still does the allocation to a small memory (~ size of >>>> object) area from the freelist, but at least we did extra work to >>>> refill the local CFLS_LAB. Need to do some analysis to see how >>>> often we refill so the overhead doesn't get too high. >>>> The only issue I run into is how I can in a nice way get access to >>>> the ParNewTracer from ParNewGeneration to call the report function. >>>> Let's sync up next week and see how it can be solved. >>>> >>>> The tenured GC requires something similar to be supported, however >>>> since ParNewGC is deprecated for usage without CMS in JDK 8 we >>>> might want to skip that combination. >>>> >>>> >>>>> >>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>> - Would it make sense to differentiate, either by separate >>>>>>> events or by >>>>>>> a field in a event, between promotions to to-space and to the old >>>>>>> generation? >>>>>>> - The are two events for TLAB allocations, >>>>>>> java/object_alloc_in_new_TLAB and java/object_alloc_outside_TLAB. >>>>>>> What do you think about using two events for PLAB allocations >>>>>>> as well: >>>>>>> - java/object_alloc_in_new_PLAB >>>>>>> - java/object_alloc_outside_PLAB >>>>>> I think this is a matter of taste and probably how similar we >>>>>> want the >>>>>> event to be to the existing allocation event. I personally prefer a >>>>>> single event but if the GC team and serviceability team feel it >>>>>> would be >>>>>> better to have two I can certainly rewrite. The reason for me >>>>>> preferring >>>>>> a single event is just ease of analysis, you can easily filter a >>>>>> list of >>>>>> events on a field, it is harder to merge two different events with >>>>>> different fields and work with them in an straight forward fashion. >>>>>> >>>>>> Any general preference for having a single or multiple events? >>>>> >>>>> I would prefer to have two events in this case and try to follow >>>>> the existing allocation events as much as possible (both in naming >>>>> and in style). Keeping the tenured field (I missed it the first >>>>> time I read the patch) is good. >>>>> >>>> Yes, tenured would be independent of having two events, only PLAB >>>> size and directAllocation would be affected when having two event >>>> types. >>>> >>>> *Erik Gahlin*, any preference from your end? >>>> >>>> >>>> >>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>> - In PSPromotionManager, instead of utilizing the C++ friendship >>>>>>> with >>>>>>> PSScavenge, should we add a getter function for the gc_tracer? >>>>>> Created a getter function. >>>>> >>>>> Thanks! If you make report_promotion_sample const, then the getter >>>>> can return a const ParallelScavengeTracer*, right? >>>>> >>>> Done >>>> >>>> //Staffan >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Thu Nov 6 10:22:46 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 06 Nov 2014 14:22:46 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B51A1.7080905@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> Message-ID: <545B4BF6.4040209@oracle.com> Hi Katja, You are right, there will be no conflict, because test ignores any external VM flags. So, adding @requires seems unnecessary here, but... Ignoring external options is bad thing, such "selfish" tests are not applicable for other areas, like GC, compiler, RT. @requires will allow to modify tests to include external vm options without any risk of bumping into conflict and extend area of test applicability. But if you still believe, that @requires is not necessary - it's not a problem, tests could be kept as is. Thanks, Dima On 06.11.2014 16:27, Yekaterina Kantserova wrote: > > Hi Evgeniya, > > As David has pointed out these jps tests are not testing gc. The > -XX:+UseParallelGC is just an arbitrary chosen test flag. There should > not be any conflicts either since these tests are running in driver mode: > > ... > * @run driver TestJpsJar > ... > > which means no flags from above are accepted. > > Thanks, > Katja > > > > On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >> Hi David, >> >> tag added because tests contain string >> cmd.addAll(JpsHelper.getVmArgs()); >> >> and JpsHelper defines >> ... >> public static final String[] VM_ARGS = {"-Xmx512m", >> "-XX:+UseParallelGC"}; >> ... >> public static List getVmArgs() throws IOException { >> if (testVmArgs == null) { >> testVmArgs = new ArrayList<>(); >> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >> testVmArgs.add("-XX:Flags=" + >> getVmFlagsFile().getAbsolutePath()); >> } >> return testVmArgs; >> } >> >> Tests itself wouldn't fail if we use another GC, tag added for >> cleanup-if we use for example SerialGC we must be sure that tests >> passed with this GC, not with another one. Now you will assume that >> concrete test passed with Serial GC, but it run only with Parallel >> GC. Plus there is no any sense to run test twice in TC (with >> different GC, since it use only Parallel) >> >> Thanks, >> Evgeniya Stepanova >> On 06.11.2014 6:20, David Holmes wrote: >>> Hi Evgeniya, >>> >>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>> Hi, >>>> >>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>> JDK-8019361 >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>> >>>> Problem: Some tests explicitly set GC and fail when another GC is set >>>> outside >>> >>> I don't see why you have done this for the >>> >>> test/sun/tools/jps/TestJps*.java >>> >>> tests. They don't set any GC flags. >>> >>>> Solution: Such tests marked with the jtreg tag "requires" to skip test >>>> if there is a conflict >>> >>> Just wondering: Does a skipped test get a .jtr file showing it was >>> skipped; or does it only appear in the higher-level jtreg log? >>> >>> Thanks, >>> David >>> >>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>> without >>>> any GC flag). Tests are being excluded as expected. No tests failed >>>> because of the conflict. >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> >>>> // >> >> -- >> /Evgeniya Stepanova/ > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Thu Nov 6 12:44:31 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 06 Nov 2014 13:44:31 +0100 Subject: RFR: 8061467: Bad page size passed to setup_large_pages() on Solaris In-Reply-To: <545AA09F.4040504@oracle.com> References: <544FDC0F.2040805@oracle.com> <1415102977.5218.8.camel@oracle.com> <1415200465.2912.11.camel@oracle.com> <545AA09F.4040504@oracle.com> Message-ID: <1415277871.3211.4.camel@oracle.com> Hi, On Wed, 2014-11-05 at 14:11 -0800, Jon Masamitsu wrote: > Thomas, > > Change look good but in the test, did you consider > adding -XX:+UseLargePages to the command line in > the test? On Solaris large pages are on by default > but not so with all of the platforms. no, I did not, I was in a bit of a hurry to not miss promotion that day, as this bug prevents a lot of tests from running. I added JDK-8063110 for that specifically. Thanks, Thomas From staffan.friberg at oracle.com Thu Nov 6 13:00:02 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Thu, 06 Nov 2014 14:00:02 +0100 Subject: RFR: JDK-8055845 - Add trace event for promoted objects In-Reply-To: <545B5520.7010407@oracle.com> References: <53FB6432.1010501@oracle.com> <53FB6A86.1080908@oracle.com> <53FB7338.9030208@oracle.com> <53FC4A57.3030100@oracle.com> <53FDEB15.2070102@oracle.com> <53FEF5F4.5060509@oracle.com> <53FF9C00.5040306@oracle.com> <54001EFF.3030802@oracle.com> <5400FF6F.5040203@oracle.com> <540A3730.1060603@oracle.com> <5416D64C.9020207@oracle.com> <545B4986.1050700@oracle.com> <545B5520.7010407@oracle.com> Message-ID: <545B70D2.9080008@oracle.com> Good catch, fixed. // Too large; allocate the object individually. obj = sp->par_allocate(word_sz); if (obj != NULL) { gc_tracer()->report_promotion_outside_plab_event(old, word_sz, age, false); } Please let me know if anyone wants a full new webrev with this. Thanks, Staffan On 11/06/2014 12:01 PM, Bengt Rutisson wrote: > > Hi Staffan, > > On 2014-11-06 11:12, Staffan Friberg wrote: >> Hi, >> >> After further off list discussion it was decided to keep the >> gc_tracer in par_promote as is. >> >> I have uploaded a new webrev, >> http://cr.openjdk.java.net/~sfriberg/8055845/webrev.05 >> >> The main change here is a rewrite of the G1 code which is cleaner and >> also reuses the read age. By sending the markOop down through the >> call we can now trust the read age and do not need to reread it when >> incrementing which improves the YC performance slightly as it avoids >> the rather complex bit extraction. > > > Looks good to me. One detail in parNewGeneration.cpp: > > 274 } else { > 275 // Too large; allocate the object individually. > 276 gc_tracer()->report_promotion_outside_plab_event(old, word_sz, > age, false); > 277 obj = sp->par_allocate(word_sz); > 278 } > > Seems like par_allocate() return NULL. Maybe we should check that > before reporting the event. Similarly to what you do in the other GCs, > for example G1: > > g1Allocator.cpp > > 141 obj = _g1h->par_allocate_during_gc(purpose, word_sz, context); > 142 if (obj != NULL > 143 && > _g1h->_gc_tracer_stw->should_report_promotion_outside_plab_event()) { > 144 bool tenured = _g1h->heap_region_containing_raw(obj)->is_old(); > 145 _g1h->_gc_tracer_stw->report_promotion_outside_plab_event(old, > word_sz, > 146 age, tenured); > 147 } > > Thanks, > Bengt > > > >> >> Cheers, >> Staffan >> >> On 09/15/2014 02:06 PM, Bengt Rutisson wrote: >>> >>> Hi Staffan, >>> >>> psPromotionManager.inline.hpp >>> >>> I think the PSPromotionManager::copy_to_survivor_space() might send >>> multiple events. If the allocation to the young gen fails we will >>> fall through to do an old gen allocation. But we send the events >>> before we realize that the allocation has failed, i.e. new_obj is NULL. >>> >>> I talked to Erik a bit about how to handle the gc_tracer in >>> par_promote. He'll get back to you with some thoughts on that. >>> >>> Thanks, >>> Bengt >>> >>> >>> On 2014-09-06 00:20, Staffan Friberg wrote: >>>> Hi, >>>> >>>> I have uploaded a new webrev here, >>>> cr.openjdk.java.net/~sfriberg/8055845/webrev.03 >>>> >>>> It contains several changes >>>> >>>> - Split event into two events (PromoteObjectInNewPLAB, >>>> PromoteObjectOutsidePLAB) >>>> - Moved events to "vm/gc/detailed/PromoteObject..." >>>> - Supporting ParNew+CMS and ParNew+SerialOld tenuring >>>> - Not sure if the way I do it with passing the >>>> ParNewTracer is the best solution, please let me know if you have >>>> an idea how to improve it >>>> - Simplified the G1 code to avoid sending age and having a >>>> single call site >>>> - Fixed so that the generated event has size information in >>>> bytes rather than words >>>> >>>> Thanks for offline comments and suggestions from Dmitry and Thomas. >>>> >>>> Cheers, >>>> Staffan >>>> >>>> On 08/29/2014 03:32 PM, Staffan Friberg wrote: >>>>> Hi Erik, >>>>> >>>>> On 08/28/2014 11:34 PM, Erik Helin wrote: >>>>>> (it seems like we lost hotspot-gc-dev at openjdk.java.net somewhere >>>>>> in this thread, I'm adding it back) >>>>>> >>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>> Hi Erik, >>>>>>> >>>>>>> Thanks for the comments. >>>>>>>> - Aren't the events for promotion to the tenured generation >>>>>>>> (SerialOld) >>>>>>>> and the CMS generation missing? >>>>>>> The reason for leaving out these two, Serial completely and CMS >>>>>>> promotion, was due to that neither as far as I understand make >>>>>>> use of >>>>>>> PLABs. >>>>>> >>>>>> I might be wrong here, but looking at the function >>>>>> TenuredGeneration::par_promote (in tenuredGeneration.cpp) it >>>>>> looks to me like SerialOld is using PLABs when ParNew is >>>>>> promoting objects from young to old. >>>>>> >>>>>> As for CMS, looking at ConcurrentMarkSweepGeneration::par_promote >>>>>> (in concurrentMarkSweepGeneration.cpp) it seems like each >>>>>> CMSParGCThreadState has a CFLS_LAB (CompactibleFreeListSpace >>>>>> Local Allocation Buffer) that is a thread-local allocation >>>>>> buffer. See compactibleFreeListSpace.{hpp,cpp} for more details. >>>>>> >>>>>> Given this, I think we should add events for Serial and CMS as well. >>>>>> >>>>> >>>>> Ok I see what you mean with CMS, basically the equivalent to >>>>> getting a PLAB would be when we refill the CFLS_LAB in the alloc >>>>> function. It still does the allocation to a small memory (~ size >>>>> of object) area from the freelist, but at least we did extra work >>>>> to refill the local CFLS_LAB. Need to do some analysis to see how >>>>> often we refill so the overhead doesn't get too high. >>>>> The only issue I run into is how I can in a nice way get access to >>>>> the ParNewTracer from ParNewGeneration to call the report >>>>> function. Let's sync up next week and see how it can be solved. >>>>> >>>>> The tenured GC requires something similar to be supported, however >>>>> since ParNewGC is deprecated for usage without CMS in JDK 8 we >>>>> might want to skip that combination. >>>>> >>>>> >>>>>> >>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>>> - Would it make sense to differentiate, either by separate >>>>>>>> events or by >>>>>>>> a field in a event, between promotions to to-space and to the >>>>>>>> old >>>>>>>> generation? >>>>>>>> - The are two events for TLAB allocations, >>>>>>>> java/object_alloc_in_new_TLAB and >>>>>>>> java/object_alloc_outside_TLAB. >>>>>>>> What do you think about using two events for PLAB allocations >>>>>>>> as well: >>>>>>>> - java/object_alloc_in_new_PLAB >>>>>>>> - java/object_alloc_outside_PLAB >>>>>>> I think this is a matter of taste and probably how similar we >>>>>>> want the >>>>>>> event to be to the existing allocation event. I personally prefer a >>>>>>> single event but if the GC team and serviceability team feel it >>>>>>> would be >>>>>>> better to have two I can certainly rewrite. The reason for me >>>>>>> preferring >>>>>>> a single event is just ease of analysis, you can easily filter a >>>>>>> list of >>>>>>> events on a field, it is harder to merge two different events with >>>>>>> different fields and work with them in an straight forward fashion. >>>>>>> >>>>>>> Any general preference for having a single or multiple events? >>>>>> >>>>>> I would prefer to have two events in this case and try to follow >>>>>> the existing allocation events as much as possible (both in >>>>>> naming and in style). Keeping the tenured field (I missed it the >>>>>> first time I read the patch) is good. >>>>>> >>>>> Yes, tenured would be independent of having two events, only PLAB >>>>> size and directAllocation would be affected when having two event >>>>> types. >>>>> >>>>> *Erik Gahlin*, any preference from your end? >>>>> >>>>> >>>>> >>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>>> - In PSPromotionManager, instead of utilizing the C++ >>>>>>>> friendship with >>>>>>>> PSScavenge, should we add a getter function for the gc_tracer? >>>>>>> Created a getter function. >>>>>> >>>>>> Thanks! If you make report_promotion_sample const, then the >>>>>> getter can return a const ParallelScavengeTracer*, right? >>>>>> >>>>> Done >>>>> >>>>> //Staffan >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Nov 6 12:53:44 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 06 Nov 2014 13:53:44 +0100 Subject: RFR: JDK-8055845 - Add trace event for promoted objects In-Reply-To: <545B70D2.9080008@oracle.com> References: <53FB6432.1010501@oracle.com> <53FB6A86.1080908@oracle.com> <53FB7338.9030208@oracle.com> <53FC4A57.3030100@oracle.com> <53FDEB15.2070102@oracle.com> <53FEF5F4.5060509@oracle.com> <53FF9C00.5040306@oracle.com> <54001EFF.3030802@oracle.com> <5400FF6F.5040203@oracle.com> <540A3730.1060603@oracle.com> <5416D64C.9020207@oracle.com> <545B4986.1050700@oracle.com> <545B5520.7010407@oracle.com> <545B70D2.9080008@oracle.com> Message-ID: <545B6F58.2000907@oracle.com> On 2014-11-06 14:00, Staffan Friberg wrote: > Good catch, fixed. > > // Too large; allocate the object individually. > obj = sp->par_allocate(word_sz); > if (obj != NULL) { > gc_tracer()->report_promotion_outside_plab_event(old, word_sz, > age, false); > } > > Please let me know if anyone wants a full new webrev with this. Looks good. Reviewed. Bengt > > Thanks, > Staffan > > > On 11/06/2014 12:01 PM, Bengt Rutisson wrote: >> >> Hi Staffan, >> >> On 2014-11-06 11:12, Staffan Friberg wrote: >>> Hi, >>> >>> After further off list discussion it was decided to keep the >>> gc_tracer in par_promote as is. >>> >>> I have uploaded a new webrev, >>> http://cr.openjdk.java.net/~sfriberg/8055845/webrev.05 >>> >>> The main change here is a rewrite of the G1 code which is cleaner >>> and also reuses the read age. By sending the markOop down through >>> the call we can now trust the read age and do not need to reread it >>> when incrementing which improves the YC performance slightly as it >>> avoids the rather complex bit extraction. >> >> >> Looks good to me. One detail in parNewGeneration.cpp: >> >> 274 } else { >> 275 // Too large; allocate the object individually. >> 276 gc_tracer()->report_promotion_outside_plab_event(old, word_sz, >> age, false); >> 277 obj = sp->par_allocate(word_sz); >> 278 } >> >> Seems like par_allocate() return NULL. Maybe we should check that >> before reporting the event. Similarly to what you do in the other >> GCs, for example G1: >> >> g1Allocator.cpp >> >> 141 obj = _g1h->par_allocate_during_gc(purpose, word_sz, context); >> 142 if (obj != NULL >> 143 && >> _g1h->_gc_tracer_stw->should_report_promotion_outside_plab_event()) { >> 144 bool tenured = >> _g1h->heap_region_containing_raw(obj)->is_old(); >> 145 _g1h->_gc_tracer_stw->report_promotion_outside_plab_event(old, >> word_sz, >> 146 age, tenured); >> 147 } >> >> Thanks, >> Bengt >> >> >> >>> >>> Cheers, >>> Staffan >>> >>> On 09/15/2014 02:06 PM, Bengt Rutisson wrote: >>>> >>>> Hi Staffan, >>>> >>>> psPromotionManager.inline.hpp >>>> >>>> I think the PSPromotionManager::copy_to_survivor_space() might send >>>> multiple events. If the allocation to the young gen fails we will >>>> fall through to do an old gen allocation. But we send the events >>>> before we realize that the allocation has failed, i.e. new_obj is NULL. >>>> >>>> I talked to Erik a bit about how to handle the gc_tracer in >>>> par_promote. He'll get back to you with some thoughts on that. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> >>>> On 2014-09-06 00:20, Staffan Friberg wrote: >>>>> Hi, >>>>> >>>>> I have uploaded a new webrev here, >>>>> cr.openjdk.java.net/~sfriberg/8055845/webrev.03 >>>>> >>>>> It contains several changes >>>>> >>>>> - Split event into two events (PromoteObjectInNewPLAB, >>>>> PromoteObjectOutsidePLAB) >>>>> - Moved events to "vm/gc/detailed/PromoteObject..." >>>>> - Supporting ParNew+CMS and ParNew+SerialOld tenuring >>>>> - Not sure if the way I do it with passing the >>>>> ParNewTracer is the best solution, please let me know if you have >>>>> an idea how to improve it >>>>> - Simplified the G1 code to avoid sending age and having a >>>>> single call site >>>>> - Fixed so that the generated event has size information in >>>>> bytes rather than words >>>>> >>>>> Thanks for offline comments and suggestions from Dmitry and Thomas. >>>>> >>>>> Cheers, >>>>> Staffan >>>>> >>>>> On 08/29/2014 03:32 PM, Staffan Friberg wrote: >>>>>> Hi Erik, >>>>>> >>>>>> On 08/28/2014 11:34 PM, Erik Helin wrote: >>>>>>> (it seems like we lost hotspot-gc-dev at openjdk.java.net somewhere >>>>>>> in this thread, I'm adding it back) >>>>>>> >>>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> Thanks for the comments. >>>>>>>>> - Aren't the events for promotion to the tenured generation >>>>>>>>> (SerialOld) >>>>>>>>> and the CMS generation missing? >>>>>>>> The reason for leaving out these two, Serial completely and CMS >>>>>>>> promotion, was due to that neither as far as I understand make >>>>>>>> use of >>>>>>>> PLABs. >>>>>>> >>>>>>> I might be wrong here, but looking at the function >>>>>>> TenuredGeneration::par_promote (in tenuredGeneration.cpp) it >>>>>>> looks to me like SerialOld is using PLABs when ParNew is >>>>>>> promoting objects from young to old. >>>>>>> >>>>>>> As for CMS, looking at >>>>>>> ConcurrentMarkSweepGeneration::par_promote (in >>>>>>> concurrentMarkSweepGeneration.cpp) it seems like each >>>>>>> CMSParGCThreadState has a CFLS_LAB (CompactibleFreeListSpace >>>>>>> Local Allocation Buffer) that is a thread-local allocation >>>>>>> buffer. See compactibleFreeListSpace.{hpp,cpp} for more details. >>>>>>> >>>>>>> Given this, I think we should add events for Serial and CMS as >>>>>>> well. >>>>>>> >>>>>> >>>>>> Ok I see what you mean with CMS, basically the equivalent to >>>>>> getting a PLAB would be when we refill the CFLS_LAB in the alloc >>>>>> function. It still does the allocation to a small memory (~ size >>>>>> of object) area from the freelist, but at least we did extra work >>>>>> to refill the local CFLS_LAB. Need to do some analysis to see how >>>>>> often we refill so the overhead doesn't get too high. >>>>>> The only issue I run into is how I can in a nice way get access >>>>>> to the ParNewTracer from ParNewGeneration to call the report >>>>>> function. Let's sync up next week and see how it can be solved. >>>>>> >>>>>> The tenured GC requires something similar to be supported, >>>>>> however since ParNewGC is deprecated for usage without CMS in JDK >>>>>> 8 we might want to skip that combination. >>>>>> >>>>>> >>>>>>> >>>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>>>> - Would it make sense to differentiate, either by separate >>>>>>>>> events or by >>>>>>>>> a field in a event, between promotions to to-space and to >>>>>>>>> the old >>>>>>>>> generation? >>>>>>>>> - The are two events for TLAB allocations, >>>>>>>>> java/object_alloc_in_new_TLAB and >>>>>>>>> java/object_alloc_outside_TLAB. >>>>>>>>> What do you think about using two events for PLAB >>>>>>>>> allocations as well: >>>>>>>>> - java/object_alloc_in_new_PLAB >>>>>>>>> - java/object_alloc_outside_PLAB >>>>>>>> I think this is a matter of taste and probably how similar we >>>>>>>> want the >>>>>>>> event to be to the existing allocation event. I personally >>>>>>>> prefer a >>>>>>>> single event but if the GC team and serviceability team feel it >>>>>>>> would be >>>>>>>> better to have two I can certainly rewrite. The reason for me >>>>>>>> preferring >>>>>>>> a single event is just ease of analysis, you can easily filter >>>>>>>> a list of >>>>>>>> events on a field, it is harder to merge two different events with >>>>>>>> different fields and work with them in an straight forward >>>>>>>> fashion. >>>>>>>> >>>>>>>> Any general preference for having a single or multiple events? >>>>>>> >>>>>>> I would prefer to have two events in this case and try to follow >>>>>>> the existing allocation events as much as possible (both in >>>>>>> naming and in style). Keeping the tenured field (I missed it the >>>>>>> first time I read the patch) is good. >>>>>>> >>>>>> Yes, tenured would be independent of having two events, only PLAB >>>>>> size and directAllocation would be affected when having two event >>>>>> types. >>>>>> >>>>>> *Erik Gahlin*, any preference from your end? >>>>>> >>>>>> >>>>>> >>>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>>>> - In PSPromotionManager, instead of utilizing the C++ >>>>>>>>> friendship with >>>>>>>>> PSScavenge, should we add a getter function for the gc_tracer? >>>>>>>> Created a getter function. >>>>>>> >>>>>>> Thanks! If you make report_promotion_sample const, then the >>>>>>> getter can return a const ParallelScavengeTracer*, right? >>>>>>> >>>>>> Done >>>>>> >>>>>> //Staffan >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Thu Nov 6 13:05:48 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Thu, 06 Nov 2014 17:05:48 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B7006.5080104@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> Message-ID: <545B722C.3020406@oracle.com> Hi Katja, Ok, this seems to be a perfect solution. Thank you. I'll change the diff accordingly. Thanks, Evgeniya Stepanova On 06.11.2014 16:56, Yekaterina Kantserova wrote: > Hi Dima, > > On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >> Hi Katja, >> >> You are right, there will be no conflict, because test ignores any >> external VM flags. >> So, adding @requires seems unnecessary here, but... >> >> Ignoring external options is bad thing, such "selfish" tests are not >> applicable for other areas, like GC, compiler, RT. > > This particular case is to test the defined flags are shown up as > expected. > > Evgeniya, > > would you mind to change JpsHelper.java instead? > > +++ b/test/sun/tools/jps/JpsHelper.java > @@ -93,7 +93,7 @@ > /** > * VM arguments to start test application with > */ > - public static final String[] VM_ARGS = {"-Xmx512m", > "-XX:+UseParallelGC"}; > + public static final String[] VM_ARGS = {"-Xmx512m", > "-XX:+PrintGCDetails"}; > /** > * VM flag to start test application with > */ > > Best regards, > Katja > > > >> >> @requires will allow to modify tests to include external vm options >> without any risk of bumping into conflict and extend area of test >> applicability. >> >> But if you still believe, that @requires is not necessary - it's not >> a problem, tests could be kept as is. >> >> Thanks, >> Dima >> >> >> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>> >>> Hi Evgeniya, >>> >>> As David has pointed out these jps tests are not testing gc. The >>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>> should not be any conflicts either since these tests are running in >>> driver mode: >>> >>> ... >>> * @run driver TestJpsJar >>> ... >>> >>> which means no flags from above are accepted. >>> >>> Thanks, >>> Katja >>> >>> >>> >>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>> Hi David, >>>> >>>> tag added because tests contain string >>>> cmd.addAll(JpsHelper.getVmArgs()); >>>> >>>> and JpsHelper defines >>>> ... >>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>> "-XX:+UseParallelGC"}; >>>> ... >>>> public static List getVmArgs() throws IOException { >>>> if (testVmArgs == null) { >>>> testVmArgs = new ArrayList<>(); >>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>> testVmArgs.add("-XX:Flags=" + >>>> getVmFlagsFile().getAbsolutePath()); >>>> } >>>> return testVmArgs; >>>> } >>>> >>>> Tests itself wouldn't fail if we use another GC, tag added for >>>> cleanup-if we use for example SerialGC we must be sure that tests >>>> passed with this GC, not with another one. Now you will assume that >>>> concrete test passed with Serial GC, but it run only with Parallel >>>> GC. Plus there is no any sense to run test twice in TC (with >>>> different GC, since it use only Parallel) >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 06.11.2014 6:20, David Holmes wrote: >>>>> Hi Evgeniya, >>>>> >>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>> Hi, >>>>>> >>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>> JDK-8019361 >>>>>> >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>> >>>>>> Problem: Some tests explicitly set GC and fail when another GC is >>>>>> set >>>>>> outside >>>>> >>>>> I don't see why you have done this for the >>>>> >>>>> test/sun/tools/jps/TestJps*.java >>>>> >>>>> tests. They don't set any GC flags. >>>>> >>>>>> Solution: Such tests marked with the jtreg tag "requires" to skip >>>>>> test >>>>>> if there is a conflict >>>>> >>>>> Just wondering: Does a skipped test get a .jtr file showing it was >>>>> skipped; or does it only appear in the higher-level jtreg log? >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>> without >>>>>> any GC flag). Tests are being excluded as expected. No tests failed >>>>>> because of the conflict. >>>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> >>>>>> // >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >>> >>> >> > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.helin at oracle.com Thu Nov 6 13:51:50 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 06 Nov 2014 14:51:50 +0100 Subject: RFR: 8061467: Bad page size passed to setup_large_pages() on Solaris In-Reply-To: <1415200465.2912.11.camel@oracle.com> References: <544FDC0F.2040805@oracle.com> <1415102977.5218.8.camel@oracle.com> <1415200465.2912.11.camel@oracle.com> Message-ID: <545B7CF6.5010909@oracle.com> Hi Thomas, thanks for handling this! Erik On 2014-11-05 16:14, Thomas Schatzl wrote: > Hi all, > > since Erik is on vacation and this bug prevents a lot of tests from > running, I took over and added a (simple) regression test case. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8061467 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8061467/webrev/ > > Testing: > jprt, Solaris numa machine showing the crash before, and no crash after > the change, local testing, jprt test case runs > > Since the test is extremely simple (class with empty main method), I am > going to keep Erik as the original author of this change as his change > is the one that required the hard work. > > Thanks, > Thomas > > On Tue, 2014-11-04 at 13:09 +0100, Thomas Schatzl wrote: >> Hi Erik, >> >> On Tue, 2014-10-28 at 19:10 +0100, Erik Helin wrote: >>> Hi all, >>> >>> this small patch fixes a call to Solaris::setup_large_pages that does >>> not check if the `alignment_hint` parameter is a valid page size (the >>> only such call left). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~ehelin/8061467/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8061467 >>> >>> Testing: >>> - Verified that the fix solves the issue on a Solaris machine that >>> supported NUMA. >>> - JPRT >> >> looks good. Could you add a simple regression test like the one >> suggested in the CR? >> >> Thanks, >> Thomas >> >> > > From evgeniya.stepanova at oracle.com Thu Nov 6 14:36:39 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Thu, 06 Nov 2014 18:36:39 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B793D.5000008@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> Message-ID: <545B8777.6080403@oracle.com> New webrev: http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ Thanks, Evgeniya Stepanova On 06.11.2014 17:35, Yekaterina Kantserova wrote: > Thanks a lot! > > On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >> Hi Katja, >> >> Ok, this seems to be a perfect solution. Thank you. I'll change the >> diff accordingly. >> >> >> Thanks, >> Evgeniya Stepanova >> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>> Hi Dima, >>> >>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>> Hi Katja, >>>> >>>> You are right, there will be no conflict, because test ignores any >>>> external VM flags. >>>> So, adding @requires seems unnecessary here, but... >>>> >>>> Ignoring external options is bad thing, such "selfish" tests are >>>> not applicable for other areas, like GC, compiler, RT. >>> >>> This particular case is to test the defined flags are shown up as >>> expected. >>> >>> Evgeniya, >>> >>> would you mind to change JpsHelper.java instead? >>> >>> +++ b/test/sun/tools/jps/JpsHelper.java >>> @@ -93,7 +93,7 @@ >>> /** >>> * VM arguments to start test application with >>> */ >>> - public static final String[] VM_ARGS = {"-Xmx512m", >>> "-XX:+UseParallelGC"}; >>> + public static final String[] VM_ARGS = {"-Xmx512m", >>> "-XX:+PrintGCDetails"}; >>> /** >>> * VM flag to start test application with >>> */ >>> >>> Best regards, >>> Katja >>> >>> >>> >>>> >>>> @requires will allow to modify tests to include external vm >>>> options without any risk of bumping into conflict and extend area >>>> of test applicability. >>>> >>>> But if you still believe, that @requires is not necessary - it's >>>> not a problem, tests could be kept as is. >>>> >>>> Thanks, >>>> Dima >>>> >>>> >>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>> >>>>> Hi Evgeniya, >>>>> >>>>> As David has pointed out these jps tests are not testing gc. The >>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>> should not be any conflicts either since these tests are running >>>>> in driver mode: >>>>> >>>>> ... >>>>> * @run driver TestJpsJar >>>>> ... >>>>> >>>>> which means no flags from above are accepted. >>>>> >>>>> Thanks, >>>>> Katja >>>>> >>>>> >>>>> >>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>> Hi David, >>>>>> >>>>>> tag added because tests contain string >>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>> >>>>>> and JpsHelper defines >>>>>> ... >>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>> "-XX:+UseParallelGC"}; >>>>>> ... >>>>>> public static List getVmArgs() throws IOException { >>>>>> if (testVmArgs == null) { >>>>>> testVmArgs = new ArrayList<>(); >>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>> testVmArgs.add("-XX:Flags=" + >>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>> } >>>>>> return testVmArgs; >>>>>> } >>>>>> >>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>> cleanup-if we use for example SerialGC we must be sure that tests >>>>>> passed with this GC, not with another one. Now you will assume >>>>>> that concrete test passed with Serial GC, but it run only with >>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>> (with different GC, since it use only Parallel) >>>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>>> JDK-8019361 >>>>>>>> >>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>> >>>>>>>> Problem: Some tests explicitly set GC and fail when another GC >>>>>>>> is set >>>>>>>> outside >>>>>>> >>>>>>> I don't see why you have done this for the >>>>>>> >>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>> >>>>>>> tests. They don't set any GC flags. >>>>>>> >>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>> skip test >>>>>>>> if there is a conflict >>>>>>> >>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>> was skipped; or does it only appear in the higher-level jtreg log? >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>>> without >>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>> failed >>>>>>>> because of the conflict. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> >>>>>>>> // >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>>> >>>>> >>>>> >>>> >>> >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcus.larsson at oracle.com Thu Nov 6 14:53:14 2014 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Thu, 06 Nov 2014 15:53:14 +0100 Subject: RFR: 8062943: REDO - Parallelize clearing the next mark bitmap Message-ID: <545B8B5A.8080001@oracle.com> Hi, Can I have some reviews for the following re-do patch to parallelize clearing of the next mark bitmap [1]. The previous patch would cause some asserts in HeapRegionManager::par_iterate() to fail because iteration was happening concurrently with humongous heap region reclamation. These asserts are invalid if iteration is performed concurrently. With this updated patch I add a concurrent flag to the par_iterate() function that when set causes iteration to ignore the humongous status of heap regions, and simply iterate over regions one-by-one. An alternative solution would be to just remove the special handling of the humongous regions altogether. Right now there are no closures that depend on this behavior from the par_iterate() function (they either ignore humongous regions or skip over continues-humongous regions). Future changes might however want this behavior, and the asserts are probably helpful in the cases they can be used, so I believe it might be a good idea to keep this functionality. Webrev: http://cr.openjdk.java.net/~mlarsson/8062943/webrev.00/ Incremental webrev from original patch: http://cr.openjdk.java.net/~mlarsson/8062943/webrev.00-inc/ Bug: https://bugs.openjdk.java.net/browse/JDK-8062943 Testing: Local JTReg Thanks, Marcus [1]: https://bugs.openjdk.java.net/browse/JDK-8049341 From filipp.zhinkin at oracle.com Thu Nov 6 14:02:34 2014 From: filipp.zhinkin at oracle.com (Filipp Zhinkin) Date: Thu, 06 Nov 2014 18:02:34 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B8777.6080403@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> Message-ID: <545B7F7A.7070406@oracle.com> Hi Evgeniya, it seems like there is no need to use Serial GC in java/lang/ref/EnqueuePollRace.java [1], so it does make sense to remove that option instead of adding the @requires tag. Thanks, Filipp. [1] https://bugs.openjdk.java.net/browse/JDK-8051723 On 11/06/2014 06:36 PM, Evgeniya Stepanova wrote: > New webrev: > http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ > > Thanks, > Evgeniya Stepanova > On 06.11.2014 17:35, Yekaterina Kantserova wrote: >> Thanks a lot! >> >> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>> Hi Katja, >>> >>> Ok, this seems to be a perfect solution. Thank you. I'll change the diff >>> accordingly. >>> >>> >>> Thanks, >>> Evgeniya Stepanova >>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>> Hi Dima, >>>> >>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>> Hi Katja, >>>>> >>>>> You are right, there will be no conflict, because test ignores any >>>>> external VM flags. >>>>> So, adding @requires seems unnecessary here, but... >>>>> >>>>> Ignoring external options is bad thing, such "selfish" tests are not >>>>> applicable for other areas, like GC, compiler, RT. >>>> >>>> This particular case is to test the defined flags are shown up as expected. >>>> >>>> Evgeniya, >>>> >>>> would you mind to change JpsHelper.java instead? >>>> >>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>> @@ -93,7 +93,7 @@ >>>> /** >>>> * VM arguments to start test application with >>>> */ >>>> - public static final String[] VM_ARGS = {"-Xmx512m", "-XX:+UseParallelGC"}; >>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>> "-XX:+PrintGCDetails"}; >>>> /** >>>> * VM flag to start test application with >>>> */ >>>> >>>> Best regards, >>>> Katja >>>> >>>> >>>> >>>>> >>>>> @requires will allow to modify tests to include external vm options >>>>> without any risk of bumping into conflict and extend area of test >>>>> applicability. >>>>> >>>>> But if you still believe, that @requires is not necessary - it's not a >>>>> problem, tests could be kept as is. >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>> >>>>>> Hi Evgeniya, >>>>>> >>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There should >>>>>> not be any conflicts either since these tests are running in driver mode: >>>>>> >>>>>> ... >>>>>> * @run driver TestJpsJar >>>>>> ... >>>>>> >>>>>> which means no flags from above are accepted. >>>>>> >>>>>> Thanks, >>>>>> Katja >>>>>> >>>>>> >>>>>> >>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> tag added because tests contain string >>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>> >>>>>>> and JpsHelper defines >>>>>>> ... >>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", "-XX:+UseParallelGC"}; >>>>>>> ... >>>>>>> public static List getVmArgs() throws IOException { >>>>>>> if (testVmArgs == null) { >>>>>>> testVmArgs = new ArrayList<>(); >>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>> } >>>>>>> return testVmArgs; >>>>>>> } >>>>>>> >>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>> cleanup-if we use for example SerialGC we must be sure that tests passed >>>>>>> with this GC, not with another one. Now you will assume that concrete >>>>>>> test passed with Serial GC, but it run only with Parallel GC. Plus there >>>>>>> is no any sense to run test twice in TC (with different GC, since it use >>>>>>> only Parallel) >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>>>> JDK-8019361 >>>>>>>>> >>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>> >>>>>>>>> Problem: Some tests explicitly set GC and fail when another GC is set >>>>>>>>> outside >>>>>>>> >>>>>>>> I don't see why you have done this for the >>>>>>>> >>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>> >>>>>>>> tests. They don't set any GC flags. >>>>>>>> >>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to skip test >>>>>>>>> if there is a conflict >>>>>>>> >>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it was >>>>>>>> skipped; or does it only appear in the higher-level jtreg log? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and without >>>>>>>>> any GC flag). Tests are being excluded as expected. No tests failed >>>>>>>>> because of the conflict. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> >>>>>>>>> // >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> -- >>> /Evgeniya Stepanova/ >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From yekaterina.kantserova at oracle.com Thu Nov 6 10:27:24 2014 From: yekaterina.kantserova at oracle.com (Yekaterina Kantserova) Date: Thu, 06 Nov 2014 11:27:24 +0100 Subject: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B47FC.1070305@oracle.com> References: <545A4367.1070407@oracle.com> <545ADAD7.7040902@oracle.com> <545B47FC.1070305@oracle.com> Message-ID: <545B4D0C.2010005@oracle.com> Hi Evgeniya, As David has pointed out these jps tests are not testing gc. The -XX:+UseParallelGC is just an arbitrary chosen test flag. There should not be any conflicts either since these tests are running in driver mode: ... * @run driver TestJpsJar ... which means no flags from above are accepted. Thanks, Katja On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: > Hi David, > > tag added because tests contain string > cmd.addAll(JpsHelper.getVmArgs()); > > and JpsHelper defines > ... > public static final String[] VM_ARGS = {"-Xmx512m", "-XX:+UseParallelGC"}; > ... > public static List getVmArgs() throws IOException { > if (testVmArgs == null) { > testVmArgs = new ArrayList<>(); > testVmArgs.addAll(Arrays.asList(VM_ARGS)); > testVmArgs.add("-XX:Flags=" + > getVmFlagsFile().getAbsolutePath()); > } > return testVmArgs; > } > > Tests itself wouldn't fail if we use another GC, tag added for > cleanup-if we use for example SerialGC we must be sure that tests > passed with this GC, not with another one. Now you will assume that > concrete test passed with Serial GC, but it run only with Parallel GC. > Plus there is no any sense to run test twice in TC (with different GC, > since it use only Parallel) > > Thanks, > Evgeniya Stepanova > On 06.11.2014 6:20, David Holmes wrote: >> Hi Evgeniya, >> >> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>> Hi, >>> >>> Please review changes for 8062536, the OpenJDK/jdk part of the >>> JDK-8019361 >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>> >>> Problem: Some tests explicitly set GC and fail when another GC is set >>> outside >> >> I don't see why you have done this for the >> >> test/sun/tools/jps/TestJps*.java >> >> tests. They don't set any GC flags. >> >>> Solution: Such tests marked with the jtreg tag "requires" to skip test >>> if there is a conflict >> >> Just wondering: Does a skipped test get a .jtr file showing it was >> skipped; or does it only appear in the higher-level jtreg log? >> >> Thanks, >> David >> >>> Tested locally with different GC flags (-XX:+UseG1GC, >>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and without >>> any GC flag). Tests are being excluded as expected. No tests failed >>> because of the conflict. >>> >>> Thanks, >>> Evgeniya Stepanova >>> >>> // > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From yekaterina.kantserova at oracle.com Thu Nov 6 10:55:07 2014 From: yekaterina.kantserova at oracle.com (Yekaterina Kantserova) Date: Thu, 06 Nov 2014 11:55:07 +0100 Subject: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B4D0C.2010005@oracle.com> References: <545A4367.1070407@oracle.com> <545ADAD7.7040902@oracle.com> <545B47FC.1070305@oracle.com> <545B4D0C.2010005@oracle.com> Message-ID: <545B538B.4060403@oracle.com> If it's confusing the tests are using -XX:+UseParallelGC we can change them to use some other neutral flag. "Plus there is no any sense to run test twice in TC (with different GC, since it use only Parallel)" I agree it's meaningless to run them with all possible GC combinations because these tests are GC neutral. // Katja On 11/06/2014 11:27 AM, Yekaterina Kantserova wrote: > Hi Evgeniya, > > As David has pointed out these jps tests are not testing gc. The > -XX:+UseParallelGC is just an arbitrary chosen test flag. There should > not be any conflicts either since these tests are running in driver mode: > > ... > * @run driver TestJpsJar > ... > > which means no flags from above are accepted. > > Thanks, > Katja > > > > On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >> Hi David, >> >> tag added because tests contain string >> cmd.addAll(JpsHelper.getVmArgs()); >> >> and JpsHelper defines >> ... >> public static final String[] VM_ARGS = {"-Xmx512m", >> "-XX:+UseParallelGC"}; >> ... >> public static List getVmArgs() throws IOException { >> if (testVmArgs == null) { >> testVmArgs = new ArrayList<>(); >> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >> testVmArgs.add("-XX:Flags=" + >> getVmFlagsFile().getAbsolutePath()); >> } >> return testVmArgs; >> } >> >> Tests itself wouldn't fail if we use another GC, tag added for >> cleanup-if we use for example SerialGC we must be sure that tests >> passed with this GC, not with another one. Now you will assume that >> concrete test passed with Serial GC, but it run only with Parallel >> GC. Plus there is no any sense to run test twice in TC (with >> different GC, since it use only Parallel) >> >> Thanks, >> Evgeniya Stepanova >> On 06.11.2014 6:20, David Holmes wrote: >>> Hi Evgeniya, >>> >>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>> Hi, >>>> >>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>> JDK-8019361 >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>> >>>> Problem: Some tests explicitly set GC and fail when another GC is set >>>> outside >>> >>> I don't see why you have done this for the >>> >>> test/sun/tools/jps/TestJps*.java >>> >>> tests. They don't set any GC flags. >>> >>>> Solution: Such tests marked with the jtreg tag "requires" to skip test >>>> if there is a conflict >>> >>> Just wondering: Does a skipped test get a .jtr file showing it was >>> skipped; or does it only appear in the higher-level jtreg log? >>> >>> Thanks, >>> David >>> >>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>> without >>>> any GC flag). Tests are being excluded as expected. No tests failed >>>> because of the conflict. >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> >>>> // >> >> -- >> /Evgeniya Stepanova/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yekaterina.kantserova at oracle.com Thu Nov 6 12:56:38 2014 From: yekaterina.kantserova at oracle.com (Yekaterina Kantserova) Date: Thu, 06 Nov 2014 13:56:38 +0100 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B4BF6.4040209@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> Message-ID: <545B7006.5080104@oracle.com> Hi Dima, On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: > Hi Katja, > > You are right, there will be no conflict, because test ignores any > external VM flags. > So, adding @requires seems unnecessary here, but... > > Ignoring external options is bad thing, such "selfish" tests are not > applicable for other areas, like GC, compiler, RT. This particular case is to test the defined flags are shown up as expected. Evgeniya, would you mind to change JpsHelper.java instead? +++ b/test/sun/tools/jps/JpsHelper.java @@ -93,7 +93,7 @@ /** * VM arguments to start test application with */ - public static final String[] VM_ARGS = {"-Xmx512m", "-XX:+UseParallelGC"}; + public static final String[] VM_ARGS = {"-Xmx512m", "-XX:+PrintGCDetails"}; /** * VM flag to start test application with */ Best regards, Katja > > @requires will allow to modify tests to include external vm options > without any risk of bumping into conflict and extend area of test > applicability. > > But if you still believe, that @requires is not necessary - it's not a > problem, tests could be kept as is. > > Thanks, > Dima > > > On 06.11.2014 16:27, Yekaterina Kantserova wrote: >> >> Hi Evgeniya, >> >> As David has pointed out these jps tests are not testing gc. The >> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >> should not be any conflicts either since these tests are running in >> driver mode: >> >> ... >> * @run driver TestJpsJar >> ... >> >> which means no flags from above are accepted. >> >> Thanks, >> Katja >> >> >> >> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>> Hi David, >>> >>> tag added because tests contain string >>> cmd.addAll(JpsHelper.getVmArgs()); >>> >>> and JpsHelper defines >>> ... >>> public static final String[] VM_ARGS = {"-Xmx512m", >>> "-XX:+UseParallelGC"}; >>> ... >>> public static List getVmArgs() throws IOException { >>> if (testVmArgs == null) { >>> testVmArgs = new ArrayList<>(); >>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>> testVmArgs.add("-XX:Flags=" + >>> getVmFlagsFile().getAbsolutePath()); >>> } >>> return testVmArgs; >>> } >>> >>> Tests itself wouldn't fail if we use another GC, tag added for >>> cleanup-if we use for example SerialGC we must be sure that tests >>> passed with this GC, not with another one. Now you will assume that >>> concrete test passed with Serial GC, but it run only with Parallel >>> GC. Plus there is no any sense to run test twice in TC (with >>> different GC, since it use only Parallel) >>> >>> Thanks, >>> Evgeniya Stepanova >>> On 06.11.2014 6:20, David Holmes wrote: >>>> Hi Evgeniya, >>>> >>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>> Hi, >>>>> >>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>> JDK-8019361 >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>> >>>>> Problem: Some tests explicitly set GC and fail when another GC is set >>>>> outside >>>> >>>> I don't see why you have done this for the >>>> >>>> test/sun/tools/jps/TestJps*.java >>>> >>>> tests. They don't set any GC flags. >>>> >>>>> Solution: Such tests marked with the jtreg tag "requires" to skip >>>>> test >>>>> if there is a conflict >>>> >>>> Just wondering: Does a skipped test get a .jtr file showing it was >>>> skipped; or does it only appear in the higher-level jtreg log? >>>> >>>> Thanks, >>>> David >>>> >>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>> without >>>>> any GC flag). Tests are being excluded as expected. No tests failed >>>>> because of the conflict. >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> >>>>> // >>> >>> -- >>> /Evgeniya Stepanova/ >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yekaterina.kantserova at oracle.com Thu Nov 6 13:35:57 2014 From: yekaterina.kantserova at oracle.com (Yekaterina Kantserova) Date: Thu, 06 Nov 2014 14:35:57 +0100 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B722C.3020406@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> Message-ID: <545B793D.5000008@oracle.com> Thanks a lot! On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: > Hi Katja, > > Ok, this seems to be a perfect solution. Thank you. I'll change the > diff accordingly. > > > Thanks, > Evgeniya Stepanova > On 06.11.2014 16:56, Yekaterina Kantserova wrote: >> Hi Dima, >> >> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>> Hi Katja, >>> >>> You are right, there will be no conflict, because test ignores any >>> external VM flags. >>> So, adding @requires seems unnecessary here, but... >>> >>> Ignoring external options is bad thing, such "selfish" tests are not >>> applicable for other areas, like GC, compiler, RT. >> >> This particular case is to test the defined flags are shown up as >> expected. >> >> Evgeniya, >> >> would you mind to change JpsHelper.java instead? >> >> +++ b/test/sun/tools/jps/JpsHelper.java >> @@ -93,7 +93,7 @@ >> /** >> * VM arguments to start test application with >> */ >> - public static final String[] VM_ARGS = {"-Xmx512m", >> "-XX:+UseParallelGC"}; >> + public static final String[] VM_ARGS = {"-Xmx512m", >> "-XX:+PrintGCDetails"}; >> /** >> * VM flag to start test application with >> */ >> >> Best regards, >> Katja >> >> >> >>> >>> @requires will allow to modify tests to include external vm options >>> without any risk of bumping into conflict and extend area of test >>> applicability. >>> >>> But if you still believe, that @requires is not necessary - it's not >>> a problem, tests could be kept as is. >>> >>> Thanks, >>> Dima >>> >>> >>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>> >>>> Hi Evgeniya, >>>> >>>> As David has pointed out these jps tests are not testing gc. The >>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>> should not be any conflicts either since these tests are running in >>>> driver mode: >>>> >>>> ... >>>> * @run driver TestJpsJar >>>> ... >>>> >>>> which means no flags from above are accepted. >>>> >>>> Thanks, >>>> Katja >>>> >>>> >>>> >>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>> Hi David, >>>>> >>>>> tag added because tests contain string >>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>> >>>>> and JpsHelper defines >>>>> ... >>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>> "-XX:+UseParallelGC"}; >>>>> ... >>>>> public static List getVmArgs() throws IOException { >>>>> if (testVmArgs == null) { >>>>> testVmArgs = new ArrayList<>(); >>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>> testVmArgs.add("-XX:Flags=" + >>>>> getVmFlagsFile().getAbsolutePath()); >>>>> } >>>>> return testVmArgs; >>>>> } >>>>> >>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>> cleanup-if we use for example SerialGC we must be sure that tests >>>>> passed with this GC, not with another one. Now you will assume >>>>> that concrete test passed with Serial GC, but it run only with >>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>> (with different GC, since it use only Parallel) >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>> Hi Evgeniya, >>>>>> >>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>> JDK-8019361 >>>>>>> >>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>> >>>>>>> Problem: Some tests explicitly set GC and fail when another GC >>>>>>> is set >>>>>>> outside >>>>>> >>>>>> I don't see why you have done this for the >>>>>> >>>>>> test/sun/tools/jps/TestJps*.java >>>>>> >>>>>> tests. They don't set any GC flags. >>>>>> >>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>> skip test >>>>>>> if there is a conflict >>>>>> >>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>> was skipped; or does it only appear in the higher-level jtreg log? >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>> without >>>>>>> any GC flag). Tests are being excluded as expected. No tests failed >>>>>>> because of the conflict. >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> >>>>>>> // >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>>> >>>> >>> >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Thu Nov 6 20:56:08 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 6 Nov 2014 15:56:08 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <545B38F2.9040806@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> Message-ID: <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> On Nov 6, 2014, at 4:01 AM, Bengt Rutisson wrote: > What does the test verify? When I run the test on a build that does > not have your fix it still passes: > > $ java -showversion -jar /localhome/tests/jtreg/lib/jtreg.jar -noreport test/gc/TestScavengeALot.java > java version "1.9.0-ea" > Java(TM) SE Runtime Environment (build 1.9.0-ea-b35) > Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-ea-b35, mixed mode) > > Test results: passed: 1 > > Also, since this is a CMS specific test I think it should be located > in the "test/gc/concurrentMarkSweep". Without the proposed changes, the test program segfaults during VM initialization when using a *non-product* build and using *either* G1 or CMS as the collector (plus the other options specified in the test's @run line), e.g. the main() for the test program is never reached with those configurations. I may be confused about how to go about testing in this situation, but the jtreg invocation above isn't one of the failing configurations (it appears to be a product build, and the default GC is being used, which isn't either G1 or CMS.) My understanding is that at least some of our test infrastructure runs each test with each of several GC configurations. That doesn't appear to be true for jtreg run directly though; to test different GC configurations I think one needs to use -javaoptions in the jtreg invocation. By leaving the GC unspecified in the @run line, I'm allowing the external invocation to choose which GC to use. It seems at least harmless to run the test for non-concurrent collectors (and has the benefit of some minimal testing of the -XX:+ScavengeALot feature with those collectors too), and we'd like the test to be run for any concurrent collector externally specified. I could instead explicitly specify the collector to use, by providing two @run lines, one for G1 and one for CMS. But then I should add a @requires line, and I don't know how to write that @requires line in the face of multiple @run lines with different requirements. (This seems like a deficiency in the new @requires mechanism.) I think if I were going to explicitly specify the collector, I'd need two copies of the test, each with one @run line for the desired collector, with a corresponding @requires line. (Script-based test trampolines to a common .java test implementation would be an alternative if the .java code was less trivial.) There's no reason to run the test against a product build, and in fact the test would fail in such a configuration were it not for the use of -XX:+IgnoreUnrecognizedVMOptions (yuck!). There doesn't seem to be a better way at present to deal with that case; but see https://bugs.openjdk.java.net/browse/CODETOOLS-7901086 Anyway, that's how I arrived at the proposed test program. I'm happy to fix it if there's some mistake in how I got there. From kim.barrett at oracle.com Thu Nov 6 21:35:30 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 6 Nov 2014 16:35:30 -0500 Subject: RFR: 8062943: REDO - Parallelize clearing the next mark bitmap In-Reply-To: <545B8B5A.8080001@oracle.com> References: <545B8B5A.8080001@oracle.com> Message-ID: On Nov 6, 2014, at 9:53 AM, Marcus Larsson wrote: > > Hi, > > Can I have some reviews for the following re-do patch to parallelize clearing of the next mark bitmap [1]. > > The previous patch would cause some asserts in HeapRegionManager::par_iterate() to fail because iteration was happening concurrently with humongous heap region reclamation. These asserts are invalid if iteration is performed concurrently. > > With this updated patch I add a concurrent flag to the par_iterate() function that when set causes iteration to ignore the humongous status of heap regions, and simply iterate over regions one-by-one. > > An alternative solution would be to just remove the special handling of the humongous regions altogether. Right now there are no closures that depend on this behavior from the par_iterate() function (they either ignore humongous regions or skip over continues-humongous regions). Future changes might however want this behavior, and the asserts are probably helpful in the cases they can be used, so I believe it might be a good idea to keep this functionality. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8062943/webrev.00/ > > Incremental webrev from original patch: > http://cr.openjdk.java.net/~mlarsson/8062943/webrev.00-inc/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8062943 > > Testing: > Local JTReg > > Thanks, > Marcus > > [1]: https://bugs.openjdk.java.net/browse/JDK-8049341 Looks good. From jwha at google.com Fri Nov 7 01:36:09 2014 From: jwha at google.com (Jungwoo Ha) Date: Thu, 6 Nov 2014 17:36:09 -0800 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> Message-ID: > > > > ------------------------------------------------------------------------------ > > > src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp > 271 // Support for CMSFastPromotionFailure > 272 reset_promotion_failed(); > > Why this and not just initialize the data member in the constructor > initializer list? > > Okay. I will fix that. > > ------------------------------------------------------------------------------ > > [All of code discussed here is in par_promote().] > > The tests of CMSFastPromotionFailure && has_promotion_failed() that > are outside the lock have a sort of DCLP (double checked locking > pattern) feel to them. The goal here is to avoid the locked region > and go do something else, instead of waiting for the current lock > holder to complete the work the checker would be doing if only they > could get the lock. > > I think I would like it better if the outside the lock tests were > moved closer to the lock entries, to emphasize that relationship, and > to reduce the code affected by the introduction of this set of > changes. That might result in the thread that gives up doing a little > more work before giving up. I'm particularly thinking of > > 1358 if (CMSFastPromotionFailure && has_promotion_failed()) { > 1359 return NULL; > 1360 } > > removal of which would let execution proceed to > > 1373 if (!expand_and_ensure_spooling_space(promoInfo)) { > 1374 return NULL; > > and give up because the moved test caused expand_xxx to return false. > I don't know if the placement of the test at line 1358 rather than > inside expand_and_ensure_spooling_space() would be significant. I > suspect not, but without measurements I don't have a lot of confidence > in that supposition. But I think that change would improve the > understandability of the code. > > In the other case, > > 1381 if (CMSFastPromotionFailure && has_promotion_failed()) { > 1382 return NULL; > 1383 } > 1384 obj_ptr = expand_and_par_lab_allocate(ps, alloc_sz); > > I would definitely prefer that test be moved into > expand_and_par_lab_allocate(). This is the only call to that > function, and the first thing it does is lock the mutex and re-perform > the test. > These are really important for performance. If we remove line 1358, the pause time with the given example goes up by 2x. I'll move 1381 to inside expand_and_par_lab_allocate(). > > > ------------------------------------------------------------------------------ > > I'm suspicious of the during-review change to make > _has_promotion_failed non-atomic. > > I see the argument for the interaction between set and test; the set > occurs in lock context so gets released on the way out of the lock, > while the first time each core gets a stale false value it will > acquire the lock and acquire the updated set value, so that future > tests on the same core will avoid the code in question, until the > reset finally happens. > > However, I think the present interaction between reset and test is not > safe. A test seeing a stale true value leads to an inappropriate skip > of the protected code, with nothing obvious to prevent that from > occurring "forever". > > > ------------------------------------------------------------------------------ > I am sure the code is correct under x86 as is, but please let me know how to fix it for other architecture. I'll post the update after the discussion is settled. --Jungwoo -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Nov 7 05:37:58 2014 From: david.holmes at oracle.com (David Holmes) Date: Fri, 07 Nov 2014 15:37:58 +1000 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B8777.6080403@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> Message-ID: <545C5AB6.6030103@oracle.com> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: > New webrev: > http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ In: test/java/lang/management/RuntimeMXBean/TestInputArgument.sh the use of the gc options seems incidental - it's just picking two innocuous options to use - similar to the JpsHelper case. You could replace +UseParallelGC with something like +UseFastJNIAccessors (platform independent and normally true). David ----- > Thanks, > Evgeniya Stepanova > On 06.11.2014 17:35, Yekaterina Kantserova wrote: >> Thanks a lot! >> >> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>> Hi Katja, >>> >>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>> diff accordingly. >>> >>> >>> Thanks, >>> Evgeniya Stepanova >>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>> Hi Dima, >>>> >>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>> Hi Katja, >>>>> >>>>> You are right, there will be no conflict, because test ignores any >>>>> external VM flags. >>>>> So, adding @requires seems unnecessary here, but... >>>>> >>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>> not applicable for other areas, like GC, compiler, RT. >>>> >>>> This particular case is to test the defined flags are shown up as >>>> expected. >>>> >>>> Evgeniya, >>>> >>>> would you mind to change JpsHelper.java instead? >>>> >>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>> @@ -93,7 +93,7 @@ >>>> /** >>>> * VM arguments to start test application with >>>> */ >>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>> "-XX:+UseParallelGC"}; >>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>> "-XX:+PrintGCDetails"}; >>>> /** >>>> * VM flag to start test application with >>>> */ >>>> >>>> Best regards, >>>> Katja >>>> >>>> >>>> >>>>> >>>>> @requires will allow to modify tests to include external vm >>>>> options without any risk of bumping into conflict and extend area >>>>> of test applicability. >>>>> >>>>> But if you still believe, that @requires is not necessary - it's >>>>> not a problem, tests could be kept as is. >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>> >>>>>> Hi Evgeniya, >>>>>> >>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>> should not be any conflicts either since these tests are running >>>>>> in driver mode: >>>>>> >>>>>> ... >>>>>> * @run driver TestJpsJar >>>>>> ... >>>>>> >>>>>> which means no flags from above are accepted. >>>>>> >>>>>> Thanks, >>>>>> Katja >>>>>> >>>>>> >>>>>> >>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> tag added because tests contain string >>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>> >>>>>>> and JpsHelper defines >>>>>>> ... >>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>> "-XX:+UseParallelGC"}; >>>>>>> ... >>>>>>> public static List getVmArgs() throws IOException { >>>>>>> if (testVmArgs == null) { >>>>>>> testVmArgs = new ArrayList<>(); >>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>> } >>>>>>> return testVmArgs; >>>>>>> } >>>>>>> >>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>> cleanup-if we use for example SerialGC we must be sure that tests >>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>>> (with different GC, since it use only Parallel) >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>>>> JDK-8019361 >>>>>>>>> >>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>> >>>>>>>>> Problem: Some tests explicitly set GC and fail when another GC >>>>>>>>> is set >>>>>>>>> outside >>>>>>>> >>>>>>>> I don't see why you have done this for the >>>>>>>> >>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>> >>>>>>>> tests. They don't set any GC flags. >>>>>>>> >>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>> skip test >>>>>>>>> if there is a conflict >>>>>>>> >>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>> was skipped; or does it only appear in the higher-level jtreg log? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>>>> without >>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>> failed >>>>>>>>> because of the conflict. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> >>>>>>>>> // >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> -- >>> /Evgeniya Stepanova/ >> > > -- > /Evgeniya Stepanova/ From evgeniya.stepanova at oracle.com Fri Nov 7 07:49:44 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Fri, 07 Nov 2014 11:49:44 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545C5AB6.6030103@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> Message-ID: <545C7998.10900@oracle.com> Hi David! Ok, I'll change the diff. Thank you On 07.11.2014 9:37, David Holmes wrote: > On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >> New webrev: >> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ > > In: > > test/java/lang/management/RuntimeMXBean/TestInputArgument.sh > > the use of the gc options seems incidental - it's just picking two > innocuous options to use - similar to the JpsHelper case. You could > replace +UseParallelGC with something like +UseFastJNIAccessors > (platform independent and normally true). > > David > ----- > >> Thanks, >> Evgeniya Stepanova >> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>> Thanks a lot! >>> >>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>> Hi Katja, >>>> >>>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>>> diff accordingly. >>>> >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>> Hi Dima, >>>>> >>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>> Hi Katja, >>>>>> >>>>>> You are right, there will be no conflict, because test ignores any >>>>>> external VM flags. >>>>>> So, adding @requires seems unnecessary here, but... >>>>>> >>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>> not applicable for other areas, like GC, compiler, RT. >>>>> >>>>> This particular case is to test the defined flags are shown up as >>>>> expected. >>>>> >>>>> Evgeniya, >>>>> >>>>> would you mind to change JpsHelper.java instead? >>>>> >>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>> @@ -93,7 +93,7 @@ >>>>> /** >>>>> * VM arguments to start test application with >>>>> */ >>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>> "-XX:+UseParallelGC"}; >>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>> "-XX:+PrintGCDetails"}; >>>>> /** >>>>> * VM flag to start test application with >>>>> */ >>>>> >>>>> Best regards, >>>>> Katja >>>>> >>>>> >>>>> >>>>>> >>>>>> @requires will allow to modify tests to include external vm >>>>>> options without any risk of bumping into conflict and extend area >>>>>> of test applicability. >>>>>> >>>>>> But if you still believe, that @requires is not necessary - it's >>>>>> not a problem, tests could be kept as is. >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>> >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>> should not be any conflicts either since these tests are running >>>>>>> in driver mode: >>>>>>> >>>>>>> ... >>>>>>> * @run driver TestJpsJar >>>>>>> ... >>>>>>> >>>>>>> which means no flags from above are accepted. >>>>>>> >>>>>>> Thanks, >>>>>>> Katja >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> tag added because tests contain string >>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>> >>>>>>>> and JpsHelper defines >>>>>>>> ... >>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>> ... >>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>> if (testVmArgs == null) { >>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>> } >>>>>>>> return testVmArgs; >>>>>>>> } >>>>>>>> >>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>> cleanup-if we use for example SerialGC we must be sure that tests >>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>>>> (with different GC, since it use only Parallel) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>> Hi Evgeniya, >>>>>>>>> >>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>>>>> JDK-8019361 >>>>>>>>>> >>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>> >>>>>>>>>> Problem: Some tests explicitly set GC and fail when another GC >>>>>>>>>> is set >>>>>>>>>> outside >>>>>>>>> >>>>>>>>> I don't see why you have done this for the >>>>>>>>> >>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>> >>>>>>>>> tests. They don't set any GC flags. >>>>>>>>> >>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>> skip test >>>>>>>>>> if there is a conflict >>>>>>>>> >>>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>>> was skipped; or does it only appear in the higher-level jtreg >>>>>>>>> log? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>>>>> without >>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>> failed >>>>>>>>>> because of the conflict. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> >>>>>>>>>> // >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Fri Nov 7 08:21:15 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Fri, 07 Nov 2014 12:21:15 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545B7F7A.7070406@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545B7F7A.7070406@oracle.com> Message-ID: <545C80FB.3050300@oracle.com> Hi Filipp, Ok, then I'll just remove option :) Thanks On 06.11.2014 18:02, Filipp Zhinkin wrote: > Hi Evgeniya, > > it seems like there is no need to use Serial GC in > java/lang/ref/EnqueuePollRace.java [1], > so it does make sense to remove that option instead of adding the > @requires tag. > > Thanks, > Filipp. > > [1] https://bugs.openjdk.java.net/browse/JDK-8051723 > > On 11/06/2014 06:36 PM, Evgeniya Stepanova wrote: >> New webrev: >> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >> >> Thanks, >> Evgeniya Stepanova >> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>> Thanks a lot! >>> >>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>> Hi Katja, >>>> >>>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>>> diff accordingly. >>>> >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>> Hi Dima, >>>>> >>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>> Hi Katja, >>>>>> >>>>>> You are right, there will be no conflict, because test ignores >>>>>> any external VM flags. >>>>>> So, adding @requires seems unnecessary here, but... >>>>>> >>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>> not applicable for other areas, like GC, compiler, RT. >>>>> >>>>> This particular case is to test the defined flags are shown up as >>>>> expected. >>>>> >>>>> Evgeniya, >>>>> >>>>> would you mind to change JpsHelper.java instead? >>>>> >>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>> @@ -93,7 +93,7 @@ >>>>> /** >>>>> * VM arguments to start test application with >>>>> */ >>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>> "-XX:+UseParallelGC"}; >>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>> "-XX:+PrintGCDetails"}; >>>>> /** >>>>> * VM flag to start test application with >>>>> */ >>>>> >>>>> Best regards, >>>>> Katja >>>>> >>>>> >>>>> >>>>>> >>>>>> @requires will allow to modify tests to include external vm >>>>>> options without any risk of bumping into conflict and extend area >>>>>> of test applicability. >>>>>> >>>>>> But if you still believe, that @requires is not necessary - it's >>>>>> not a problem, tests could be kept as is. >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>> >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>> should not be any conflicts either since these tests are running >>>>>>> in driver mode: >>>>>>> >>>>>>> ... >>>>>>> * @run driver TestJpsJar >>>>>>> ... >>>>>>> >>>>>>> which means no flags from above are accepted. >>>>>>> >>>>>>> Thanks, >>>>>>> Katja >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> tag added because tests contain string >>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>> >>>>>>>> and JpsHelper defines >>>>>>>> ... >>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>> ... >>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>> if (testVmArgs == null) { >>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>> } >>>>>>>> return testVmArgs; >>>>>>>> } >>>>>>>> >>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>> cleanup-if we use for example SerialGC we must be sure that >>>>>>>> tests passed with this GC, not with another one. Now you will >>>>>>>> assume that concrete test passed with Serial GC, but it run >>>>>>>> only with Parallel GC. Plus there is no any sense to run test >>>>>>>> twice in TC (with different GC, since it use only Parallel) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>> Hi Evgeniya, >>>>>>>>> >>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of >>>>>>>>>> the JDK-8019361 >>>>>>>>>> >>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>> >>>>>>>>>> Problem: Some tests explicitly set GC and fail when another >>>>>>>>>> GC is set >>>>>>>>>> outside >>>>>>>>> >>>>>>>>> I don't see why you have done this for the >>>>>>>>> >>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>> >>>>>>>>> tests. They don't set any GC flags. >>>>>>>>> >>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>> skip test >>>>>>>>>> if there is a conflict >>>>>>>>> >>>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>>> was skipped; or does it only appear in the higher-level jtreg >>>>>>>>> log? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>> and without >>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>> failed >>>>>>>>>> because of the conflict. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> >>>>>>>>>> // >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Fri Nov 7 11:34:08 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Fri, 07 Nov 2014 15:34:08 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545C5AB6.6030103@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> Message-ID: <545CAE30.9010708@oracle.com> David, Filipp, Katja Diff have been updated one more time: java/lang/management/RuntimeMXBean/TestInputArgument.sh and test/java/lang/ref/EnqueuePollRace.java have been changed http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ Thanks On 07.11.2014 9:37, David Holmes wrote: > On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >> New webrev: >> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ > > In: > > test/java/lang/management/RuntimeMXBean/TestInputArgument.sh > > the use of the gc options seems incidental - it's just picking two > innocuous options to use - similar to the JpsHelper case. You could > replace +UseParallelGC with something like +UseFastJNIAccessors > (platform independent and normally true). > > David > ----- > >> Thanks, >> Evgeniya Stepanova >> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>> Thanks a lot! >>> >>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>> Hi Katja, >>>> >>>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>>> diff accordingly. >>>> >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>> Hi Dima, >>>>> >>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>> Hi Katja, >>>>>> >>>>>> You are right, there will be no conflict, because test ignores any >>>>>> external VM flags. >>>>>> So, adding @requires seems unnecessary here, but... >>>>>> >>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>> not applicable for other areas, like GC, compiler, RT. >>>>> >>>>> This particular case is to test the defined flags are shown up as >>>>> expected. >>>>> >>>>> Evgeniya, >>>>> >>>>> would you mind to change JpsHelper.java instead? >>>>> >>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>> @@ -93,7 +93,7 @@ >>>>> /** >>>>> * VM arguments to start test application with >>>>> */ >>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>> "-XX:+UseParallelGC"}; >>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>> "-XX:+PrintGCDetails"}; >>>>> /** >>>>> * VM flag to start test application with >>>>> */ >>>>> >>>>> Best regards, >>>>> Katja >>>>> >>>>> >>>>> >>>>>> >>>>>> @requires will allow to modify tests to include external vm >>>>>> options without any risk of bumping into conflict and extend area >>>>>> of test applicability. >>>>>> >>>>>> But if you still believe, that @requires is not necessary - it's >>>>>> not a problem, tests could be kept as is. >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>> >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>> should not be any conflicts either since these tests are running >>>>>>> in driver mode: >>>>>>> >>>>>>> ... >>>>>>> * @run driver TestJpsJar >>>>>>> ... >>>>>>> >>>>>>> which means no flags from above are accepted. >>>>>>> >>>>>>> Thanks, >>>>>>> Katja >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> tag added because tests contain string >>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>> >>>>>>>> and JpsHelper defines >>>>>>>> ... >>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>> ... >>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>> if (testVmArgs == null) { >>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>> } >>>>>>>> return testVmArgs; >>>>>>>> } >>>>>>>> >>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>> cleanup-if we use for example SerialGC we must be sure that tests >>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>>>> (with different GC, since it use only Parallel) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>> Hi Evgeniya, >>>>>>>>> >>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>>>>> JDK-8019361 >>>>>>>>>> >>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>> >>>>>>>>>> Problem: Some tests explicitly set GC and fail when another GC >>>>>>>>>> is set >>>>>>>>>> outside >>>>>>>>> >>>>>>>>> I don't see why you have done this for the >>>>>>>>> >>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>> >>>>>>>>> tests. They don't set any GC flags. >>>>>>>>> >>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>> skip test >>>>>>>>>> if there is a conflict >>>>>>>>> >>>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>>> was skipped; or does it only appear in the higher-level jtreg >>>>>>>>> log? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>>>>> without >>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>> failed >>>>>>>>>> because of the conflict. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> >>>>>>>>>> // >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From filipp.zhinkin at oracle.com Fri Nov 7 10:35:17 2014 From: filipp.zhinkin at oracle.com (Filipp Zhinkin) Date: Fri, 07 Nov 2014 14:35:17 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545CAE30.9010708@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> Message-ID: <545CA065.5030900@oracle.com> Thank you for taking care of it, the fix looks good to me (not a R-reviewer). Thanks, Filipp. On 11/07/2014 03:34 PM, Evgeniya Stepanova wrote: > David, Filipp, Katja > > Diff have been updated one more time: > java/lang/management/RuntimeMXBean/TestInputArgument.sh and > test/java/lang/ref/EnqueuePollRace.java have been changed > http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ > > Thanks > > On 07.11.2014 9:37, David Holmes wrote: >> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>> New webrev: >>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >> >> In: >> >> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >> >> the use of the gc options seems incidental - it's just picking two innocuous >> options to use - similar to the JpsHelper case. You could replace >> +UseParallelGC with something like +UseFastJNIAccessors (platform independent >> and normally true). >> >> David >> ----- >> >>> Thanks, >>> Evgeniya Stepanova >>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>> Thanks a lot! >>>> >>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>> Hi Katja, >>>>> >>>>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>>>> diff accordingly. >>>>> >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>> Hi Dima, >>>>>> >>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>> Hi Katja, >>>>>>> >>>>>>> You are right, there will be no conflict, because test ignores any >>>>>>> external VM flags. >>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>> >>>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>> >>>>>> This particular case is to test the defined flags are shown up as >>>>>> expected. >>>>>> >>>>>> Evgeniya, >>>>>> >>>>>> would you mind to change JpsHelper.java instead? >>>>>> >>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>> @@ -93,7 +93,7 @@ >>>>>> /** >>>>>> * VM arguments to start test application with >>>>>> */ >>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>> "-XX:+UseParallelGC"}; >>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>> "-XX:+PrintGCDetails"}; >>>>>> /** >>>>>> * VM flag to start test application with >>>>>> */ >>>>>> >>>>>> Best regards, >>>>>> Katja >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> @requires will allow to modify tests to include external vm >>>>>>> options without any risk of bumping into conflict and extend area >>>>>>> of test applicability. >>>>>>> >>>>>>> But if you still believe, that @requires is not necessary - it's >>>>>>> not a problem, tests could be kept as is. >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>> >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>>> should not be any conflicts either since these tests are running >>>>>>>> in driver mode: >>>>>>>> >>>>>>>> ... >>>>>>>> * @run driver TestJpsJar >>>>>>>> ... >>>>>>>> >>>>>>>> which means no flags from above are accepted. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Katja >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> tag added because tests contain string >>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>> >>>>>>>>> and JpsHelper defines >>>>>>>>> ... >>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>> ... >>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>> if (testVmArgs == null) { >>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>> } >>>>>>>>> return testVmArgs; >>>>>>>>> } >>>>>>>>> >>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>>> cleanup-if we use for example SerialGC we must be sure that tests >>>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>>>>>> JDK-8019361 >>>>>>>>>>> >>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Problem: Some tests explicitly set GC and fail when another GC >>>>>>>>>>> is set >>>>>>>>>>> outside >>>>>>>>>> >>>>>>>>>> I don't see why you have done this for the >>>>>>>>>> >>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>> >>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>> >>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>> skip test >>>>>>>>>>> if there is a conflict >>>>>>>>>> >>>>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>>>> was skipped; or does it only appear in the higher-level jtreg log? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>>>>>> without >>>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>>> failed >>>>>>>>>>> because of the conflict. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> >>>>>>>>>>> // >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /Evgeniya Stepanova/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>> >>> -- >>> /Evgeniya Stepanova/ > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Fri Nov 7 11:32:25 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 07 Nov 2014 12:32:25 +0100 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> Message-ID: <545CADC9.1010208@oracle.com> Hi Kim and Jungwoo, On 2014-11-07 02:36, Jungwoo Ha wrote: > > > ------------------------------------------------------------------------------ > > src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp > 271 // Support for CMSFastPromotionFailure > 272 reset_promotion_failed(); > > Why this and not just initialize the data member in the constructor > initializer list? > > > Okay. I will fix that. > > ------------------------------------------------------------------------------ > > [All of code discussed here is in par_promote().] > > The tests of CMSFastPromotionFailure && has_promotion_failed() that > are outside the lock have a sort of DCLP (double checked locking > pattern) feel to them. The goal here is to avoid the locked region > and go do something else, instead of waiting for the current lock > holder to complete the work the checker would be doing if only they > could get the lock. > > I think I would like it better if the outside the lock tests were > moved closer to the lock entries, to emphasize that relationship, and > to reduce the code affected by the introduction of this set of > changes. That might result in the thread that gives up doing a little > more work before giving up. I'm particularly thinking of > > 1358 if (CMSFastPromotionFailure && has_promotion_failed()) { > 1359 return NULL; > 1360 } > > removal of which would let execution proceed to > > 1373 if (!expand_and_ensure_spooling_space(promoInfo)) { > 1374 return NULL; > > and give up because the moved test caused expand_xxx to return false. > I don't know if the placement of the test at line 1358 rather than > inside expand_and_ensure_spooling_space() would be significant. I > suspect not, but without measurements I don't have a lot of confidence > in that supposition. But I think that change would improve the > understandability of the code. > > In the other case, > > 1381 if (CMSFastPromotionFailure && has_promotion_failed()) { > 1382 return NULL; > 1383 } > 1384 obj_ptr = expand_and_par_lab_allocate(ps, alloc_sz); > > I would definitely prefer that test be moved into > expand_and_par_lab_allocate(). This is the only call to that > function, and the first thing it does is lock the mutex and re-perform > the test. > > > These are really important for performance. If we remove line 1358, > the pause time with the given example goes up by 2x. > I'll move 1381 to inside expand_and_par_lab_allocate(). > > > ------------------------------------------------------------------------------ > > I'm suspicious of the during-review change to make > _has_promotion_failed non-atomic. > > I see the argument for the interaction between set and test; the set > occurs in lock context so gets released on the way out of the lock, > while the first time each core gets a stale false value it will > acquire the lock and acquire the updated set value, so that future > tests on the same core will avoid the code in question, until the > reset finally happens. > > However, I think the present interaction between reset and test is not > safe. A test seeing a stale true value leads to an inappropriate skip > of the protected code, with nothing obvious to prevent that from > occurring "forever". > > ------------------------------------------------------------------------------ > > > I am sure the code is correct under x86 as is, but please let me know > how to fix it for other architecture. > > I'll post the update after the discussion is settled. I don't think a thread can ever see a stale true value. We only reset after a GC and there is a lot of things happening, including several locks being taken and released, before we start a new GC. So, as far as I can tell this does not require any more barriers. I may be missing something though. If I am, can you give a more specific example, Kim? Thanks, Bengt > > --Jungwoo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Fri Nov 7 12:16:49 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 07 Nov 2014 13:16:49 +0100 Subject: RFR: 8062943: REDO - Parallelize clearing the next mark bitmap In-Reply-To: <545B8B5A.8080001@oracle.com> References: <545B8B5A.8080001@oracle.com> Message-ID: <1415362609.2911.2.camel@oracle.com> Hi, On Thu, 2014-11-06 at 15:53 +0100, Marcus Larsson wrote: > Hi, > > Can I have some reviews for the following re-do patch to parallelize > clearing of the next mark bitmap [1]. > > The previous patch would cause some asserts in > HeapRegionManager::par_iterate() to fail because iteration was happening > concurrently with humongous heap region reclamation. These asserts are > invalid if iteration is performed concurrently. > > With this updated patch I add a concurrent flag to the par_iterate() > function that when set causes iteration to ignore the humongous status > of heap regions, and simply iterate over regions one-by-one. The change looks good. Thanks for updating the comment to the "concurrent" parameter after giving me a first look. Thanks, Thomas From marcus.larsson at oracle.com Fri Nov 7 14:11:25 2014 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 07 Nov 2014 15:11:25 +0100 Subject: RFR: 8062943: REDO - Parallelize clearing the next mark bitmap In-Reply-To: <1415362609.2911.2.camel@oracle.com> References: <545B8B5A.8080001@oracle.com> <1415362609.2911.2.camel@oracle.com> Message-ID: <545CD30D.5020302@oracle.com> Thanks for reviewing, Kim and Thomas! Marcus On 07/11/14 13:16, Thomas Schatzl wrote: > Hi, > > On Thu, 2014-11-06 at 15:53 +0100, Marcus Larsson wrote: >> Hi, >> >> Can I have some reviews for the following re-do patch to parallelize >> clearing of the next mark bitmap [1]. >> >> The previous patch would cause some asserts in >> HeapRegionManager::par_iterate() to fail because iteration was happening >> concurrently with humongous heap region reclamation. These asserts are >> invalid if iteration is performed concurrently. >> >> With this updated patch I add a concurrent flag to the par_iterate() >> function that when set causes iteration to ignore the humongous status >> of heap regions, and simply iterate over regions one-by-one. > The change looks good. Thanks for updating the comment to the > "concurrent" parameter after giving me a first look. > > Thanks, > Thomas > > From marcus.larsson at oracle.com Fri Nov 7 15:32:29 2014 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 07 Nov 2014 16:32:29 +0100 Subject: RFR (XXS): 8064348: Add TraceEvent::is_enabled() for embedded/minimal builds Message-ID: <545CE60D.8070200@oracle.com> Hi, Can I please have reviews for the following small patch to enable usage of TraceEvent::is_enabled() even when INCLUDE_TRACE is not defined? Webrev: http://cr.openjdk.java.net/~mlarsson/8064348/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8064348 Thanks, Marcus From erik.helin at oracle.com Fri Nov 7 15:40:31 2014 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 07 Nov 2014 16:40:31 +0100 Subject: RFR (XXS): 8064348: Add TraceEvent::is_enabled() for embedded/minimal builds In-Reply-To: <545CE60D.8070200@oracle.com> References: <545CE60D.8070200@oracle.com> Message-ID: <545CE7EF.7090303@oracle.com> Hi Markus, looks good, Reviewed. I can sponsor this change. If it is ok for the svc team, I would like to push this via the jdk9/hs-gc/hotspot repo (not via jdk9/hs-rt/hotspot) since we have other upcoming patches depending on this one. Thanks, Erik On 2014-11-07 16:32, Marcus Larsson wrote: > Hi, > > Can I please have reviews for the following small patch to enable usage > of TraceEvent::is_enabled() even when INCLUDE_TRACE is not defined? > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8064348/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8064348 > > Thanks, > Marcus From bengt.rutisson at oracle.com Fri Nov 7 15:42:52 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 07 Nov 2014 16:42:52 +0100 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> Message-ID: <545CE87C.4020200@oracle.com> Hi Kim, On 2014-11-06 21:56, Kim Barrett wrote: > On Nov 6, 2014, at 4:01 AM, Bengt Rutisson wrote: >> What does the test verify? When I run the test on a build that does >> not have your fix it still passes: >> >> $ java -showversion -jar /localhome/tests/jtreg/lib/jtreg.jar -noreport test/gc/TestScavengeALot.java >> java version "1.9.0-ea" >> Java(TM) SE Runtime Environment (build 1.9.0-ea-b35) >> Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-ea-b35, mixed mode) >> >> Test results: passed: 1 >> >> Also, since this is a CMS specific test I think it should be located >> in the "test/gc/concurrentMarkSweep". > Without the proposed changes, the test program segfaults during VM > initialization when using a *non-product* build and using *either* G1 > or CMS as the collector (plus the other options specified in the > test's @run line), e.g. the main() for the test program is never > reached with those configurations. > > I may be confused about how to go about testing in this situation, but > the jtreg invocation above isn't one of the failing configurations > (it appears to be a product build, and the default GC is being used, > which isn't either G1 or CMS.) Yes, you are right. I should have realized that I needed a debug build to use ScavengeALot. Now I get the test to fail the way you described. Thanks! > My understanding is that at least some of our test infrastructure runs > each test with each of several GC configurations. That doesn't appear > to be true for jtreg run directly though; to test different GC > configurations I think one needs to use -javaoptions in the jtreg > invocation. Right. Our nightly testing is run with different GC options in each of the four baselines we have. But this is only true for the testing that is done in on the GC repo...more on this below... > > By leaving the GC unspecified in the @run line, I'm allowing the > external invocation to choose which GC to use. It seems at least > harmless to run the test for non-concurrent collectors (and has the > benefit of some minimal testing of the -XX:+ScavengeALot feature with > those collectors too), and we'd like the test to be run for any > concurrent collector externally specified. > > I could instead explicitly specify the collector to use, by providing > two @run lines, one for G1 and one for CMS. But then I should add a > @requires line, and I don't know how to write that @requires line in > the face of multiple @run lines with different requirements. (This > seems like a deficiency in the new @requires mechanism.) I think if I > were going to explicitly specify the collector, I'd need two copies of > the test, each with one @run line for the desired collector, with a > corresponding @requires line. (Script-based test trampolines to a > common .java test implementation would be an alternative if the .java > code was less trivial.) Yes, I totally agree. The current @requires functionality is not enough for our needs. I pointed this out in the recent review request to add the @requires tag to many GC tests. And I pointed that out when the @requires tag was first suggested about a year ago. Unfortunately this feedback has been left out for some reason. Anyway, I think it would be good to explicitly add the GC to the @run command. Otherwise these tests will be run in many places, nightly testing for the other groups, PIT testing, manual testing, ad hoc aurora runs, et.c. without actually testing anything. Since the test is so simple I think splitting it up into two files is the simplest workaround for the @requires limitation for now. > > There's no reason to run the test against a product build, and in fact > the test would fail in such a configuration were it not for the use of > -XX:+IgnoreUnrecognizedVMOptions (yuck!). There doesn't seem to be a > better way at present to deal with that case; but see > https://bugs.openjdk.java.net/browse/CODETOOLS-7901086 Yes, the IgnoreUnrecognizedVMOptions is a really ugly hack. It should at least take a parameter saying which command line option to ignore if not present. As it is now we run most of our testing with this flag and that hides tests with incorrect command lines. > > Anyway, that's how I arrived at the proposed test program. I'm happy > to fix it if there's some mistake in how I got there. > I think you did a good job with the test. Our infrastructure for selecting how tests are run has lots of room for improvement. Bengt From kim.barrett at oracle.com Fri Nov 7 17:33:12 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Nov 2014 12:33:12 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <545CE87C.4020200@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> Message-ID: <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> On Nov 7, 2014, at 10:42 AM, Bengt Rutisson wrote: > > Yes, I totally agree. The current @requires functionality is not enough for our needs. I pointed this out in the recent review request to add the @requires tag to many GC tests. And I pointed that out when the @requires tag was first suggested about a year ago. Unfortunately this feedback has been left out for some reason. It was by following along with that review that I became aware of the possible pitfalls with @requires, which helped me recognize the difficulties with this particular case. I wonder how common tests with multiple @run lines that might have different requirements are, or could be in the future? That seems like it could happen sufficiently frequently that I think I will file another bug against the @requires feature. > Anyway, I think it would be good to explicitly add the GC to the @run command. Otherwise these tests will be run in many places, nightly testing for the other groups, PIT testing, manual testing, ad hoc aurora runs, et.c. without actually testing anything. > > Since the test is so simple I think splitting it up into two files is the simplest workaround for the @requires limitation for now. Thinking about this more overnight, I was approaching the same conclusion; so that?s what I?ll do. Thanks for your review and helpful discussion. From staffan.larsen at oracle.com Fri Nov 7 19:49:58 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 7 Nov 2014 20:49:58 +0100 Subject: RFR (XXS): 8064348: Add TraceEvent::is_enabled() for embedded/minimal builds In-Reply-To: <545CE7EF.7090303@oracle.com> References: <545CE60D.8070200@oracle.com> <545CE7EF.7090303@oracle.com> Message-ID: > On 7 nov 2014, at 16:40, Erik Helin wrote: > > Hi Markus, > > looks good, Reviewed. I can sponsor this change. > > If it is ok for the svc team, I would like to push this via the jdk9/hs-gc/hotspot repo (not via jdk9/hs-rt/hotspot) since we have other upcoming patches depending on this one. That is ok with me if you do a jprt run with the jdk_jfr tests. /Staffan > > Thanks, > Erik > > On 2014-11-07 16:32, Marcus Larsson wrote: >> Hi, >> >> Can I please have reviews for the following small patch to enable usage >> of TraceEvent::is_enabled() even when INCLUDE_TRACE is not defined? >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8064348/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8064348 >> >> Thanks, >> Marcus From kim.barrett at oracle.com Fri Nov 7 22:47:23 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Nov 2014 17:47:23 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> Message-ID: On Nov 7, 2014, at 12:33 PM, Kim Barrett wrote: > >> Anyway, I think it would be good to explicitly add the GC to the @run command. Otherwise these tests will be run in many places, nightly testing for the other groups, PIT testing, manual testing, ad hoc aurora runs, et.c. without actually testing anything. >> >> Since the test is so simple I think splitting it up into two files is the simplest workaround for the @requires limitation for now. > > Thinking about this more overnight, I was approaching the same conclusion; so that?s what I?ll do. New webrev with test split for G1 and CMS: http://cr.openjdk.java.net/~kbarrett/8062036/webrev03 Note that the CMS test intermittently fails even with these changes, due to https://bugs.openjdk.java.net/browse/JDK-8060467. I'll either need to wait for John Coombs to push his fix for that one (it's already been reviewed), or add a temporary @ignore line to my CMS test. From mandy.chung at oracle.com Fri Nov 7 23:41:50 2014 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 07 Nov 2014 15:41:50 -0800 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545CAE30.9010708@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> Message-ID: <545D58BE.3030404@oracle.com> On 11/7/14 3:34 AM, Evgeniya Stepanova wrote: > > http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ What are the valid values ofvm.gc? It's a bit awk for null as a valid value and I would imagine "*" wildcard may be a better value? Since you are touching test/java/lang/management/** tests, we can remove these shell tests and add multiple @run in the java test instead. It'd be great if you can take this opportunity to remove these shell tests as they are not really necessary. thanks Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Fri Nov 7 23:58:22 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Nov 2014 18:58:22 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> Message-ID: On Nov 7, 2014, at 12:33 PM, Kim Barrett wrote: > > I wonder how common tests with multiple @run lines that might have different requirements are, or could be in the future? That seems like it could happen sufficiently frequently that I think I will file another bug against the @requires feature. https://bugs.openjdk.java.net/browse/CODETOOLS-7901090 From kim.barrett at oracle.com Sat Nov 8 00:03:07 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 7 Nov 2014 19:03:07 -0500 Subject: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545D58BE.3030404@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <545D58BE.3030404@oracle.com> Message-ID: <87912E2F-25C4-43F2-995E-C38A14B1B864@oracle.com> On Nov 7, 2014, at 6:41 PM, Mandy Chung wrote: > > On 11/7/14 3:34 AM, Evgeniya Stepanova wrote: >> >> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ > > What are the valid values of vm.gc? It's a bit awk for null as a valid value and I would imagine "*" wildcard may be a better value? > > Since you are touching test/java/lang/management/** tests, we can remove these shell tests and add multiple @run in the java test instead. It'd be great if you can take this opportunity to remove these shell tests as they are not really necessary. Note that @requires can?t cope with multiple @run lines with differing requirements. I just ran into this with a test that needs to be run against either G1 or CMS. Rather than one test with multiple @run lines, I needed to use multiple tests, each with a single @run line and corresponding @requires constraint. See https://bugs.openjdk.java.net/browse/CODETOOLS-7901090. From jwha at google.com Sat Nov 8 01:41:49 2014 From: jwha at google.com (Jungwoo Ha) Date: Fri, 7 Nov 2014 17:41:49 -0800 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <545CADC9.1010208@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <545CADC9.1010208@oracle.com> Message-ID: http://cr.openjdk.java.net/~rasbold/8061259/webrev.05/ I'll be OOO next week, so I created the up-to-date webrev. Let me know if there is anything else that needs to be done. On Fri, Nov 7, 2014 at 3:32 AM, Bengt Rutisson wrote: > > Hi Kim and Jungwoo, > > > On 2014-11-07 02:36, Jungwoo Ha wrote: > > >> >> ------------------------------------------------------------------------------ >> >> >> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp >> 271 // Support for CMSFastPromotionFailure >> 272 reset_promotion_failed(); >> >> Why this and not just initialize the data member in the constructor >> initializer list? >> >> > Okay. I will fix that. > > >> >> ------------------------------------------------------------------------------ >> >> [All of code discussed here is in par_promote().] >> >> The tests of CMSFastPromotionFailure && has_promotion_failed() that >> are outside the lock have a sort of DCLP (double checked locking >> pattern) feel to them. The goal here is to avoid the locked region >> and go do something else, instead of waiting for the current lock >> holder to complete the work the checker would be doing if only they >> could get the lock. >> >> I think I would like it better if the outside the lock tests were >> moved closer to the lock entries, to emphasize that relationship, and >> to reduce the code affected by the introduction of this set of >> changes. That might result in the thread that gives up doing a little >> more work before giving up. I'm particularly thinking of >> >> 1358 if (CMSFastPromotionFailure && has_promotion_failed()) { >> 1359 return NULL; >> 1360 } >> >> removal of which would let execution proceed to >> >> 1373 if (!expand_and_ensure_spooling_space(promoInfo)) { >> 1374 return NULL; >> >> and give up because the moved test caused expand_xxx to return false. >> I don't know if the placement of the test at line 1358 rather than >> inside expand_and_ensure_spooling_space() would be significant. I >> suspect not, but without measurements I don't have a lot of confidence >> in that supposition. But I think that change would improve the >> understandability of the code. >> >> In the other case, >> >> 1381 if (CMSFastPromotionFailure && has_promotion_failed()) { >> 1382 return NULL; >> 1383 } >> 1384 obj_ptr = expand_and_par_lab_allocate(ps, alloc_sz); >> >> I would definitely prefer that test be moved into >> expand_and_par_lab_allocate(). This is the only call to that >> function, and the first thing it does is lock the mutex and re-perform >> the test. >> > > These are really important for performance. If we remove line 1358, the > pause time with the given example goes up by 2x. > I'll move 1381 to inside expand_and_par_lab_allocate(). > > >> >> >> ------------------------------------------------------------------------------ >> >> I'm suspicious of the during-review change to make >> _has_promotion_failed non-atomic. >> >> I see the argument for the interaction between set and test; the set >> occurs in lock context so gets released on the way out of the lock, >> while the first time each core gets a stale false value it will >> acquire the lock and acquire the updated set value, so that future >> tests on the same core will avoid the code in question, until the >> reset finally happens. >> >> However, I think the present interaction between reset and test is not >> safe. A test seeing a stale true value leads to an inappropriate skip >> of the protected code, with nothing obvious to prevent that from >> occurring "forever". >> >> >> ------------------------------------------------------------------------------ >> > > I am sure the code is correct under x86 as is, but please let me know > how to fix it for other architecture. > > I'll post the update after the discussion is settled. > > > I don't think a thread can ever see a stale true value. We only reset > after a GC and there is a lot of things happening, including several locks > being taken and released, before we start a new GC. So, as far as I can > tell this does not require any more barriers. I may be missing something > though. If I am, can you give a more specific example, Kim? > > Thanks, > Bengt > > > --Jungwoo > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dmitry.Fazunenko at oracle.com Sat Nov 8 08:11:36 2014 From: Dmitry.Fazunenko at oracle.com (Dmitry Fazunenko) Date: Sat, 08 Nov 2014 11:11:36 +0300 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545D58BE.3030404@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <545D58BE.3030404@oracle.com> Message-ID: <545DD038.7030108@oracle.com> Hi Mandy, On 08.11.2014 02:41, Mandy Chung wrote: > On 11/7/14 3:34 AM, Evgeniya Stepanova wrote: >> >> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ > > What are the valid values ofvm.gc? It's a bit awk for null as a valid > value and I would imagine "*" wildcard may be a better value? vm.gc variable is set by jtreg based on passed -XX:+Use???GC command line vm flag. If such flag is not given vm.gc will be set to null (or "null" which is same). So, vm.gc could be set to any value. In reality the following strings are expected: null, "Parallel", "Serial", "G1", "ConcMarkSweep". null means "not set", it doesn't mean "any". If a test is applicable for all GC - you don't need @requires vm.gc == "*" > > Since you are touching test/java/lang/management/** tests, we can > remove these shell tests and add multiple @run in the java test > instead. It'd be great if you can take this opportunity to remove > these shell tests as they are not really necessary. Yes, agree, shell is not necessary here. But such refactoring is better to be done as part of another RFE. Thanks, Dima > > thanks > Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From kirk at kodewerk.com Sat Nov 8 15:01:34 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Sat, 8 Nov 2014 16:01:34 +0100 Subject: more funny log entries Message-ID: Hi all, I found this? young with initial-mark followed by a Full GC in the middle of a complete concurrent mark cycle? Looks like the Full GC didn?t interrupt the concurrent mark cycle. Is that a reasonable assumption? java -version 1.7.0_51-b13 running on OSX (Mavericks) Regards, Kirk 100081.531: [GC pause (young) (initial-mark)100081.540: [SoftReference, 0 refs, 0.0000300 secs]100081.540: [WeakReference, 0 refs, 0.0000030 secs]100081.540: [FinalReference, 0 refs, 0.0000020 secs]100081.540: [PhantomReference, 0 refs, 0.0000030 secs]100081.540: [JNI Weak Reference, 0.0000030 secs], 0.0084910 secs] [Parallel Time: 6.0 ms, GC Workers: 8] [GC Worker Start (ms): Min: 100081533.5, Avg: 100081533.6, Max: 100081533.7, Diff: 0.2] [Ext Root Scanning (ms): Min: 0.1, Avg: 0.5, Max: 1.3, Diff: 1.2, Sum: 3.8] [Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0] [Processed Buffers: Min: 0, Avg: 0.1, Max: 1, Diff: 1, Sum: 1] [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0] [Object Copy (ms): Min: 3.5, Avg: 5.0, Max: 5.9, Diff: 2.4, Sum: 39.7] [Termination (ms): Min: 0.0, Avg: 0.5, Max: 1.4, Diff: 1.4, Sum: 4.1] [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0] [GC Worker Total (ms): Min: 5.9, Avg: 6.0, Max: 6.0, Diff: 0.2, Sum: 47.6] [GC Worker End (ms): Min: 100081539.5, Avg: 100081539.5, Max: 100081539.6, Diff: 0.0] [Code Root Fixup: 0.0 ms] [Clear CT: 0.1 ms] [Other: 2.4 ms] [Choose CSet: 0.0 ms] [Ref Proc: 0.2 ms] [Ref Enq: 0.0 ms] [Free CSet: 0.0 ms] [Eden: 0.0B(25.0M)->0.0B(25.0M) Survivors: 0.0B->0.0B Heap: 510.9M(512.0M)->510.9M(512.0M)] [Times: user=0.01 sys=0.01, real=0.00 secs] 100081.540: [Full GC100081.540: [GC concurrent-root-region-scan-start] 100081.540: [GC concurrent-root-region-scan-end, 0.0000080 secs] 100081.540: [GC concurrent-mark-start] 100084.860: [SoftReference, 22 refs, 0.0000340 secs]100084.860: [WeakReference, 169 refs, 0.0000180 secs]100084.860: [FinalReference, 18 refs, 0.0000050 secs]100084.860: [PhantomReference, 0 refs, 0.0000040 secs]100084.860: [JNI Weak Reference, 0.0000040 secs] 510M->510M(512M), 4.3335850 secs] [Eden: 0.0B(25.0M)->0.0B(25.0M) Survivors: 0.0B->0.0B Heap: 510.9M(512.0M)->510.9M(512.0M)] [Times: user=3.44 sys=1.62, real=4.34 secs] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From kim.barrett at oracle.com Mon Nov 10 06:09:06 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Nov 2014 01:09:06 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> Message-ID: <6F524023-4B70-41FC-9245-FED269599168@oracle.com> On Nov 7, 2014, at 5:47 PM, Kim Barrett wrote: > > On Nov 7, 2014, at 12:33 PM, Kim Barrett wrote: >> >>> Anyway, I think it would be good to explicitly add the GC to the @run command. Otherwise these tests will be run in many places, nightly testing for the other groups, PIT testing, manual testing, ad hoc aurora runs, et.c. without actually testing anything. >>> >>> Since the test is so simple I think splitting it up into two files is the simplest workaround for the @requires limitation for now. >> >> Thinking about this more overnight, I was approaching the same conclusion; so that?s what I?ll do. > > New webrev with test split for G1 and CMS: > > http://cr.openjdk.java.net/~kbarrett/8062036/webrev03 > > Note that the CMS test intermittently fails even with these changes, due > to https://bugs.openjdk.java.net/browse/JDK-8060467. > > I'll either need to wait for John Coombs to push his fix for that one > (it's already been reviewed), or add a temporary @ignore line to my > CMS test. It turns out the reference to JDK-8060467 from JDK-8062036 was to the wrong bug; it should have been JDK-8060463. So John?s fix for the former doesn?t address the misbehavior of the new test being added for JDK-8062036. I didn?t notice this previously because I didn?t carefully check the failure with CMS that I was seeing against the referenced bug until it still failed in the same way after incorporating John?s fix into my forest. So I?ll be adding ?@ignore 8060463? to the CMS test in the above webrev. From marcus.larsson at oracle.com Mon Nov 10 08:37:51 2014 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 10 Nov 2014 09:37:51 +0100 Subject: RFR (XXS): 8064348: Add TraceEvent::is_enabled() for embedded/minimal builds In-Reply-To: References: <545CE60D.8070200@oracle.com> <545CE7EF.7090303@oracle.com> Message-ID: <5460795F.8090106@oracle.com> Hi, I somehow managed to add the const qualifier to this method although it shouldn't be there. Updated webrev (without the const): http://cr.openjdk.java.net/~mlarsson/8064348/webrev.01/ Thanks, Marcus On 07/11/14 20:49, Staffan Larsen wrote: >> On 7 nov 2014, at 16:40, Erik Helin wrote: >> >> Hi Markus, >> >> looks good, Reviewed. I can sponsor this change. >> >> If it is ok for the svc team, I would like to push this via the jdk9/hs-gc/hotspot repo (not via jdk9/hs-rt/hotspot) since we have other upcoming patches depending on this one. > That is ok with me if you do a jprt run with the jdk_jfr tests. > > /Staffan > >> Thanks, >> Erik >> >> On 2014-11-07 16:32, Marcus Larsson wrote: >>> Hi, >>> >>> Can I please have reviews for the following small patch to enable usage >>> of TraceEvent::is_enabled() even when INCLUDE_TRACE is not defined? >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8064348/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8064348 >>> >>> Thanks, >>> Marcus From andrey.x.zakharov at oracle.com Mon Nov 10 10:36:29 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Mon, 10 Nov 2014 14:36:29 +0400 Subject: [RFR] 8016752: Test case for 8005857/8016740 (assert in GC_locker from PSOldGen::expand with -XX:+PrintGCDetails and Verbose) In-Reply-To: <1415094901.5218.5.camel@oracle.com> References: <54476AD3.3050907@oracle.com> <1415094901.5218.5.camel@oracle.com> Message-ID: <5460952D.4070101@oracle.com> Hi, Thomas Yes, its fails very stable on 7u40b31 # Internal Error (/HUDSON/workspace/jdk7u40-2-build-linux-amd64-product/jdk7u40/hotspot/src/share/vm/memory/gcLocker.hpp:94), pid=15099, tid=139928946063104 # assert(_needs_gc || SafepointSynchronize::is_at_safepoint()) failed: only read at safepoint with elapsed time (seconds): 1.273 on local linux x64 box. And passed on b32. Also, there are batches in comments of bugs for builds before fix and with fix. On 04.11.2014 13:55, Thomas Schatzl wrote: > Hi Andrey, > > On Wed, 2014-10-22 at 12:29 +0400, Andrey Zakharov wrote: >> Hi, team. >> Here is simple test for JVM don't crash when its with PSOldGen, low >> memory and with -XX:+PrintGCDetails and -XX:+Verbose flags >> (https://bugs.openjdk.java.net/browse/JDK-8016740) based on Crasher >> attached to original incident bug >> >> webrev: >> http://cr.openjdk.java.net/~azakharov/8016752/webrev/ > The test looks okay, but did you check that if you revert JDK-8016740, > the test actually fails? > The original reproducer was stressing the GC a lot more as it tried to > cause crashes for 60 seconds. > > If that is the case, it looks okay to me. > > Thanks, > Thomas > > From bengt.rutisson at oracle.com Mon Nov 10 12:57:19 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 10 Nov 2014 13:57:19 +0100 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <6F524023-4B70-41FC-9245-FED269599168@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> <6F524023-4B70-41FC-9245-FED269599168@oracle.com> Message-ID: <5460B62F.1010307@oracle.com> Hi Kim, On 2014-11-10 07:09, Kim Barrett wrote: > On Nov 7, 2014, at 5:47 PM, Kim Barrett wrote: >> On Nov 7, 2014, at 12:33 PM, Kim Barrett wrote: >>>> Anyway, I think it would be good to explicitly add the GC to the @run command. Otherwise these tests will be run in many places, nightly testing for the other groups, PIT testing, manual testing, ad hoc aurora runs, et.c. without actually testing anything. >>>> >>>> Since the test is so simple I think splitting it up into two files is the simplest workaround for the @requires limitation for now. >>> Thinking about this more overnight, I was approaching the same conclusion; so that?s what I?ll do. >> New webrev with test split for G1 and CMS: >> >> http://cr.openjdk.java.net/~kbarrett/8062036/webrev03 >> >> Note that the CMS test intermittently fails even with these changes, due >> to https://bugs.openjdk.java.net/browse/JDK-8060467. >> >> I'll either need to wait for John Coombs to push his fix for that one >> (it's already been reviewed), or add a temporary @ignore line to my >> CMS test. > It turns out the reference to JDK-8060467 from JDK-8062036 was to the wrong bug; it should have been JDK-8060463. So John?s fix for the former doesn?t address the misbehavior of the new test being added for JDK-8062036. I didn?t notice this previously because I didn?t carefully check the failure with CMS that I was seeing against the referenced bug until it still failed in the same way after incorporating John?s fix into my forest. So I?ll be adding ?@ignore 8060463? to the CMS test in the above webrev. Thanks for splitting the tests up. They look fine to me now. Except that now that they are specialized I guess they could be moved into the test/gc/cms and test/gc/g1 folders. Is the -XX:SurvivorAlignmentInBytes=2k really required for the test to fail without the patch? In that case I think it is not enough to ignore the test since I *think* Jon's idea to fix JDK-8060463 is to limit what values are allowed for SurvivorAlignmentInBytes. Jon would know more about that. Bengt > > From bengt.rutisson at oracle.com Mon Nov 10 17:28:56 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 10 Nov 2014 18:28:56 +0100 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545DD038.7030108@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <545D58BE.3030404@oracle.com> <545DD038.7030108@oracle.com> Message-ID: <5460F5D8.1070609@oracle.com> On 2014-11-08 09:11, Dmitry Fazunenko wrote: > Hi Mandy, > > On 08.11.2014 02:41, Mandy Chung wrote: >> On 11/7/14 3:34 AM, Evgeniya Stepanova wrote: >>> >>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >> >> What are the valid values ofvm.gc? It's a bit awk for null as a >> valid value and I would imagine "*" wildcard may be a better value? > > vm.gc variable is set by jtreg based on passed -XX:+Use???GC command > line vm flag. > If such flag is not given vm.gc will be set to null (or "null" which > is same). > > So, vm.gc could be set to any value. In reality the following strings > are expected: null, "Parallel", "Serial", "G1", "ConcMarkSweep". > > null means "not set", it doesn't mean "any". > If a test is applicable for all GC - you don't need @requires vm.gc == "*" Note that the vm.gc abstraction is not really flexible enough for our purposes. The way to specify a GC is constantly evolving and it is not suitable to have this logic in the JTreg harness. I've filed this bug about the issue: https://bugs.openjdk.java.net/browse/CODETOOLS-7901091 Bengt > >> >> Since you are touching test/java/lang/management/** tests, we can >> remove these shell tests and add multiple @run in the java test >> instead. It'd be great if you can take this opportunity to remove >> these shell tests as they are not really necessary. > > Yes, agree, shell is not necessary here. But such refactoring is > better to be done as part of another RFE. > > Thanks, > Dima > > >> >> thanks >> Mandy > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dmitry.Fazunenko at oracle.com Mon Nov 10 21:14:59 2014 From: Dmitry.Fazunenko at oracle.com (Dmitry Fazunenko) Date: Tue, 11 Nov 2014 00:14:59 +0300 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <5460F5D8.1070609@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <545D58BE.3030404@oracle.com> <545DD038.7030108@oracle.com> <5460F5D8.1070609@oracle.com> Message-ID: <54612AD3.7040904@oracle.com> On 10.11.2014 20:28, Bengt Rutisson wrote: > > On 2014-11-08 09:11, Dmitry Fazunenko wrote: >> Hi Mandy, >> >> On 08.11.2014 02:41, Mandy Chung wrote: >>> On 11/7/14 3:34 AM, Evgeniya Stepanova wrote: >>>> >>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >>> >>> What are the valid values ofvm.gc? It's a bit awk for null as a >>> valid value and I would imagine "*" wildcard may be a better value? >> >> vm.gc variable is set by jtreg based on passed -XX:+Use???GC command >> line vm flag. >> If such flag is not given vm.gc will be set to null (or "null" which >> is same). >> >> So, vm.gc could be set to any value. In reality the following strings >> are expected: null, "Parallel", "Serial", "G1", "ConcMarkSweep". >> >> null means "not set", it doesn't mean "any". >> If a test is applicable for all GC - you don't need @requires vm.gc >> == "*" > > Note that the vm.gc abstraction is not really flexible enough for our > purposes. The way to specify a GC is constantly evolving and it is not > suitable to have this logic in the JTreg harness. I've filed this bug > about the issue: > > https://bugs.openjdk.java.net/browse/CODETOOLS-7901091 You're absolutely right! @requires vm.gc has limited use. Using VM specific class to decide if a test applicable or not will be much more flexible for us. I'm not sure that this approach is applicable for jtreg... But let's see -- Thanks, Dima > > Bengt > >> >>> >>> Since you are touching test/java/lang/management/** tests, we can >>> remove these shell tests and add multiple @run in the java test >>> instead. It'd be great if you can take this opportunity to remove >>> these shell tests as they are not really necessary. >> >> Yes, agree, shell is not necessary here. But such refactoring is >> better to be done as part of another RFE. >> >> Thanks, >> Dima >> >> >>> >>> thanks >>> Mandy >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Nov 10 21:26:49 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Nov 2014 16:26:49 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <5460B62F.1010307@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> <6F524023-4B70-41FC-9245-FED269599168@oracle.com> <5460B62F.1010307@oracle.com> Message-ID: On Nov 10, 2014, at 7:57 AM, Bengt Rutisson wrote: > > Thanks for splitting the tests up. They look fine to me now. Except that now that they are specialized I guess they could be moved into the test/gc/cms and test/gc/g1 folders. I thought about that, but felt that keeping the two closely related tests close together was better than separating them into the respective GC folders. > Is the -XX:SurvivorAlignmentInBytes=2k really required for the test to fail without the patch? In that case I think it is not enough to ignore the test since I *think* Jon's idea to fix JDK-8060463 is to limit what values are allowed for SurvivorAlignmentInBytes. Jon would know more about that. I talked to Jon, and he confirmed that limiting the permitted values for SurvivorAlignmentInBytes to something sane was the intended approach. So that is indeed a problem for my regression test. That test was cribbed from the 8062036 reproducer. I have not yet been able to find an alternative trigger, though searching for one has led to some interesting spelunking. Of course, that doesn't mean there isn't an alternative; the number of options and their possible interactions gives a pretty large search space. To trigger for G1 we need to somehow cause the concurrent mark thread to perform a GC remark fairly early in VM initialization. My attempts to create that situation have so far resulted in being too late to hit the problematic window, or blowing up for other reasons (such as heap size just being too small). Much as I think tests are a good thing, I'm getting to the point of wondering whether a regression test for this is worth the effort. From derek.white at oracle.com Mon Nov 10 22:43:47 2014 From: derek.white at oracle.com (Derek White) Date: Mon, 10 Nov 2014 17:43:47 -0500 Subject: Questions on bug JDK-8061611 "Remove deprecated command line flags" Message-ID: <54613FA3.8090700@oracle.com> I'm tackling a "starter bug" JDK-8061611 " Remove deprecated command line flags" that sounds like it should be pretty trivial, but some part of this reveal some unexpectedly twisted behavior. I guess that makes it the perfect starter GC bug :-) One of the command line flags to remove is -XX:CMSParPromoteBlocksToClaim. It has been deprecated for a while. Using it prints the message "Please use -XX:OldPLABSize in place of -XX:CMSParPromoteBlocksToClaim in the future" Two little twists: 1) The variable CMSParPromoteBlocksToClaim is still used separately from OldPLABSize when initializing the "_blocks_to_claim" array in compactibleFreeListSpace.cpp. It looks like it uses it when calculating the initial averages for AdaptiveWeightedAverage. Is it important to retain the ability to set the variable CMSParPromoteBlocksToClaim separately from OldPLABSize? And see last question***. 2) In the CMS case, there is code that attempts to synchronize the OldPLABSize and CMSParPromoteBlocksToClaim variables in Arguments::set_cms_and_parnew_gc_flags(). If OldPLABSize is set (via command line or ergo), then CMSParPromoteBlocksToClaim is set to the same value unless it is already set too. Then in any case OldPLABSize is set to CMSParPromoteBlocksToClaim. But note the default values: OldPLABSize = 1024 CMSParPromoteBlocksToClaim = 16 So if both OldPLABSize and CMSParPromoteBlocksToClaim were left at default values, then OldPLABSize just got set to 16, which seems way too small to be useful. Not only is the OldPLABSize default 1024, but in set_parnew_gc_flags() we just reset the value to 1024 (again). So this seems pretty intentional, and setting to 16 seems accidental.This sounds bad??? $ java -XX:+UseConcMarkSweepGC -XX:+PrintFlagsFinal -version | grep OldPLABSize uintx OldPLABSize := 16 {product} In fact, I don't see why the "Size of old gen promotion LAB's (in HeapWords)" should be equal to "Number of blocks to attempt to claim when refilling CMS LAB". *** Aren't these different units? Thanks for any thoughts, - Derek -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Nov 10 23:04:49 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Nov 2014 18:04:49 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> Message-ID: <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> On Nov 6, 2014, at 8:36 PM, Jungwoo Ha wrote: > These are really important for performance. If we remove line 1358, > the pause time with the given example goes up by 2x. I wasn't suggesting eliminating the check at line 1358 altogether; rather, I was suggesting moving it into expand_and_ensure_spooling_space(), right before the mutex locking that begins that function. There's another reason to question the check at line 1358 though, and suggest it should be moved closer to (or inside of) expand_and_ensure_spooling_space(). It looks like the two places that are calling set_promotion_failure() are doing so for different reasons. The check at line 1358 will short circuit all further par_promote() calls in the collection cycle, even those for which allocation would have succeeded if not for that check. For example, in the old code, failure to promote some largish object will not necessarily block later promotion attempts for other, smaller objects. The proposed change modifies that behavior. Since the calling code seems to be going to some effort to deal with par_promote() returning NULL (indicating promotion failure) and keep going, it's not clear to me whether this behavioral change is desirable. And if it *is* desirable, it seems like a better place to do the short circuiting might be higher up the call chain, and possibly simplifying the handling of par_promote() returning NULL. Note that I've not yet investigated where such a higher point in the call chain might be. OTOH, it seems plausible to me that keeping the test at line 1358 as is (e.g. don't move it, which assumes we're ok with the behavioral change discussed above) makes the test that was at 1381 (and which I suggested moving into expand_and_par_allocate()) pretty much redundant. It would be interesting to know whether that check at 1381 fires more than extremely rarely. Of course, this alternative would be contrary to my desire to have the racy flag access located as close as possible to the lock and recheck code. From kim.barrett at oracle.com Mon Nov 10 23:45:30 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 10 Nov 2014 18:45:30 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> Message-ID: <2C55A99D-C39A-4022-958B-48705FBAC93A@oracle.com> On Nov 10, 2014, at 6:04 PM, Kim Barrett wrote: > > The proposed change modifies that behavior. Since the calling code > seems to be going to some effort to deal with par_promote() returning > NULL (indicating promotion failure) and keep going, it's not clear to > me whether this behavioral change is desirable. And if it *is* > desirable, it seems like a better place to do the short circuiting > might be higher up the call chain, and possibly simplifying the > handling of par_promote() returning NULL. Note that I've not yet > investigated where such a higher point in the call chain might be. If the behavioral change associated with the check at line 1358 *is* desirable, then it looks to me as if it could / should be hoisted out into the callers of par_promote(), where there is already a _promotion_failed flag with associated accessor. I think that existing flag tracks the same state as the proposed new flag. So the callers of par_promote() could be responsible for suppressing those calls based on the existing flag, rather than introducing a new flag. From stefan.karlsson at oracle.com Tue Nov 11 12:58:04 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 11 Nov 2014 13:58:04 +0100 Subject: [8u40] RFR: 8064556: G1: ParallelGCThreads=0 may cause assert(!MetadataOnStackMark::has_buffer_for_thread(Thread::current())) failed: Should be empty Message-ID: <546207DC.8060700@oracle.com> Hi all, Please review this patch to weaken the failing assert: http://cr.openjdk.java.net/~stefank/8064556/webrev.01 https://bugs.openjdk.java.net/browse/JDK-8064556 The asserts in pre_work_verification and post_work_verification were put in place to make sure that the worker threads entered and exited with clean slates. However, in JDK 8 this code path can be taken by the VMThread, which will have registered Metadata when running the MetadataOnStackMark constructor in the unloading phase of ConcurrentMark::weakRefsWork. This is different from JDK 9, which always use worker threads to execute this path. The patch has been verified with the failing test that found this issue. thanks, StefanK From mikael.gerdin at oracle.com Tue Nov 11 13:15:33 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 11 Nov 2014 14:15:33 +0100 Subject: [8u40] RFR: 8064556: G1: ParallelGCThreads=0 may cause assert(!MetadataOnStackMark::has_buffer_for_thread(Thread::current())) failed: Should be empty In-Reply-To: <546207DC.8060700@oracle.com> References: <546207DC.8060700@oracle.com> Message-ID: <54620BF5.4090104@oracle.com> Hi Stefan, On 2014-11-11 13:58, Stefan Karlsson wrote: > Hi all, > > Please review this patch to weaken the failing assert: > http://cr.openjdk.java.net/~stefank/8064556/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8064556 The change looks good. /Mikael > > The asserts in pre_work_verification and post_work_verification were put > in place to make sure that the worker threads entered and exited with > clean slates. However, in JDK 8 this code path can be taken by the > VMThread, which will have registered Metadata when running the > MetadataOnStackMark constructor in the unloading phase of > ConcurrentMark::weakRefsWork. This is different from JDK 9, which always > use worker threads to execute this path. > > The patch has been verified with the failing test that found this issue. > > thanks, > StefanK > From stefan.karlsson at oracle.com Tue Nov 11 13:03:46 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 11 Nov 2014 14:03:46 +0100 Subject: [8u40] RFR: 8064556: G1: ParallelGCThreads=0 may cause assert(!MetadataOnStackMark::has_buffer_for_thread(Thread::current())) failed: Should be empty In-Reply-To: <54620BF5.4090104@oracle.com> References: <546207DC.8060700@oracle.com> <54620BF5.4090104@oracle.com> Message-ID: <54620932.1000003@oracle.com> On 2014-11-11 14:15, Mikael Gerdin wrote: > Hi Stefan, > > On 2014-11-11 13:58, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to weaken the failing assert: >> http://cr.openjdk.java.net/~stefank/8064556/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8064556 > > The change looks good. Thanks, Mikael. StefanK > > /Mikael > >> >> The asserts in pre_work_verification and post_work_verification were put >> in place to make sure that the worker threads entered and exited with >> clean slates. However, in JDK 8 this code path can be taken by the >> VMThread, which will have registered Metadata when running the >> MetadataOnStackMark constructor in the unloading phase of >> ConcurrentMark::weakRefsWork. This is different from JDK 9, which always >> use worker threads to execute this path. >> >> The patch has been verified with the failing test that found this issue. >> >> thanks, >> StefanK >> From bengt.rutisson at oracle.com Tue Nov 11 13:09:48 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 11 Nov 2014 14:09:48 +0100 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> <6F524023-4B70-41FC-9245-FED269599168@oracle.com> <5460B62F.1010307@oracle.com> Message-ID: <54620A9C.2060806@oracle.com> Hi Kim, On 11/10/14 10:26 PM, Kim Barrett wrote: > On Nov 10, 2014, at 7:57 AM, Bengt Rutisson wrote: >> Thanks for splitting the tests up. They look fine to me now. Except that now that they are specialized I guess they could be moved into the test/gc/cms and test/gc/g1 folders. > I thought about that, but felt that keeping the two closely related tests close together was better than separating them into the respective GC folders. > >> Is the -XX:SurvivorAlignmentInBytes=2k really required for the test to fail without the patch? In that case I think it is not enough to ignore the test since I *think* Jon's idea to fix JDK-8060463 is to limit what values are allowed for SurvivorAlignmentInBytes. Jon would know more about that. > I talked to Jon, and he confirmed that limiting the permitted values > for SurvivorAlignmentInBytes to something sane was the intended > approach. So that is indeed a problem for my regression test. That > test was cribbed from the 8062036 reproducer. > > I have not yet been able to find an alternative trigger, though > searching for one has led to some interesting spelunking. Of course, > that doesn't mean there isn't an alternative; the number of options > and their possible interactions gives a pretty large search space. > > To trigger for G1 we need to somehow cause the concurrent mark thread > to perform a GC remark fairly early in VM initialization. My attempts > to create that situation have so far resulted in being too late to hit > the problematic window, or blowing up for other reasons (such as heap > size just being too small). > > Much as I think tests are a good thing, I'm getting to the point of > wondering whether a regression test for this is worth the effort. Yes, I agree with you. We should probably just tag the bug with noreg-hard and leave it for now. That also relieves us of the burden to decide on which directory to put the tests in. ;) Thanks for trying hard to fix the tests. Bengt > From thomas.schatzl at oracle.com Tue Nov 11 15:45:55 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 11 Nov 2014 16:45:55 +0100 Subject: [8u40] RFR: 8064556: G1: ParallelGCThreads=0 may cause assert(!MetadataOnStackMark::has_buffer_for_thread(Thread::current())) failed: Should be empty In-Reply-To: <546207DC.8060700@oracle.com> References: <546207DC.8060700@oracle.com> Message-ID: <1415720755.12676.1.camel@oracle.com> Hi, On Tue, 2014-11-11 at 13:58 +0100, Stefan Karlsson wrote: > Hi all, > > Please review this patch to weaken the failing assert: > http://cr.openjdk.java.net/~stefank/8064556/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8064556 > > The asserts in pre_work_verification and post_work_verification were put > in place to make sure that the worker threads entered and exited with > clean slates. However, in JDK 8 this code path can be taken by the > VMThread, which will have registered Metadata when running the > MetadataOnStackMark constructor in the unloading phase of > ConcurrentMark::weakRefsWork. This is different from JDK 9, which always > use worker threads to execute this path. > > The patch has been verified with the failing test that found this issue. looks good. Thomas From thomas.schatzl at oracle.com Tue Nov 11 15:46:43 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 11 Nov 2014 16:46:43 +0100 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <54620A9C.2060806@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> <6F524023-4B70-41FC-9245-FED269599168@oracle.com> <5460B62F.1010307@oracle.com> <54620A9C.2060806@oracle.com> Message-ID: <1415720803.12676.2.camel@oracle.com> Hi, On Tue, 2014-11-11 at 14:09 +0100, Bengt Rutisson wrote: > Hi Kim, > > On 11/10/14 10:26 PM, Kim Barrett wrote: > > On Nov 10, 2014, at 7:57 AM, Bengt Rutisson wrote: > >> Thanks for splitting the tests up. They look fine to me now. Except that now that they are specialized I guess they could be moved into the test/gc/cms and test/gc/g1 folders. > > I thought about that, but felt that keeping the two closely related tests close together was better than separating them into the respective GC folders. > > > >> Is the -XX:SurvivorAlignmentInBytes=2k really required for the test to fail without the patch? In that case I think it is not enough to ignore the test since I *think* Jon's idea to fix JDK-8060463 is to limit what values are allowed for SurvivorAlignmentInBytes. Jon would know more about that. > > I talked to Jon, and he confirmed that limiting the permitted values > > for SurvivorAlignmentInBytes to something sane was the intended > > approach. So that is indeed a problem for my regression test. That > > test was cribbed from the 8062036 reproducer. > > > > I have not yet been able to find an alternative trigger, though > > searching for one has led to some interesting spelunking. Of course, > > that doesn't mean there isn't an alternative; the number of options > > and their possible interactions gives a pretty large search space. > > > > To trigger for G1 we need to somehow cause the concurrent mark thread > > to perform a GC remark fairly early in VM initialization. My attempts > > to create that situation have so far resulted in being too late to hit > > the problematic window, or blowing up for other reasons (such as heap > > size just being too small). > > > > Much as I think tests are a good thing, I'm getting to the point of > > wondering whether a regression test for this is worth the effort. > > Yes, I agree with you. We should probably just tag the bug with > noreg-hard and leave it for now. That also relieves us of the burden to > decide on which directory to put the tests in. ;) > > Thanks for trying hard to fix the tests. > Fine with me. Thanks for your effort. Thanks, Thomas From stefan.karlsson at oracle.com Tue Nov 11 15:36:36 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 11 Nov 2014 16:36:36 +0100 Subject: [8u40] RFR: 8064556: G1: ParallelGCThreads=0 may cause assert(!MetadataOnStackMark::has_buffer_for_thread(Thread::current())) failed: Should be empty In-Reply-To: <1415720755.12676.1.camel@oracle.com> References: <546207DC.8060700@oracle.com> <1415720755.12676.1.camel@oracle.com> Message-ID: <54622D04.1090204@oracle.com> On 2014-11-11 16:45, Thomas Schatzl wrote: > Hi, > > On Tue, 2014-11-11 at 13:58 +0100, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to weaken the failing assert: >> http://cr.openjdk.java.net/~stefank/8064556/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8064556 >> >> The asserts in pre_work_verification and post_work_verification were put >> in place to make sure that the worker threads entered and exited with >> clean slates. However, in JDK 8 this code path can be taken by the >> VMThread, which will have registered Metadata when running the >> MetadataOnStackMark constructor in the unloading phase of >> ConcurrentMark::weakRefsWork. This is different from JDK 9, which always >> use worker threads to execute this path. >> >> The patch has been verified with the failing test that found this issue. > looks good. Thanks, Thomas. StefanK > > Thomas > > From staffan.friberg at oracle.com Tue Nov 11 16:25:42 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Tue, 11 Nov 2014 17:25:42 +0100 Subject: RFR: JDK-8064473 - Improved handling of age during object copy in G1 Message-ID: <54623886.4060602@oracle.com> Hi, Small update to object copying in G1 to reduce the number of times age has to be read and also make sure the markOop can be trusted by reading and saving it prior doing the forwarded check. This change is broken out from the PLAB event RFE. bug: https://bugs.openjdk.java.net/browse/JDK-8064473 webrev: http://cr.openjdk.java.net/~sfriberg/JDK-8064473/webrev.01 Thanks, Staffan From kim.barrett at oracle.com Tue Nov 11 18:43:31 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 11 Nov 2014 13:43:31 -0500 Subject: RFR: 8062036: ConcurrentMarkThread::slt may be invoked before ConcurrentMarkThread::makeSurrogateLockerThread causing intermittent crashes In-Reply-To: <1415720803.12676.2.camel@oracle.com> References: <67DC79B3-DB1F-4914-9B6B-DB460F0A472B@oracle.com> <5457F42F.2070509@oracle.com> <1415204663.2912.18.camel@oracle.com> <545B38F2.9040806@oracle.com> <2C11F66E-8AD1-4AF8-9A14-84A0C145302E@oracle.com> <545CE87C.4020200@oracle.com> <6D5684CC-B166-4498-BEB8-592EA71C6599@oracle.com> <6F524023-4B70-41FC-9245-FED269599168@oracle.com> <5460B62F.1010307@oracle.com> <54620A9C.2060806@oracle.com> <1415720803.12676.2.camel@oracle.com> Message-ID: <62B7BBD1-5396-4EBC-B1A8-CB9C9E908171@oracle.com> On Nov 11, 2014, at 10:46 AM, Thomas Schatzl wrote: > > On Tue, 2014-11-11 at 14:09 +0100, Bengt Rutisson wrote: >>> Much as I think tests are a good thing, I'm getting to the point of >>> wondering whether a regression test for this is worth the effort. >> >> Yes, I agree with you. We should probably just tag the bug with >> noreg-hard and leave it for now. That also relieves us of the burden to >> decide on which directory to put the tests in. ;) >> >> Thanks for trying hard to fix the tests. >> > > Fine with me. Thanks for your effort. Thanks for your help. I?m doing a final repo update and test, and should be handing off to Jon as sponsor later today. From andrey.x.zakharov at oracle.com Wed Nov 12 10:59:17 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Wed, 12 Nov 2014 14:59:17 +0400 Subject: [RFR] 8016752: Test case for 8005857/8016740 (assert in GC_locker from PSOldGen::expand with -XX:+PrintGCDetails and Verbose) In-Reply-To: <5460952D.4070101@oracle.com> References: <54476AD3.3050907@oracle.com> <1415094901.5218.5.camel@oracle.com> <5460952D.4070101@oracle.com> Message-ID: <54633D85.5060006@oracle.com> Hi, team As this test requires fastdebug build, here is update according to latest changes to jtreg (@requires tag). Also I've suggested patch to jtreg for support matching against regexp patterns in @requires expressions (like "foo bar" ~= ".*bar") *(its blocker for original tesbug now) webrev: http://cr.openjdk.java.net/~azakharov/8016752/webrev.02/ testbug: https://bugs.openjdk.java.net/browse/JDK-8016752 Thanks. On 10.11.2014 14:36, Andrey Zakharov wrote: > Hi, Thomas > Yes, its fails very stable on 7u40b31 > # Internal Error > (/HUDSON/workspace/jdk7u40-2-build-linux-amd64-product/jdk7u40/hotspot/src/share/vm/memory/gcLocker.hpp:94), > pid=15099, tid=139928946063104 > # assert(_needs_gc || SafepointSynchronize::is_at_safepoint()) > failed: only read at safepoint > > with elapsed time (seconds): 1.273 > on local linux x64 box. > And passed on b32. > > Also, there are batches in comments of bugs for builds before fix and > with fix. > > > On 04.11.2014 13:55, Thomas Schatzl wrote: >> Hi Andrey, >> >> On Wed, 2014-10-22 at 12:29 +0400, Andrey Zakharov wrote: >>> Hi, team. >>> Here is simple test for JVM don't crash when its with PSOldGen, low >>> memory and with -XX:+PrintGCDetails and -XX:+Verbose flags >>> (https://bugs.openjdk.java.net/browse/JDK-8016740) based on Crasher >>> attached to original incident bug >>> >>> webrev: >>> http://cr.openjdk.java.net/~azakharov/8016752/webrev/ >> The test looks okay, but did you check that if you revert JDK-8016740, >> the test actually fails? >> The original reproducer was stressing the GC a lot more as it tried to >> cause crashes for 60 seconds. >> >> If that is the case, it looks okay to me. >> >> Thanks, >> Thomas >> >> > From bengt.rutisson at oracle.com Wed Nov 12 13:03:07 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 12 Nov 2014 14:03:07 +0100 Subject: RFR (L): 8064702: Remove the CMS foreground collector Message-ID: <54635A8B.3050402@oracle.com> Hi everyone, Can I have a couple of reviews for this change to remove the foreground collector in CMS? http://cr.openjdk.java.net/~brutisso/8064702/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8064702 This work is part of JEP 214 (http://openjdk.java.net/jeps/214). There are quite a few lines changed. Mostly it is just removal of code and it should hopefully be fairly straight forward to review. A couple of things are worth noting: The name "foreground collection" is used for two different things in CMS. Both the full GC that does a serial mark and compact of the whole heap (the fall back to SerialGC) and the foreground mode of completing a CMS cycle in a stop-the-world phase. We still need the fall back to the SerialGC for when we get a concurrent mode failure, so this change is just to remove the later feature. That is to remove the ability to interrupt a concurrent CMS cycle and finish it up in a stopped manner. This means that there is still code and comments using the word "foreground" even after the changes I propose. In particular the protocol to synchronize between the CMS collector and the MarkSweep serial full GC is left intact and uses locks and flags with foreground names. The intent of the change is to remove all traces of the actual foreground mode of the CMS cycle while changing as little as possible of the SerialGC interaction. Since there were these two modes of the foreground collector before many methods needed to know in which mode they were operating. This state was communicated by passing around parameters that were often called asynch and clear_all_soft_refs. Now that there is only one mode (the SerialGC one) I've tried to remove these parameters as much as possible. The interpretation being that asynch means the SerialGC (so always true now) and that we should always clear all soft refs when we do a full GC and never clear all soft refs when we do a concurrent CMS cycle. Thanks, Bengt From dmitry.fazunenko at oracle.com Wed Nov 12 12:45:36 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Wed, 12 Nov 2014 16:45:36 +0400 Subject: [RFR] 8016752: Test case for 8005857/8016740 (assert in GC_locker from PSOldGen::expand with -XX:+PrintGCDetails and Verbose) In-Reply-To: <54633D85.5060006@oracle.com> References: <54476AD3.3050907@oracle.com> <1415094901.5218.5.camel@oracle.com> <5460952D.4070101@oracle.com> <54633D85.5060006@oracle.com> Message-ID: <54635670.2070909@oracle.com> Hi Andrey, I think it's too early to rely on '~=' in @requires. There is no guarantee that it will be supported. (I hope it will). You also need to add @requires and list GC the test will work with. Thanks, Dima On 12.11.2014 14:59, Andrey Zakharov wrote: > Hi, team > As this test requires fastdebug build, here is update according to > latest changes to jtreg (@requires tag). > Also I've suggested patch to jtreg for support matching against regexp > patterns in @requires expressions (like "foo bar" ~= ".*bar") *(its > blocker for original tesbug now) > > webrev: > http://cr.openjdk.java.net/~azakharov/8016752/webrev.02/ > > testbug: > https://bugs.openjdk.java.net/browse/JDK-8016752 > > Thanks. > > On 10.11.2014 14:36, Andrey Zakharov wrote: >> Hi, Thomas >> Yes, its fails very stable on 7u40b31 >> # Internal Error >> (/HUDSON/workspace/jdk7u40-2-build-linux-amd64-product/jdk7u40/hotspot/src/share/vm/memory/gcLocker.hpp:94), >> pid=15099, tid=139928946063104 >> # assert(_needs_gc || SafepointSynchronize::is_at_safepoint()) >> failed: only read at safepoint >> >> with elapsed time (seconds): 1.273 >> on local linux x64 box. >> And passed on b32. >> >> Also, there are batches in comments of bugs for builds before fix and >> with fix. >> >> >> On 04.11.2014 13:55, Thomas Schatzl wrote: >>> Hi Andrey, >>> >>> On Wed, 2014-10-22 at 12:29 +0400, Andrey Zakharov wrote: >>>> Hi, team. >>>> Here is simple test for JVM don't crash when its with PSOldGen, low >>>> memory and with -XX:+PrintGCDetails and -XX:+Verbose flags >>>> (https://bugs.openjdk.java.net/browse/JDK-8016740) based on Crasher >>>> attached to original incident bug >>>> >>>> webrev: >>>> http://cr.openjdk.java.net/~azakharov/8016752/webrev/ >>> The test looks okay, but did you check that if you revert JDK-8016740, >>> the test actually fails? >>> The original reproducer was stressing the GC a lot more as it tried to >>> cause crashes for 60 seconds. >>> >>> If that is the case, it looks okay to me. >>> >>> Thanks, >>> Thomas >>> >>> >> > From stefan.karlsson at oracle.com Wed Nov 12 13:43:45 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 12 Nov 2014 14:43:45 +0100 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <54635A8B.3050402@oracle.com> References: <54635A8B.3050402@oracle.com> Message-ID: <54636411.2000006@oracle.com> Hi Bengt, On 2014-11-12 14:03, Bengt Rutisson wrote: > > Hi everyone, > > Can I have a couple of reviews for this change to remove the > foreground collector in CMS? > > http://cr.openjdk.java.net/~brutisso/8064702/webrev.00/ I haven't completed the review, but here are some comments: http://cr.openjdk.java.net/~brutisso/8064702/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.inline.hpp.udiff.html Could you explain/motivate this change?: inline void Par_MarkFromRootsClosure::do_yield_check() { if (ConcurrentMarkSweepThread::should_yield() && - !_collector->foregroundGCIsActive() && - _yield) { + !_collector->foregroundGCIsActive()) { do_yield_work(); } } http://cr.openjdk.java.net/~brutisso/8064702/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp.frames.html You removed the call to checkpointRootsInitialWork(bool asynch) that uses false as a parameter. I think you should remove that parameter (just like you did for the other functions) and remove the code that isn't executed anymore. thanks, StefanK > > https://bugs.openjdk.java.net/browse/JDK-8064702 > > This work is part of JEP 214 (http://openjdk.java.net/jeps/214). There > are quite a few lines changed. Mostly it is just removal of code and > it should hopefully be fairly straight forward to review. > > A couple of things are worth noting: > > The name "foreground collection" is used for two different things in > CMS. Both the full GC that does a serial mark and compact of the whole > heap (the fall back to SerialGC) and the foreground mode of completing > a CMS cycle in a stop-the-world phase. We still need the fall back to > the SerialGC for when we get a concurrent mode failure, so this change > is just to remove the later feature. That is to remove the ability to > interrupt a concurrent CMS cycle and finish it up in a stopped manner. > > This means that there is still code and comments using the word > "foreground" even after the changes I propose. In particular the > protocol to synchronize between the CMS collector and the MarkSweep > serial full GC is left intact and uses locks and flags with foreground > names. > > The intent of the change is to remove all traces of the actual > foreground mode of the CMS cycle while changing as little as possible > of the SerialGC interaction. > > Since there were these two modes of the foreground collector before > many methods needed to know in which mode they were operating. This > state was communicated by passing around parameters that were often > called asynch and clear_all_soft_refs. Now that there is only one mode > (the SerialGC one) I've tried to remove these parameters as much as > possible. The interpretation being that asynch means the SerialGC (so > always true now) and that we should always clear all soft refs when we > do a full GC and never clear all soft refs when we do a concurrent CMS > cycle. > > Thanks, > Bengt From evgeniya.stepanova at oracle.com Wed Nov 12 14:49:19 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 12 Nov 2014 18:49:19 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5458B960.8000305@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> Message-ID: <5463736F.7050109@oracle.com> Hi everyone! Since the decision was made to change only tests that fail because of conflict for now (skip "selfish" tests), I post new webrev for hotspot part of the JDK-8019361 : http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ Thanks, Evgeniya Stepanova On 04.11.2014 15:32, Dmitry Fazunenko wrote: > Nice plan! Please feel free to send me any feedback/questions > regarding @requires > > Thanks, > Dima > > > On 04.11.2014 11:40, Bengt Rutisson wrote: >> >> Hi Dima, >> >> Thanks for the answers. I think the currently proposed patch is a >> good start. We will have to evolve the @requires tag in the future, >> but let's have that discussion separate from this review. And we can >> start that discussion later when we have more experience with the >> current version of @requires. >> >> Thanks for doing this! >> Bengt >> >> >> >> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>> Hi Bengt, >>> >>> That's great that we have very closed visions! >>> >>> The general comment: currently, jtreg doesn't support any sort of >>> plugins, so you can't provide a VM specific handler of the @requires >>> or another tag. This is very annoying limitation and we have to live >>> with it. >>> >>> A few more comments inline. >>> >>> >>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>> >>>> >>>> Hi Dima, >>>> >>>> Answers inline. >>>> >>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>> Hi Bengt, >>>>> >>>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>>> >>>>> See comments inline. >>>>> >>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Evgeniya, >>>>>> >>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of >>>>>>> the JDK-8019361 >>>>>>> >>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>> >>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>>> another GC. >>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>> skip test if there is a conflict >>>>>> >>>>>> Thanks for fixing this! It is really great that we finally start >>>>>> sorting this out. >>>>>> >>>>>> First a general comment. The @requires tag has been developed >>>>>> without much cooperation with the GC team. We did have a lot of >>>>>> feedback when it was first presented a year ago, but it does not >>>>>> seem like this feedback was incorporated into the @requires that >>>>>> was eventually built. >>>>> >>>>> We tried to implement as much developer's wishes as possible. But >>>>> not everything is possible, sorry for that. >>>> >>>> Yes, I'm sure you have done your best. It's just that we have been >>>> requesting this feature for 3 years and I was expecting us to be >>>> able to influence the feature much more than was the case now. >>> >>> My personal hope: @requires will address ~90% of existing issues. >>> >>>> >>>>> >>>>>> >>>>>> I think this change that gets proposed now is a big step forward >>>>>> and I won't object to it. But I am pretty convinced that we will >>>>>> soon run in to the limitations of the current @requires >>>>>> implementation and we will have to redo this work. >>>>>> >>>>>> Some of the points I don't really like about the @requires tag are: >>>>>> >>>>>> - the "vm.gc" abstraction is more limiting than helping. It would >>>>>> have been better to just "require" any command line flag. >>>>> "vm.gc" is an alias to a very popular flag. It's also possible to >>>>> use: >>>>> vm.opt.UseG1GC == true instead. >>>>> >>>>> The table with all vars available in jtreg: >>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>> >>>> The problem with having this matching built in to JTreg is that it >>>> makes it very hard to change. When we discussed this a year ago I >>>> think we said that JTreg should only provide a means to test >>>> against the command line and a hook for running some java code in >>>> the @requires tag. That way we could put logic like this in a test >>>> library that is under our control. This would make it easy for us >>>> to change and also enables us to use different logic for different >>>> versions. >>> >>> I would be glad to have own harness... >>> >>>> >>>>> >>>>>> - the requirement should be per @run tag. Right now we have to do >>>>>> what you did in this change and use vm.gc=null even when some >>>>>> tests could actually have been run when a GC was specified. >>>>> it would be great, but it will unlikely happen in jtreg, as well >>>>> as test case support. >>>> >>>> what do you mean with test case support? Hi Evgeniya, >>> >>> Under test case support I mean ability to treat each @run as a >>> separate test. Now >>> >>> @test >>> @run -XX:g1RegSize=1m MyTest >>> @run -XX:g1RegSize=2m MyTest >>> @run -XX:g1RegSize=4m MyTest >>> class MyTest { >>> } >>> >>> is always a single test. You can't exclude, or re-run a part of it. >>> >>> >>>> >>>>> >>>>>> - there are many tests that require more than just a specific GC. >>>>>> Often there are other flags that can't be changed either for the >>>>>> test to work properly. >>>>> >>>>> yes. conflicting GC is just the most popular problem caused by >>>>> conflicting options. >>>>> If we address this issue and we are satisfied with solution, we >>>>> could move further. >>>> >>>> Yes, I agree that taking one step at the time is good. Personally I >>>> would have preferred that the first step was a "just run the >>>> command line as specified in the @run tag" step. >>>> >>>>> >>>>>> >>>>>> Maybe this is not the right place to discuss the current >>>>>> implementation of the @requires tag. I just want to say that I'm >>>>>> not too happy about how the @requires tag turned out. But >>>>>> assuming we have to use it the way it is now I guess the proposed >>>>>> changeset looks good. >>>>> >>>>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>>>> And thanks for reviewing it! >>>> >>>> Agreed. And as I said, I think the patch looks ok. I have not >>>> looked at all tests. But if they now pass with the combinations >>>> that we test with I guess they should be ok. >>> >>> Excellent! Thanks a lot! >>> >>>> >>>>> >>>>>> >>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>> without any GC flag). Tests are being excluded as expected. No >>>>>>> tests failed because of the conflict. >>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>> -XX:+UseConcMarkSweepGC. >>>>> >>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>> >>>> Ok. Thanks. >>>> >>>>> >>>>>> >>>>>> I think some of the test, like >>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >>>>>> with -XX:+UseParNewGC. Others, like >>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >>>>>> with -XX:-UseParNewGC. Could you test these two cases too? >>>>> >>>>> These two tests ignore vm flags. >>>>> Add @requires here is not necessary, but it will allow not execute >>>>> the tests when not needed. >>>>> So, if we run HS tests with 4 GC, we don't need to run these tests >>>>> 4 times, 1 should be enough. >>>> >>>> Do we really want to use the @requires functionality for this >>>> purpose? It seems like a way of misusing @requires. If we just want >>>> the tests to be run once I think Leonid's approach with tests lists >>>> seems more suitable. >>> >>> No, it's not a purpose of course, it's just side effect :) >>> >>> >>>> But are you sure that this is the reason for the @requires in this >>>> case? TestDefNewCMS does sound like a test that is DefNew specific. >>>> I don't see a reason to run it with ParNew. If it doesn't fail >>>> today it should probably be changed so that it does fail if it is >>>> run with the wrong GC. >>> >>> @requires - is not the silver bullet, but it's quite easy way to >>> solve a lot of issues. >>> >>> I hope, @requires will allow to reduce the number of "selfish" >>> tests, which produce a new java process to ignore vm flags coming >>> from outside. No @requires, no other mechanism could 100% protect a >>> test from running with conflicting options, but this is not the goal. >>> >>> If one runs tests with an exotic option, like a new G2 collector, >>> there shouldn't mass failures caused by options conflicts. But a few >>> failures could be handled manually. >>> >>> >>>> >>>>> >>>>>> Similarly it looks to me like there are tests that will fail if >>>>>> you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>>> >>>>>> Just a heads up. These two tests will soon be removed. I'm about >>>>>> to push a changeset that removes them: >>>>>> >>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>> okay, thank for letting us know. >>>>> >>>>>> >>>>>> Is there some way of making sure that all tests are run at one >>>>>> time or another. With this change there is a risk that some tests >>>>>> are never run and always skipped. Will we somehow be tracking >>>>>> what gets skipped and make sure that all tests are at least run >>>>>> once with the correct GC so that it is not skipped all the time? >>>>> >>>>> This is a very good question! >>>>> jtreg now doesn't report skipped tests, hopefully it will do soon, >>>>> after getting fix of: >>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>> >>>>> And yes, tracking tests which are not run is important thing. >>>>> @requires - is not the only to exclude test from execution. >>>>> >>>>> Other examples: >>>>> >>>>> /* >>>>> *@ignore >>>>> *@test >>>>> */ >>>>> ... >>>>> >>>>> /*@bug 4445555 >>>>> *@test >>>>> */ >>>>> ... >>>>> Such tests will never be run, because jtreg treats as test only >>>>> files with @test on the first place... >>>>> >>>>> So, making sure that tests do not disappear is important SQE >>>>> task, we know about that, we're thinking on solution (may be very >>>>> actively). But this subject for another discussion, not within RFR :) >>>> >>>> Right. Glad to hear that you are actively working on this! >>> >>> I was going to say "not very actively", but never mind, we know >>> about this problem. With introducing @requires mechanism it will >>> become more important! >>> >>> >>> Thanks for your comments! >>> >>> -- Dima >>> >>> >>>> >>>> Bengt >>>> >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> >>>>>> >>>>> >>>> >>> >> > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Wed Nov 12 15:21:25 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 12 Nov 2014 19:21:25 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <545CAE30.9010708@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> Message-ID: <54637AF5.9080600@oracle.com> Hi everyone! Since the decision was made to change only tests which fail because of conflict for now (skip "selfish" tests), I post new webrev for jdk part of the JDK-8019361 : http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ Thanks, Evgeniya Stepanova On 07.11.2014 15:34, Evgeniya Stepanova wrote: > David, Filipp, Katja > > Diff have been updated one more time: > java/lang/management/RuntimeMXBean/TestInputArgument.sh and > test/java/lang/ref/EnqueuePollRace.java have been changed > http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ > > Thanks > > On 07.11.2014 9:37, David Holmes wrote: >> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>> New webrev: >>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >> >> In: >> >> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >> >> the use of the gc options seems incidental - it's just picking two >> innocuous options to use - similar to the JpsHelper case. You could >> replace +UseParallelGC with something like +UseFastJNIAccessors >> (platform independent and normally true). >> >> David >> ----- >> >>> Thanks, >>> Evgeniya Stepanova >>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>> Thanks a lot! >>>> >>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>> Hi Katja, >>>>> >>>>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>>>> diff accordingly. >>>>> >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>> Hi Dima, >>>>>> >>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>> Hi Katja, >>>>>>> >>>>>>> You are right, there will be no conflict, because test ignores any >>>>>>> external VM flags. >>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>> >>>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>> >>>>>> This particular case is to test the defined flags are shown up as >>>>>> expected. >>>>>> >>>>>> Evgeniya, >>>>>> >>>>>> would you mind to change JpsHelper.java instead? >>>>>> >>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>> @@ -93,7 +93,7 @@ >>>>>> /** >>>>>> * VM arguments to start test application with >>>>>> */ >>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>> "-XX:+UseParallelGC"}; >>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>> "-XX:+PrintGCDetails"}; >>>>>> /** >>>>>> * VM flag to start test application with >>>>>> */ >>>>>> >>>>>> Best regards, >>>>>> Katja >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> @requires will allow to modify tests to include external vm >>>>>>> options without any risk of bumping into conflict and extend area >>>>>>> of test applicability. >>>>>>> >>>>>>> But if you still believe, that @requires is not necessary - it's >>>>>>> not a problem, tests could be kept as is. >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>> >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>>> should not be any conflicts either since these tests are running >>>>>>>> in driver mode: >>>>>>>> >>>>>>>> ... >>>>>>>> * @run driver TestJpsJar >>>>>>>> ... >>>>>>>> >>>>>>>> which means no flags from above are accepted. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Katja >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> tag added because tests contain string >>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>> >>>>>>>>> and JpsHelper defines >>>>>>>>> ... >>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>> ... >>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>> if (testVmArgs == null) { >>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>> } >>>>>>>>> return testVmArgs; >>>>>>>>> } >>>>>>>>> >>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>>> cleanup-if we use for example SerialGC we must be sure that tests >>>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>>>>>> JDK-8019361 >>>>>>>>>>> >>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Problem: Some tests explicitly set GC and fail when another GC >>>>>>>>>>> is set >>>>>>>>>>> outside >>>>>>>>>> >>>>>>>>>> I don't see why you have done this for the >>>>>>>>>> >>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>> >>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>> >>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>> skip test >>>>>>>>>>> if there is a conflict >>>>>>>>>> >>>>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>>>> was skipped; or does it only appear in the higher-level jtreg >>>>>>>>>> log? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>>>>>> without >>>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>>> failed >>>>>>>>>>> because of the conflict. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> >>>>>>>>>>> // >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /Evgeniya Stepanova/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>> >>> -- >>> /Evgeniya Stepanova/ > > -- > /Evgeniya Stepanova/ -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Wed Nov 12 14:23:50 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Wed, 12 Nov 2014 18:23:50 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5463736F.7050109@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> Message-ID: <54636D76.3010905@oracle.com> Hi Evgeniya, The fix looks good to me. I noticed the following minor things: - copyrights need to include the year of last change - test/gc/defnew/HeapChangeLogging.java - is listed among updated files, but doesn't contain any changes - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable 'prohibitedVmOptions' Thanks, Dima On 12.11.2014 18:49, Evgeniya Stepanova wrote: > Hi everyone! > > Since the decision was made to change only tests that fail because of > conflict for now (skip "selfish" tests), I post new webrev for hotspot > part of the JDK-8019361 > : > http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ > > Thanks, > Evgeniya Stepanova > On 04.11.2014 15:32, Dmitry Fazunenko wrote: >> Nice plan! Please feel free to send me any feedback/questions >> regarding @requires >> >> Thanks, >> Dima >> >> >> On 04.11.2014 11:40, Bengt Rutisson wrote: >>> >>> Hi Dima, >>> >>> Thanks for the answers. I think the currently proposed patch is a >>> good start. We will have to evolve the @requires tag in the future, >>> but let's have that discussion separate from this review. And we can >>> start that discussion later when we have more experience with the >>> current version of @requires. >>> >>> Thanks for doing this! >>> Bengt >>> >>> >>> >>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>> Hi Bengt, >>>> >>>> That's great that we have very closed visions! >>>> >>>> The general comment: currently, jtreg doesn't support any sort of >>>> plugins, so you can't provide a VM specific handler of the >>>> @requires or another tag. This is very annoying limitation and we >>>> have to live with it. >>>> >>>> A few more comments inline. >>>> >>>> >>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>> >>>>> >>>>> Hi Dima, >>>>> >>>>> Answers inline. >>>>> >>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>> Hi Bengt, >>>>>> >>>>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>>>> >>>>>> See comments inline. >>>>>> >>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of >>>>>>>> the JDK-8019361 >>>>>>>> >>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>> >>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>>>> another GC. >>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>> skip test if there is a conflict >>>>>>> >>>>>>> Thanks for fixing this! It is really great that we finally start >>>>>>> sorting this out. >>>>>>> >>>>>>> First a general comment. The @requires tag has been developed >>>>>>> without much cooperation with the GC team. We did have a lot of >>>>>>> feedback when it was first presented a year ago, but it does not >>>>>>> seem like this feedback was incorporated into the @requires that >>>>>>> was eventually built. >>>>>> >>>>>> We tried to implement as much developer's wishes as possible. But >>>>>> not everything is possible, sorry for that. >>>>> >>>>> Yes, I'm sure you have done your best. It's just that we have been >>>>> requesting this feature for 3 years and I was expecting us to be >>>>> able to influence the feature much more than was the case now. >>>> >>>> My personal hope: @requires will address ~90% of existing issues. >>>> >>>>> >>>>>> >>>>>>> >>>>>>> I think this change that gets proposed now is a big step forward >>>>>>> and I won't object to it. But I am pretty convinced that we will >>>>>>> soon run in to the limitations of the current @requires >>>>>>> implementation and we will have to redo this work. >>>>>>> >>>>>>> Some of the points I don't really like about the @requires tag are: >>>>>>> >>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>> would have been better to just "require" any command line flag. >>>>>> "vm.gc" is an alias to a very popular flag. It's also possible to >>>>>> use: >>>>>> vm.opt.UseG1GC == true instead. >>>>>> >>>>>> The table with all vars available in jtreg: >>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>> >>>>> The problem with having this matching built in to JTreg is that it >>>>> makes it very hard to change. When we discussed this a year ago I >>>>> think we said that JTreg should only provide a means to test >>>>> against the command line and a hook for running some java code in >>>>> the @requires tag. That way we could put logic like this in a test >>>>> library that is under our control. This would make it easy for us >>>>> to change and also enables us to use different logic for different >>>>> versions. >>>> >>>> I would be glad to have own harness... >>>> >>>>> >>>>>> >>>>>>> - the requirement should be per @run tag. Right now we have to >>>>>>> do what you did in this change and use vm.gc=null even when some >>>>>>> tests could actually have been run when a GC was specified. >>>>>> it would be great, but it will unlikely happen in jtreg, as well >>>>>> as test case support. >>>>> >>>>> what do you mean with test case support? Hi Evgeniya, >>>> >>>> Under test case support I mean ability to treat each @run as a >>>> separate test. Now >>>> >>>> @test >>>> @run -XX:g1RegSize=1m MyTest >>>> @run -XX:g1RegSize=2m MyTest >>>> @run -XX:g1RegSize=4m MyTest >>>> class MyTest { >>>> } >>>> >>>> is always a single test. You can't exclude, or re-run a part of it. >>>> >>>> >>>>> >>>>>> >>>>>>> - there are many tests that require more than just a specific >>>>>>> GC. Often there are other flags that can't be changed either for >>>>>>> the test to work properly. >>>>>> >>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>> conflicting options. >>>>>> If we address this issue and we are satisfied with solution, we >>>>>> could move further. >>>>> >>>>> Yes, I agree that taking one step at the time is good. Personally >>>>> I would have preferred that the first step was a "just run the >>>>> command line as specified in the @run tag" step. >>>>> >>>>>> >>>>>>> >>>>>>> Maybe this is not the right place to discuss the current >>>>>>> implementation of the @requires tag. I just want to say that I'm >>>>>>> not too happy about how the @requires tag turned out. But >>>>>>> assuming we have to use it the way it is now I guess the >>>>>>> proposed changeset looks good. >>>>>> >>>>>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>>>>> And thanks for reviewing it! >>>>> >>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>> looked at all tests. But if they now pass with the combinations >>>>> that we test with I guess they should be ok. >>>> >>>> Excellent! Thanks a lot! >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>>> without any GC flag). Tests are being excluded as expected. No >>>>>>>> tests failed because of the conflict. >>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>> -XX:+UseConcMarkSweepGC. >>>>>> >>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>> >>>>> Ok. Thanks. >>>>> >>>>>> >>>>>>> >>>>>>> I think some of the test, like >>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>> >>>>>> These two tests ignore vm flags. >>>>>> Add @requires here is not necessary, but it will allow not >>>>>> execute the tests when not needed. >>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>> tests 4 times, 1 should be enough. >>>>> >>>>> Do we really want to use the @requires functionality for this >>>>> purpose? It seems like a way of misusing @requires. If we just >>>>> want the tests to be run once I think Leonid's approach with tests >>>>> lists seems more suitable. >>>> >>>> No, it's not a purpose of course, it's just side effect :) >>>> >>>> >>>>> But are you sure that this is the reason for the @requires in this >>>>> case? TestDefNewCMS does sound like a test that is DefNew >>>>> specific. I don't see a reason to run it with ParNew. If it >>>>> doesn't fail today it should probably be changed so that it does >>>>> fail if it is run with the wrong GC. >>>> >>>> @requires - is not the silver bullet, but it's quite easy way to >>>> solve a lot of issues. >>>> >>>> I hope, @requires will allow to reduce the number of "selfish" >>>> tests, which produce a new java process to ignore vm flags coming >>>> from outside. No @requires, no other mechanism could 100% protect a >>>> test from running with conflicting options, but this is not the goal. >>>> >>>> If one runs tests with an exotic option, like a new G2 collector, >>>> there shouldn't mass failures caused by options conflicts. But a >>>> few failures could be handled manually. >>>> >>>> >>>>> >>>>>> >>>>>>> Similarly it looks to me like there are tests that will fail if >>>>>>> you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>>>> >>>>>>> Just a heads up. These two tests will soon be removed. I'm about >>>>>>> to push a changeset that removes them: >>>>>>> >>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>> okay, thank for letting us know. >>>>>> >>>>>>> >>>>>>> Is there some way of making sure that all tests are run at one >>>>>>> time or another. With this change there is a risk that some >>>>>>> tests are never run and always skipped. Will we somehow be >>>>>>> tracking what gets skipped and make sure that all tests are at >>>>>>> least run once with the correct GC so that it is not skipped all >>>>>>> the time? >>>>>> >>>>>> This is a very good question! >>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>> soon, after getting fix of: >>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>> >>>>>> And yes, tracking tests which are not run is important thing. >>>>>> @requires - is not the only to exclude test from execution. >>>>>> >>>>>> Other examples: >>>>>> >>>>>> /* >>>>>> *@ignore >>>>>> *@test >>>>>> */ >>>>>> ... >>>>>> >>>>>> /*@bug 4445555 >>>>>> *@test >>>>>> */ >>>>>> ... >>>>>> Such tests will never be run, because jtreg treats as test only >>>>>> files with @test on the first place... >>>>>> >>>>>> So, making sure that tests do not disappear is important SQE >>>>>> task, we know about that, we're thinking on solution (may be very >>>>>> actively). But this subject for another discussion, not within >>>>>> RFR :) >>>>> >>>>> Right. Glad to hear that you are actively working on this! >>>> >>>> I was going to say "not very actively", but never mind, we know >>>> about this problem. With introducing @requires mechanism it will >>>> become more important! >>>> >>>> >>>> Thanks for your comments! >>>> >>>> -- Dima >>>> >>>> >>>>> >>>>> Bengt >>>>> >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Wed Nov 12 15:43:14 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 12 Nov 2014 16:43:14 +0100 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <54637AF5.9080600@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <54637AF5.9080600@oracle.com> Message-ID: <54638012.3010202@oracle.com> On 2014-11-12 16:21, Evgeniya Stepanova wrote: > Hi everyone! > > Since the decision was made to change only tests which fail because of > conflict for now (skip "selfish" tests), I post new webrev for jdk > part of the JDK-8019361 > : > http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ Thanks for updating the webrev! A couple of comments: MemoryTestAllGC.sh The test is run three times, once with no params, once with ParallelGC and once with CMS. So, I think the @requires should just be vm.gc == "null". Similarly to what was done for PendingAllGC.sh. TestInputArgument.sh The changes here seem unrelated to @requires. EnqueuePollRace.java Can you explain why it is safe to remove -XX:+UseSerialGC for this test? JpsHelper.java Can you explain why it is safe to remvoe -XX:+UseParallelGC for this test? When I use Aurora to check what tests that currently are considered known because of JDK-8019361 I get a pretty long list: http://aurora.ru.oracle.com/functional/faces/CRRules.xhtml?cr=JDK-8019361 Are the tests in webrev.03 the only tests that still fail? Have the others been fixed in other ways? Thanks, Bengt > > Thanks, > Evgeniya Stepanova > On 07.11.2014 15:34, Evgeniya Stepanova wrote: >> David, Filipp, Katja >> >> Diff have been updated one more time: >> java/lang/management/RuntimeMXBean/TestInputArgument.sh and >> test/java/lang/ref/EnqueuePollRace.java have been changed >> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >> >> Thanks >> >> On 07.11.2014 9:37, David Holmes wrote: >>> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>>> New webrev: >>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >>> >>> In: >>> >>> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >>> >>> the use of the gc options seems incidental - it's just picking two >>> innocuous options to use - similar to the JpsHelper case. You could >>> replace +UseParallelGC with something like +UseFastJNIAccessors >>> (platform independent and normally true). >>> >>> David >>> ----- >>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>>> Thanks a lot! >>>>> >>>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>>> Hi Katja, >>>>>> >>>>>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>>>>> diff accordingly. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>>> Hi Dima, >>>>>>> >>>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>>> Hi Katja, >>>>>>>> >>>>>>>> You are right, there will be no conflict, because test ignores any >>>>>>>> external VM flags. >>>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>>> >>>>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>>> >>>>>>> This particular case is to test the defined flags are shown up as >>>>>>> expected. >>>>>>> >>>>>>> Evgeniya, >>>>>>> >>>>>>> would you mind to change JpsHelper.java instead? >>>>>>> >>>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>>> @@ -93,7 +93,7 @@ >>>>>>> /** >>>>>>> * VM arguments to start test application with >>>>>>> */ >>>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>> "-XX:+UseParallelGC"}; >>>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>> "-XX:+PrintGCDetails"}; >>>>>>> /** >>>>>>> * VM flag to start test application with >>>>>>> */ >>>>>>> >>>>>>> Best regards, >>>>>>> Katja >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> @requires will allow to modify tests to include external vm >>>>>>>> options without any risk of bumping into conflict and extend area >>>>>>>> of test applicability. >>>>>>>> >>>>>>>> But if you still believe, that @requires is not necessary - it's >>>>>>>> not a problem, tests could be kept as is. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dima >>>>>>>> >>>>>>>> >>>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>>> >>>>>>>>> Hi Evgeniya, >>>>>>>>> >>>>>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>>>> should not be any conflicts either since these tests are running >>>>>>>>> in driver mode: >>>>>>>>> >>>>>>>>> ... >>>>>>>>> * @run driver TestJpsJar >>>>>>>>> ... >>>>>>>>> >>>>>>>>> which means no flags from above are accepted. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Katja >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> tag added because tests contain string >>>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>>> >>>>>>>>>> and JpsHelper defines >>>>>>>>>> ... >>>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>> ... >>>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>>> if (testVmArgs == null) { >>>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>>> } >>>>>>>>>> return testVmArgs; >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>>>> cleanup-if we use for example SerialGC we must be sure that >>>>>>>>>> tests >>>>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of the >>>>>>>>>>>> JDK-8019361 >>>>>>>>>>>> >>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when another GC >>>>>>>>>>>> is set >>>>>>>>>>>> outside >>>>>>>>>>> >>>>>>>>>>> I don't see why you have done this for the >>>>>>>>>>> >>>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>>> >>>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>>> >>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>>> skip test >>>>>>>>>>>> if there is a conflict >>>>>>>>>>> >>>>>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>>>>> was skipped; or does it only appear in the higher-level >>>>>>>>>>> jtreg log? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>>>> and >>>>>>>>>>>> without >>>>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>>>> failed >>>>>>>>>>>> because of the conflict. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>> >>>>>>>>>>>> // >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >> >> -- >> /Evgeniya Stepanova/ > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Wed Nov 12 15:48:24 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 12 Nov 2014 16:48:24 +0100 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <54638012.3010202@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <54637AF5.9080600@oracle.com> <54638012.3010202@oracle.com> Message-ID: <54638148.20903@oracle.com> On 2014-11-12 16:43, Bengt Rutisson wrote: > > On 2014-11-12 16:21, Evgeniya Stepanova wrote: >> Hi everyone! >> >> Since the decision was made to change only tests which fail because >> of conflict for now (skip "selfish" tests), I post new webrev for jdk >> part of the JDK-8019361 >> : >> http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ > > Thanks for updating the webrev! > > A couple of comments: > > MemoryTestAllGC.sh > > The test is run three times, once with no params, once with ParallelGC > and once with CMS. So, I think the @requires should just be vm.gc == > "null". Similarly to what was done for PendingAllGC.sh. > > > TestInputArgument.sh > > The changes here seem unrelated to @requires. > > > EnqueuePollRace.java > > Can you explain why it is safe to remove -XX:+UseSerialGC for this test? > > > JpsHelper.java > > Can you explain why it is safe to remvoe -XX:+UseParallelGC for this test? > > > When I use Aurora to check what tests that currently are considered > known because of JDK-8019361 I get a pretty long list: > > http://aurora.ru.oracle.com/functional/faces/CRRules.xhtml?cr=JDK-8019361 > > Are the tests in webrev.03 the only tests that still fail? Have the > others been fixed in other ways? I just saw the update to the other review thread "RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests". I assume the rest of the failing tests are there. I'll have a look. Bengt > > Thanks, > Bengt > > > >> >> Thanks, >> Evgeniya Stepanova >> On 07.11.2014 15:34, Evgeniya Stepanova wrote: >>> David, Filipp, Katja >>> >>> Diff have been updated one more time: >>> java/lang/management/RuntimeMXBean/TestInputArgument.sh and >>> test/java/lang/ref/EnqueuePollRace.java have been changed >>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >>> >>> Thanks >>> >>> On 07.11.2014 9:37, David Holmes wrote: >>>> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>>>> New webrev: >>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >>>> >>>> In: >>>> >>>> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >>>> >>>> the use of the gc options seems incidental - it's just picking two >>>> innocuous options to use - similar to the JpsHelper case. You could >>>> replace +UseParallelGC with something like +UseFastJNIAccessors >>>> (platform independent and normally true). >>>> >>>> David >>>> ----- >>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>>>> Thanks a lot! >>>>>> >>>>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>>>> Hi Katja, >>>>>>> >>>>>>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>>>>>> diff accordingly. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>>>> Hi Dima, >>>>>>>> >>>>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>>>> Hi Katja, >>>>>>>>> >>>>>>>>> You are right, there will be no conflict, because test ignores >>>>>>>>> any >>>>>>>>> external VM flags. >>>>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>>>> >>>>>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>>>> >>>>>>>> This particular case is to test the defined flags are shown up as >>>>>>>> expected. >>>>>>>> >>>>>>>> Evgeniya, >>>>>>>> >>>>>>>> would you mind to change JpsHelper.java instead? >>>>>>>> >>>>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>>>> @@ -93,7 +93,7 @@ >>>>>>>> /** >>>>>>>> * VM arguments to start test application with >>>>>>>> */ >>>>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>> "-XX:+PrintGCDetails"}; >>>>>>>> /** >>>>>>>> * VM flag to start test application with >>>>>>>> */ >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Katja >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> @requires will allow to modify tests to include external vm >>>>>>>>> options without any risk of bumping into conflict and extend area >>>>>>>>> of test applicability. >>>>>>>>> >>>>>>>>> But if you still believe, that @requires is not necessary - it's >>>>>>>>> not a problem, tests could be kept as is. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>>>> >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>>>>> should not be any conflicts either since these tests are running >>>>>>>>>> in driver mode: >>>>>>>>>> >>>>>>>>>> ... >>>>>>>>>> * @run driver TestJpsJar >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>>> which means no flags from above are accepted. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Katja >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> tag added because tests contain string >>>>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>>>> >>>>>>>>>>> and JpsHelper defines >>>>>>>>>>> ... >>>>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>>> ... >>>>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>>>> if (testVmArgs == null) { >>>>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>>>> } >>>>>>>>>>> return testVmArgs; >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>>>>> cleanup-if we use for example SerialGC we must be sure that >>>>>>>>>>> tests >>>>>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>> >>>>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of >>>>>>>>>>>>> the >>>>>>>>>>>>> JDK-8019361 >>>>>>>>>>>>> >>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>> another GC >>>>>>>>>>>>> is set >>>>>>>>>>>>> outside >>>>>>>>>>>> >>>>>>>>>>>> I don't see why you have done this for the >>>>>>>>>>>> >>>>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>>>> >>>>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>>>> >>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>>>> skip test >>>>>>>>>>>>> if there is a conflict >>>>>>>>>>>> >>>>>>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>>>>>> was skipped; or does it only appear in the higher-level >>>>>>>>>>>> jtreg log? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>> -XX:+UseConcMarkSweep and >>>>>>>>>>>>> without >>>>>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>>>>> failed >>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>> >>>>>>>>>>>>> // >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>> >>> -- >>> /Evgeniya Stepanova/ >> >> -- >> /Evgeniya Stepanova/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Wed Nov 12 16:07:23 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 12 Nov 2014 20:07:23 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <54638012.3010202@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <54637AF5.9080600@oracle.com> <54638012.3010202@oracle.com> Message-ID: <546385BB.3080801@oracle.com> Hi Bengt, Please see comments inline On 12.11.2014 19:43, Bengt Rutisson wrote: > > On 2014-11-12 16:21, Evgeniya Stepanova wrote: >> Hi everyone! >> >> Since the decision was made to change only tests which fail because >> of conflict for now (skip "selfish" tests), I post new webrev for jdk >> part of the JDK-8019361 >> : >> http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ > > Thanks for updating the webrev! > > A couple of comments: > > MemoryTestAllGC.sh > > The test is run three times, once with no params, once with ParallelGC > and once with CMS. So, I think the @requires should just be vm.gc == > "null". Similarly to what was done for PendingAllGC.sh. > The third run (with CMS) is commented. Without this run UseParallelGC is valid option #runOne -XX:+UseConcMarkSweepGC MemoryTest 3 (http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/test/java/lang/management/MemoryMXBean/MemoryTestAllGC.sh.frames.html) > > TestInputArgument.sh > > The changes here seem unrelated to @requires. > This test was changed after conversation with David Holmes (see thread below) > > EnqueuePollRace.java > > Can you explain why it is safe to remove -XX:+UseSerialGC for this test? > > This test was modified after conversation with Filipp Zhinkin and Mandy Chung (https://bugs.openjdk.java.net/browse/JDK-8051723) > JpsHelper.java > > Can you explain why it is safe to remvoe -XX:+UseParallelGC for this test? > This test was changed after conversation with Katja Kantserova (see thread below), GC flag is just an arbitrary chosen test flag > > When I use Aurora to check what tests that currently are considered > known because of JDK-8019361 I get a pretty long list: > > http://aurora.ru.oracle.com/functional/faces/CRRules.xhtml?cr=JDK-8019361 > > Are the tests in webrev.03 the only tests that still fail? Have the > others been fixed in other ways? There would be 2 more changes in reviews in closed part :) Thanks, Evgeniya Stepanova > > Thanks, > Bengt > > > >> >> Thanks, >> Evgeniya Stepanova >> On 07.11.2014 15:34, Evgeniya Stepanova wrote: >>> David, Filipp, Katja >>> >>> Diff have been updated one more time: >>> java/lang/management/RuntimeMXBean/TestInputArgument.sh and >>> test/java/lang/ref/EnqueuePollRace.java have been changed >>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >>> >>> Thanks >>> >>> On 07.11.2014 9:37, David Holmes wrote: >>>> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>>>> New webrev: >>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >>>> >>>> In: >>>> >>>> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >>>> >>>> the use of the gc options seems incidental - it's just picking two >>>> innocuous options to use - similar to the JpsHelper case. You could >>>> replace +UseParallelGC with something like +UseFastJNIAccessors >>>> (platform independent and normally true). >>>> >>>> David >>>> ----- >>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>>>> Thanks a lot! >>>>>> >>>>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>>>> Hi Katja, >>>>>>> >>>>>>> Ok, this seems to be a perfect solution. Thank you. I'll change the >>>>>>> diff accordingly. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>>>> Hi Dima, >>>>>>>> >>>>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>>>> Hi Katja, >>>>>>>>> >>>>>>>>> You are right, there will be no conflict, because test ignores >>>>>>>>> any >>>>>>>>> external VM flags. >>>>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>>>> >>>>>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>>>> >>>>>>>> This particular case is to test the defined flags are shown up as >>>>>>>> expected. >>>>>>>> >>>>>>>> Evgeniya, >>>>>>>> >>>>>>>> would you mind to change JpsHelper.java instead? >>>>>>>> >>>>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>>>> @@ -93,7 +93,7 @@ >>>>>>>> /** >>>>>>>> * VM arguments to start test application with >>>>>>>> */ >>>>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>> "-XX:+PrintGCDetails"}; >>>>>>>> /** >>>>>>>> * VM flag to start test application with >>>>>>>> */ >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Katja >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> @requires will allow to modify tests to include external vm >>>>>>>>> options without any risk of bumping into conflict and extend area >>>>>>>>> of test applicability. >>>>>>>>> >>>>>>>>> But if you still believe, that @requires is not necessary - it's >>>>>>>>> not a problem, tests could be kept as is. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>>>> >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> As David has pointed out these jps tests are not testing gc. The >>>>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>>>>> should not be any conflicts either since these tests are running >>>>>>>>>> in driver mode: >>>>>>>>>> >>>>>>>>>> ... >>>>>>>>>> * @run driver TestJpsJar >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>>> which means no flags from above are accepted. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Katja >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> tag added because tests contain string >>>>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>>>> >>>>>>>>>>> and JpsHelper defines >>>>>>>>>>> ... >>>>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>>> ... >>>>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>>>> if (testVmArgs == null) { >>>>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>>>> } >>>>>>>>>>> return testVmArgs; >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>>>>> cleanup-if we use for example SerialGC we must be sure that >>>>>>>>>>> tests >>>>>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>>>>> Parallel GC. Plus there is no any sense to run test twice in TC >>>>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>> >>>>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part of >>>>>>>>>>>>> the >>>>>>>>>>>>> JDK-8019361 >>>>>>>>>>>>> >>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>> another GC >>>>>>>>>>>>> is set >>>>>>>>>>>>> outside >>>>>>>>>>>> >>>>>>>>>>>> I don't see why you have done this for the >>>>>>>>>>>> >>>>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>>>> >>>>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>>>> >>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>>>> skip test >>>>>>>>>>>>> if there is a conflict >>>>>>>>>>>> >>>>>>>>>>>> Just wondering: Does a skipped test get a .jtr file showing it >>>>>>>>>>>> was skipped; or does it only appear in the higher-level >>>>>>>>>>>> jtreg log? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>> -XX:+UseConcMarkSweep and >>>>>>>>>>>>> without >>>>>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>>>>> failed >>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>> >>>>>>>>>>>>> // >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>> >>> -- >>> /Evgeniya Stepanova/ >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Wed Nov 12 16:28:15 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 12 Nov 2014 20:28:15 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <54636D76.3010905@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> Message-ID: <54638A9F.2030209@oracle.com> Hi Dmitry, You are right - I've forgotten about copyrights Copyrights and other issues you mentioned fixed. New webrev: http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ Thanks Evgeniya Stepanova On 12.11.2014 18:23, Dmitry Fazunenko wrote: > Hi Evgeniya, > > The fix looks good to me. > > I noticed the following minor things: > - copyrights need to include the year of last change > - test/gc/defnew/HeapChangeLogging.java - is listed among updated > files, but doesn't contain any changes > - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable > 'prohibitedVmOptions' > > Thanks, > Dima > > > On 12.11.2014 18:49, Evgeniya Stepanova wrote: >> Hi everyone! >> >> Since the decision was made to change only tests that fail because of >> conflict for now (skip "selfish" tests), I post new webrev for >> hotspot part of the JDK-8019361 >> : >> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >> >> Thanks, >> Evgeniya Stepanova >> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>> Nice plan! Please feel free to send me any feedback/questions >>> regarding @requires >>> >>> Thanks, >>> Dima >>> >>> >>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>> >>>> Hi Dima, >>>> >>>> Thanks for the answers. I think the currently proposed patch is a >>>> good start. We will have to evolve the @requires tag in the future, >>>> but let's have that discussion separate from this review. And we >>>> can start that discussion later when we have more experience with >>>> the current version of @requires. >>>> >>>> Thanks for doing this! >>>> Bengt >>>> >>>> >>>> >>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>> Hi Bengt, >>>>> >>>>> That's great that we have very closed visions! >>>>> >>>>> The general comment: currently, jtreg doesn't support any sort of >>>>> plugins, so you can't provide a VM specific handler of the >>>>> @requires or another tag. This is very annoying limitation and we >>>>> have to live with it. >>>>> >>>>> A few more comments inline. >>>>> >>>>> >>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>> >>>>>> >>>>>> Hi Dima, >>>>>> >>>>>> Answers inline. >>>>>> >>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>>>>> >>>>>>> See comments inline. >>>>>>> >>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of >>>>>>>>> the JDK-8019361 >>>>>>>>> >>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>> >>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>>>>> another GC. >>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>> skip test if there is a conflict >>>>>>>> >>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>> start sorting this out. >>>>>>>> >>>>>>>> First a general comment. The @requires tag has been developed >>>>>>>> without much cooperation with the GC team. We did have a lot of >>>>>>>> feedback when it was first presented a year ago, but it does >>>>>>>> not seem like this feedback was incorporated into the @requires >>>>>>>> that was eventually built. >>>>>>> >>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>> But not everything is possible, sorry for that. >>>>>> >>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>> been requesting this feature for 3 years and I was expecting us >>>>>> to be able to influence the feature much more than was the case now. >>>>> >>>>> My personal hope: @requires will address ~90% of existing issues. >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>> I think this change that gets proposed now is a big step >>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>> that we will soon run in to the limitations of the current >>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>> >>>>>>>> Some of the points I don't really like about the @requires tag are: >>>>>>>> >>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>> would have been better to just "require" any command line flag. >>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible >>>>>>> to use: >>>>>>> vm.opt.UseG1GC == true instead. >>>>>>> >>>>>>> The table with all vars available in jtreg: >>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>> >>>>>> The problem with having this matching built in to JTreg is that >>>>>> it makes it very hard to change. When we discussed this a year >>>>>> ago I think we said that JTreg should only provide a means to >>>>>> test against the command line and a hook for running some java >>>>>> code in the @requires tag. That way we could put logic like this >>>>>> in a test library that is under our control. This would make it >>>>>> easy for us to change and also enables us to use different logic >>>>>> for different versions. >>>>> >>>>> I would be glad to have own harness... >>>>> >>>>>> >>>>>>> >>>>>>>> - the requirement should be per @run tag. Right now we have to >>>>>>>> do what you did in this change and use vm.gc=null even when >>>>>>>> some tests could actually have been run when a GC was specified. >>>>>>> it would be great, but it will unlikely happen in jtreg, as well >>>>>>> as test case support. >>>>>> >>>>>> what do you mean with test case support? Hi Evgeniya, >>>>> >>>>> Under test case support I mean ability to treat each @run as a >>>>> separate test. Now >>>>> >>>>> @test >>>>> @run -XX:g1RegSize=1m MyTest >>>>> @run -XX:g1RegSize=2m MyTest >>>>> @run -XX:g1RegSize=4m MyTest >>>>> class MyTest { >>>>> } >>>>> >>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>> >>>>> >>>>>> >>>>>>> >>>>>>>> - there are many tests that require more than just a specific >>>>>>>> GC. Often there are other flags that can't be changed either >>>>>>>> for the test to work properly. >>>>>>> >>>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>>> conflicting options. >>>>>>> If we address this issue and we are satisfied with solution, we >>>>>>> could move further. >>>>>> >>>>>> Yes, I agree that taking one step at the time is good. Personally >>>>>> I would have preferred that the first step was a "just run the >>>>>> command line as specified in the @run tag" step. >>>>>> >>>>>>> >>>>>>>> >>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>> I'm not too happy about how the @requires tag turned out. But >>>>>>>> assuming we have to use it the way it is now I guess the >>>>>>>> proposed changeset looks good. >>>>>>> >>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>> jtreg :) >>>>>>> And thanks for reviewing it! >>>>>> >>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>> looked at all tests. But if they now pass with the combinations >>>>>> that we test with I guess they should be ok. >>>>> >>>>> Excellent! Thanks a lot! >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>> >>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>> >>>>>> Ok. Thanks. >>>>>> >>>>>>> >>>>>>>> >>>>>>>> I think some of the test, like >>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>> >>>>>>> These two tests ignore vm flags. >>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>> execute the tests when not needed. >>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>> tests 4 times, 1 should be enough. >>>>>> >>>>>> Do we really want to use the @requires functionality for this >>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>> want the tests to be run once I think Leonid's approach with >>>>>> tests lists seems more suitable. >>>>> >>>>> No, it's not a purpose of course, it's just side effect :) >>>>> >>>>> >>>>>> But are you sure that this is the reason for the @requires in >>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>> doesn't fail today it should probably be changed so that it does >>>>>> fail if it is run with the wrong GC. >>>>> >>>>> @requires - is not the silver bullet, but it's quite easy way to >>>>> solve a lot of issues. >>>>> >>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>> tests, which produce a new java process to ignore vm flags coming >>>>> from outside. No @requires, no other mechanism could 100% protect >>>>> a test from running with conflicting options, but this is not the >>>>> goal. >>>>> >>>>> If one runs tests with an exotic option, like a new G2 collector, >>>>> there shouldn't mass failures caused by options conflicts. But a >>>>> few failures could be handled manually. >>>>> >>>>> >>>>>> >>>>>>> >>>>>>>> Similarly it looks to me like there are tests that will fail if >>>>>>>> you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>>>>> >>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>> about to push a changeset that removes them: >>>>>>>> >>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>> okay, thank for letting us know. >>>>>>> >>>>>>>> >>>>>>>> Is there some way of making sure that all tests are run at one >>>>>>>> time or another. With this change there is a risk that some >>>>>>>> tests are never run and always skipped. Will we somehow be >>>>>>>> tracking what gets skipped and make sure that all tests are at >>>>>>>> least run once with the correct GC so that it is not skipped >>>>>>>> all the time? >>>>>>> >>>>>>> This is a very good question! >>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>> soon, after getting fix of: >>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>> >>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>> @requires - is not the only to exclude test from execution. >>>>>>> >>>>>>> Other examples: >>>>>>> >>>>>>> /* >>>>>>> *@ignore >>>>>>> *@test >>>>>>> */ >>>>>>> ... >>>>>>> >>>>>>> /*@bug 4445555 >>>>>>> *@test >>>>>>> */ >>>>>>> ... >>>>>>> Such tests will never be run, because jtreg treats as test only >>>>>>> files with @test on the first place... >>>>>>> >>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>> very actively). But this subject for another discussion, not >>>>>>> within RFR :) >>>>>> >>>>>> Right. Glad to hear that you are actively working on this! >>>>> >>>>> I was going to say "not very actively", but never mind, we know >>>>> about this problem. With introducing @requires mechanism it will >>>>> become more important! >>>>> >>>>> >>>>> Thanks for your comments! >>>>> >>>>> -- Dima >>>>> >>>>> >>>>>> >>>>>> Bengt >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Wed Nov 12 16:49:41 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 12 Nov 2014 20:49:41 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <546385BB.3080801@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <54637AF5.9080600@oracle.com> <54638012.3010202@oracle.com> <546385BB.3080801@oracle.com> Message-ID: <54638FA5.5000505@oracle.com> Forgotten copyrights were changed http://cr.openjdk.java.net/~eistepan/8062536/webrev.04/ On 12.11.2014 20:07, Evgeniya Stepanova wrote: > Hi Bengt, > > Please see comments inline > On 12.11.2014 19:43, Bengt Rutisson wrote: >> >> On 2014-11-12 16:21, Evgeniya Stepanova wrote: >>> Hi everyone! >>> >>> Since the decision was made to change only tests which fail because >>> of conflict for now (skip "selfish" tests), I post new webrev for >>> jdk part of the JDK-8019361 >>> : >>> http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ >> >> Thanks for updating the webrev! >> >> A couple of comments: >> >> MemoryTestAllGC.sh >> >> The test is run three times, once with no params, once with >> ParallelGC and once with CMS. So, I think the @requires should just >> be vm.gc == "null". Similarly to what was done for PendingAllGC.sh. >> > The third run (with CMS) is commented. Without this run UseParallelGC > is valid option > #runOne -XX:+UseConcMarkSweepGC MemoryTest 3 > (http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/test/java/lang/management/MemoryMXBean/MemoryTestAllGC.sh.frames.html) >> >> TestInputArgument.sh >> >> The changes here seem unrelated to @requires. >> > This test was changed after conversation with David Holmes (see > thread below) >> >> EnqueuePollRace.java >> >> Can you explain why it is safe to remove -XX:+UseSerialGC for this test? >> >> > This test was modified after conversation with Filipp Zhinkin and > Mandy Chung (https://bugs.openjdk.java.net/browse/JDK-8051723) >> JpsHelper.java >> >> Can you explain why it is safe to remvoe -XX:+UseParallelGC for this >> test? >> > This test was changed after conversation with Katja Kantserova (see > thread below), GC flag is just an arbitrary chosen test flag >> >> When I use Aurora to check what tests that currently are considered >> known because of JDK-8019361 I get a pretty long list: >> >> http://aurora.ru.oracle.com/functional/faces/CRRules.xhtml?cr=JDK-8019361 >> >> Are the tests in webrev.03 the only tests that still fail? Have the >> others been fixed in other ways? > There would be 2 more changes in reviews in closed part :) > > Thanks, > Evgeniya Stepanova >> >> Thanks, >> Bengt >> >> >> >>> >>> Thanks, >>> Evgeniya Stepanova >>> On 07.11.2014 15:34, Evgeniya Stepanova wrote: >>>> David, Filipp, Katja >>>> >>>> Diff have been updated one more time: >>>> java/lang/management/RuntimeMXBean/TestInputArgument.sh and >>>> test/java/lang/ref/EnqueuePollRace.java have been changed >>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >>>> >>>> Thanks >>>> >>>> On 07.11.2014 9:37, David Holmes wrote: >>>>> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>>>>> New webrev: >>>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >>>>> >>>>> In: >>>>> >>>>> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >>>>> >>>>> the use of the gc options seems incidental - it's just picking two >>>>> innocuous options to use - similar to the JpsHelper case. You >>>>> could replace +UseParallelGC with something like >>>>> +UseFastJNIAccessors (platform independent and normally true). >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>>>>> Thanks a lot! >>>>>>> >>>>>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>>>>> Hi Katja, >>>>>>>> >>>>>>>> Ok, this seems to be a perfect solution. Thank you. I'll change >>>>>>>> the >>>>>>>> diff accordingly. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>>>>> Hi Dima, >>>>>>>>> >>>>>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>>>>> Hi Katja, >>>>>>>>>> >>>>>>>>>> You are right, there will be no conflict, because test >>>>>>>>>> ignores any >>>>>>>>>> external VM flags. >>>>>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>>>>> >>>>>>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>>>>> >>>>>>>>> This particular case is to test the defined flags are shown up as >>>>>>>>> expected. >>>>>>>>> >>>>>>>>> Evgeniya, >>>>>>>>> >>>>>>>>> would you mind to change JpsHelper.java instead? >>>>>>>>> >>>>>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>>>>> @@ -93,7 +93,7 @@ >>>>>>>>> /** >>>>>>>>> * VM arguments to start test application with >>>>>>>>> */ >>>>>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>> "-XX:+PrintGCDetails"}; >>>>>>>>> /** >>>>>>>>> * VM flag to start test application with >>>>>>>>> */ >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Katja >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> @requires will allow to modify tests to include external vm >>>>>>>>>> options without any risk of bumping into conflict and extend >>>>>>>>>> area >>>>>>>>>> of test applicability. >>>>>>>>>> >>>>>>>>>> But if you still believe, that @requires is not necessary - it's >>>>>>>>>> not a problem, tests could be kept as is. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> As David has pointed out these jps tests are not testing gc. >>>>>>>>>>> The >>>>>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>>>>>> should not be any conflicts either since these tests are >>>>>>>>>>> running >>>>>>>>>>> in driver mode: >>>>>>>>>>> >>>>>>>>>>> ... >>>>>>>>>>> * @run driver TestJpsJar >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>>> which means no flags from above are accepted. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Katja >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> tag added because tests contain string >>>>>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>>>>> >>>>>>>>>>>> and JpsHelper defines >>>>>>>>>>>> ... >>>>>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>>>> ... >>>>>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>>>>> if (testVmArgs == null) { >>>>>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>>>>> } >>>>>>>>>>>> return testVmArgs; >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>>>>>> cleanup-if we use for example SerialGC we must be sure that >>>>>>>>>>>> tests >>>>>>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>>>>>> Parallel GC. Plus there is no any sense to run test twice >>>>>>>>>>>> in TC >>>>>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>> >>>>>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part >>>>>>>>>>>>>> of the >>>>>>>>>>>>>> JDK-8019361 >>>>>>>>>>>>>> >>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>> another GC >>>>>>>>>>>>>> is set >>>>>>>>>>>>>> outside >>>>>>>>>>>>> >>>>>>>>>>>>> I don't see why you have done this for the >>>>>>>>>>>>> >>>>>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>>>>> >>>>>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>>>>> >>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>>>>> skip test >>>>>>>>>>>>>> if there is a conflict >>>>>>>>>>>>> >>>>>>>>>>>>> Just wondering: Does a skipped test get a .jtr file >>>>>>>>>>>>> showing it >>>>>>>>>>>>> was skipped; or does it only appear in the higher-level >>>>>>>>>>>>> jtreg log? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>> -XX:+UseConcMarkSweep and >>>>>>>>>>>>>> without >>>>>>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>>>>>> failed >>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>> >>>>>>>>>>>>>> // >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >>> -- >>> /Evgeniya Stepanova/ >> > > -- > /Evgeniya Stepanova/ -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Wed Nov 12 16:49:11 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 12 Nov 2014 17:49:11 +0100 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <54636411.2000006@oracle.com> References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> Message-ID: <54638F87.30809@oracle.com> Hi Stefan, Thanks for looking at this! On 2014-11-12 14:43, Stefan Karlsson wrote: > Hi Bengt, > > On 2014-11-12 14:03, Bengt Rutisson wrote: >> >> Hi everyone, >> >> Can I have a couple of reviews for this change to remove the >> foreground collector in CMS? >> >> http://cr.openjdk.java.net/~brutisso/8064702/webrev.00/ > > I haven't completed the review, but here are some comments: > > http://cr.openjdk.java.net/~brutisso/8064702/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.inline.hpp.udiff.html > > > Could you explain/motivate this change?: > > inline void Par_MarkFromRootsClosure::do_yield_check() { > if (ConcurrentMarkSweepThread::should_yield() && > - !_collector->foregroundGCIsActive() && > - _yield) { > + !_collector->foregroundGCIsActive()) { > do_yield_work(); > } > } The _yield flag is a bit interesting. There are two closures for doing the marking, MarkFromRootsClosure and Par_MarkFromRootsClosure. The MarkFromRootsClosure yields based on the CMSYield flag as it is being used in CMSCollector::do_marking_st(): MarkFromRootsClosure markFromRootsClosure(this, _span, &_markBitMap, &_markStack, CMSYield); But the Par_MarkFromRootsClosure was setting the yield value based on the asynch value: // Do the marking work within a non-empty span -- // the last argument to the constructor indicates whether the // iteration should be incremental with periodic yields. Par_MarkFromRootsClosure cl(this, _collector, my_span, &_collector->_markBitMap, work_queue(i), &_collector->_markStack, _asynch); The _asynch value here came from CMSCollector::do_marking_mt() which after the removal of the foreground collector is always true. So, in the Par_MarkFromRootsClosure I could remove the _yield field since it is now always true. But in the MarkFromRootsClosure I need to leave it in to keep the same behavior as before with the CMSYield flag. It seems a bit fishy to me that the two marking closure work differently, but I'd rather not change this behavior as part of this change. > > > http://cr.openjdk.java.net/~brutisso/8064702/webrev.00/src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp.frames.html > > > You removed the call to checkpointRootsInitialWork(bool asynch) that > uses false as a parameter. I think you should remove that parameter > (just like you did for the other functions) and remove the code that > isn't executed anymore. Good catch. I removed these occurrences of asynch as well. Here's an updated webrev: http://cr.openjdk.java.net/~brutisso/8064702/webrev.01/ And just the diff compared to the last webrev: http://cr.openjdk.java.net/~brutisso/8064702/webrev.00-01.diff/ Offline Stefan also suggested to rename the parameter to reset() from asynch to concurrent since this is the only usage left of asynch and concurrent seems like a more suitable name. This is also included in the webrev above. Thanks, Bengt > > thanks, > StefanK > >> >> https://bugs.openjdk.java.net/browse/JDK-8064702 >> >> This work is part of JEP 214 (http://openjdk.java.net/jeps/214). >> There are quite a few lines changed. Mostly it is just removal of >> code and it should hopefully be fairly straight forward to review. >> >> A couple of things are worth noting: >> >> The name "foreground collection" is used for two different things in >> CMS. Both the full GC that does a serial mark and compact of the >> whole heap (the fall back to SerialGC) and the foreground mode of >> completing a CMS cycle in a stop-the-world phase. We still need the >> fall back to the SerialGC for when we get a concurrent mode failure, >> so this change is just to remove the later feature. That is to remove >> the ability to interrupt a concurrent CMS cycle and finish it up in a >> stopped manner. >> >> This means that there is still code and comments using the word >> "foreground" even after the changes I propose. In particular the >> protocol to synchronize between the CMS collector and the MarkSweep >> serial full GC is left intact and uses locks and flags with >> foreground names. >> >> The intent of the change is to remove all traces of the actual >> foreground mode of the CMS cycle while changing as little as possible >> of the SerialGC interaction. >> >> Since there were these two modes of the foreground collector before >> many methods needed to know in which mode they were operating. This >> state was communicated by passing around parameters that were often >> called asynch and clear_all_soft_refs. Now that there is only one >> mode (the SerialGC one) I've tried to remove these parameters as much >> as possible. The interpretation being that asynch means the SerialGC >> (so always true now) and that we should always clear all soft refs >> when we do a full GC and never clear all soft refs when we do a >> concurrent CMS cycle. >> >> Thanks, >> Bengt > From erik.helin at oracle.com Wed Nov 12 18:57:37 2014 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 12 Nov 2014 19:57:37 +0100 Subject: RFR: 8064721: The card tables only ever need two covering regions Message-ID: <5463ADA1.9070804@oracle.com> Hi all, this patch removes the max_covering_regions argument from the BarrierSet constructor, since all the code using a BarrierSet only needs two covering regions. Having the maximum number of covering regions as a constant is desirable because it makes it much easier to reason about the code in all the various card tables (CardTableRS, CardTableModRefBS, G1SATBCardTableModRefBS, CardTableExtension). The only way a covering region can be added is via a call to CardTableModRefBS::find_covering_region_by_base. find_covering_region_by_base adds a region if there is no covering region already having its argument `base` as start address. The only caller to find_covering_region_by_base is CardTableModRefBS::resize_covered_region. To show that two covering regions are enough, one has to show that callers of CardTableModRefBS::resize_covered_region in different generations always pass an MemRegion argument with a start address that is always the same. This way, we only need one covering region per generation, and we only ever have two generations. There calls to CardTableModRefBS::resize_covering_region comes from (found via grep, filtered declarations, definitions and forwarding): - psYoungGen.cpp - psOldGen.cpp - asPSYoungGen.cpp - concurrentMarkSweepGeneration.cpp - generation.cpp - defNewGeneration.cpp The calls in defNewGeneration.cpp always uses _virtual_space.low() as the start address of the MemRegion argument, and _virtual_space.low() never changes. In generation.cpp, the calls from the class OneContigSpaceCardGeneration always use _the_space->bottom() as start and that won't change. The call from CardGeneration uses `start`, which is the same as rs.base(), which is the same as _virtual_space.low(), which (by looking in tenuredGeneration.cpp) is the same _the_space->bottom(). So both CardGeneration and OneContigSpaceCardGeneration use the same start address. The calls in concurrentMarkSweepGeneration.cpp uses _cmsSpace->bottom() as the start of the MemRegion and that won't change. The calls in asPSYoungGen.cpp, psOldGen.cpp and psYoungGen.cpp all uses virtual_space()->low() as the start address for the MemRegion. This address might change, but the calls from these files will reach CardTableExtension::resize_covered_region, which explicitly handles the case when the start address has changed and ensures now new covering region is added (see comments and code in cardTableExtension.cpp). The only GC that specified 2 as max_covering_regions was G1. ParallelGC specified 3, most likely because each generation needed one and PermGen needed one and this was not updated when PermGen was removed. The generation collectors specified 4, because each generation wanted one each and PermGen wanted two (one one for the main space and one for the shared spaces). The other variable that was made static const is CardTableRS::_regions_to_iterate. This was previously set to max_covered_regions - 1 by the framework collectors, so 4 - 1 = 3. I also added a comment explaining why it needs to be 3 and not 2. Webrev: http://cr.openjdk.java.net/~ehelin/8064721/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8064721 Testing: - JPRT - Running all JTReg tests locally - Ad-hoc (Window 32/64, Linux 32/64, OS X 64, Solaris x86-64/SPARC): - Dacapo, Kitchensink, Weblogic+medrec, runThese - nsk.regression, vm.gc, vm.oom, vm.parallel_class_loading, vm.regression, vm.runtime, vm.compiler, vm.runtime, vm.signal, vm.debug, nsk.sysdict + some others - nashorn regression tests - jruby quick + slow - JTReg: - hotspot - jdk_core, jdk_svc Thanks, Erik From kim.barrett at oracle.com Wed Nov 12 21:30:48 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 12 Nov 2014 16:30:48 -0500 Subject: RFR: 8064721: The card tables only ever need two covering regions In-Reply-To: <5463ADA1.9070804@oracle.com> References: <5463ADA1.9070804@oracle.com> Message-ID: <46B47E05-306E-4380-AC2F-A586AEE9BBC4@oracle.com> On Nov 12, 2014, at 1:57 PM, Erik Helin wrote: > > this patch removes the max_covering_regions argument from the BarrierSet constructor, since all the code using a BarrierSet only needs two covering regions. Having the maximum number of covering regions as a constant is desirable because it makes it much easier to reason about the code in all the various card tables (CardTableRS, CardTableModRefBS, G1SATBCardTableModRefBS, CardTableExtension). > > [?] > Webrev: > http://cr.openjdk.java.net/~ehelin/8064721/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8064721 Thanks for the thorough explanation in the covering email for this RFR. ------------------------------------------------------------------------------ src/share/vm/memory/barrierSet.hpp 52 // Some barrier sets create tables whose elements correspond to parts of 53 // the heap; the CardTableModRefBS is an example. Such barrier sets will 54 // normally reserve space for such tables, and commit parts of the table 55 // "covering" parts of the heap that are committed. At most one covered 56 // region per generation is needed. 57 static const int _max_covered_regions = 2; Is this (now) constant slated for later elimination? ------------------------------------------------------------------------------ src/share/vm/memory/cardTableRS.hpp 86 // Number of generations (including permgen). 87 static const int _regions_to_iterate = 3; And from the covering email: The other variable that was made static const is CardTableRS::_regions_to_iterate. This was previously set to max_covered_regions - 1 by the framework collectors, so 4 - 1 = 3. I also added a comment explaining why it needs to be 3 and not 2. Sorry, but I'm not finding that comment on line 86 very informative, especially in light of past removal of permgen. I don't yet understand why this isn't 1 if it used to be max_covered_regions - 1 and max_covered_regions is now a constant 2. But I suspect that value is wrong, based on its usage. Maybe it should instead be 2 because the now removed increment of max_covered_regions by 2 was really a workaround for the calculation here, and needed this to include permgen (e.g. be 3). (This actually seems kind of plausible, since the offending increment had a comment indicating it was a workaround for a needed cardtable fix.) _regions_to_iterate is used as a bound on the iteration through the _last_cur_val_in_gen table, whose size is GenCollectedHeap::max_gens+1. I'm not sure what the +1 is for; I wonder if this might be another permgen leftover. In any case, it's not apparent to me why 3 is the right value now. I think 3 will work, but I think there may be further followup work to be done in this vicinity. I would be fine with having that be spawned off as a separate task, rather than being rolled into here; the analysis of this change set is already complicated enough. But perhaps there is there a plan to change the value later, or eliminate this constant altogether? If so, it would be helpful to mention that. ------------------------------------------------------------------------------ src/share/vm/memory/modRefBarrierSet.hpp 40 ModRefBarrierSet() : BarrierSet() { _kind = BarrierSet::ModRef; } Not sure why this change was made; the default construction of BarrierSet was already implicit. Not that I object to the change, just surprised by it. If any change is going to be made here, _kind should be initialized via a member initializer rather than assignment in the constructor body. ------------------------------------------------------------------------------ src/share/vm/runtime/vmStructs.cpp removal of 476 nonstatic_field(BarrierSet, _max_covered_regions, int) \ This is removing an entry that was being made available to Java code. Are we sure there aren't any users of it? [I did a grep over a full JDK tree and found no occurrences of "max_covered_regions" outside hotspot C++ code, so there don't appear to be any Java uses in the JDK. Could there be uses outside that tree?] It might be safer to change it to a static_field, rather than removing it entirely. Hm, and does removal (or even change?) of this require CCC approval? From kim.barrett at oracle.com Thu Nov 13 00:41:20 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 12 Nov 2014 19:41:20 -0500 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <54638F87.30809@oracle.com> References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> <54638F87.30809@oracle.com> Message-ID: <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> On Nov 12, 2014, at 11:49 AM, Bengt Rutisson wrote: > > Here's an updated webrev: > http://cr.openjdk.java.net/~brutisso/8064702/webrev.01/ > > And just the diff compared to the last webrev: > http://cr.openjdk.java.net/~brutisso/8064702/webrev.00-01.diff/ ------------------------------------------------------------------------------ src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp 1150 // Does a "full" (forced) collection invoked on this generation collect 1151 // all younger generations as well? Note that the second conjunct is a 1152 // hack to allow the collection of the younger gen first if the flag is 1153 // set. 1154 virtual bool full_collects_younger_generations() const { 1155 return UseCMSCompactAtFullCollection && !ScavengeBeforeFullGC; 1156 } => 1123 virtual bool full_collects_younger_generations() const { 1124 return !ScavengeBeforeFullGC; 1125 } The complete removal of the comment is eliminating the information in the final sentence, regarding the second conjunct (e.g. the code that is being retained) being a hack. Presumably someone (long ago and lost in the mists of pre-mercurial source control) thought that test was a bit of a kludge and might deserve a rethink. Note that there is an identical comment on TenuredGeneration::full_collects_younger_generations(), and the updated implementation here is now identical to the implementation for TenuredGeneration. Keeping the associated comments consistent for these identical functions would be nice. ------------------------------------------------------------------------------ src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp 1248 // For debugging purposes, change the type of collection. 1249 // If the rotation is not on the concurrent collection 1250 // type, don't start a concurrent collection. 1251 NOT_PRODUCT( 1252 if (RotateCMSCollectionTypes) { 1253 return _cmsGen->debug_concurrent_cycle(); 1254 } 1255 ) Code was changed, but the comment looks like it needs to be updated too, since there is no longer a "concurrent collection type" in the previous sense. ------------------------------------------------------------------------------ src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp 1254 if (RotateCMSCollectionTypes && 1255 (_cmsGen->debug_collection_type() != 1256 ConcurrentMarkSweepGeneration::Concurrent_collection_type)) { 1257 assert(_cmsGen->debug_collection_type() != 1258 ConcurrentMarkSweepGeneration::Unknown_collection_type, 1259 "Bad cms collection type"); 1260 return false; 1261 } => 1252 if (RotateCMSCollectionTypes) { 1253 return _cmsGen->debug_concurrent_cycle(); 1254 } I spent a large amount of time studying this change, because there are behavioral changes here that looked suspicious. I think I've decided it's ok, but RotateCMSCollectionTypes is now badly named, and was never well described and had (and still has) unclear semantics. I think I've puzzled out more or less what it means, but that required reading a lot of code, and I'm still not confident that I know what would happen if I used it, or why I might want to. ------------------------------------------------------------------------------ From jon.masamitsu at oracle.com Thu Nov 13 04:01:00 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 12 Nov 2014 20:01:00 -0800 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> Message-ID: <54642CFC.7020609@oracle.com> On 11/10/2014 3:04 PM, Kim Barrett wrote: > On Nov 6, 2014, at 8:36 PM, Jungwoo Ha wrote: >> These are really important for performance. If we remove line 1358, >> the pause time with the given example goes up by 2x. > I wasn't suggesting eliminating the check at line 1358 altogether; > rather, I was suggesting moving it into > expand_and_ensure_spooling_space(), right before the mutex locking > that begins that function. > > There's another reason to question the check at line 1358 though, and > suggest it should be moved closer to (or inside of) > expand_and_ensure_spooling_space(). It looks like the two places that > are calling set_promotion_failure() are doing so for different > reasons. The check at line 1358 will short circuit all further > par_promote() calls in the collection cycle, even those for which > allocation would have succeeded if not for that check. For example, in > the old code, failure to promote some largish object will not > necessarily block later promotion attempts for other, smaller objects. When any promotion failure occurs the concurrent CMS collections is abandoned and a full collection is done instead. There is some cleanup that is done to get the heap back into a state where the full collection can be used. Jon > The proposed change modifies that behavior. Since the calling code > seems to be going to some effort to deal with par_promote() returning > NULL (indicating promotion failure) and keep going, it's not clear to > me whether this behavioral change is desirable. And if it *is* > desirable, it seems like a better place to do the short circuiting > might be higher up the call chain, and possibly simplifying the > handling of par_promote() returning NULL. Note that I've not yet > investigated where such a higher point in the call chain might be. > > OTOH, it seems plausible to me that keeping the test at line 1358 as > is (e.g. don't move it, which assumes we're ok with the behavioral > change discussed above) makes the test that was at 1381 (and which I > suggested moving into expand_and_par_allocate()) pretty much > redundant. It would be interesting to know whether that check at 1381 > fires more than extremely rarely. Of course, this alternative would be > contrary to my desire to have the racy flag access located as close as > possible to the lock and recheck code. > From bengt.rutisson at oracle.com Thu Nov 13 07:23:33 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 08:23:33 +0100 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> <54638F87.30809@oracle.com> <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> Message-ID: <54645C75.2030707@oracle.com> Hi Kim, Thanks for looking at this! On 2014-11-13 01:41, Kim Barrett wrote: > On Nov 12, 2014, at 11:49 AM, Bengt Rutisson wrote: >> Here's an updated webrev: >> http://cr.openjdk.java.net/~brutisso/8064702/webrev.01/ >> >> And just the diff compared to the last webrev: >> http://cr.openjdk.java.net/~brutisso/8064702/webrev.00-01.diff/ > ------------------------------------------------------------------------------ > > src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.hpp > > 1150 // Does a "full" (forced) collection invoked on this generation collect > 1151 // all younger generations as well? Note that the second conjunct is a > 1152 // hack to allow the collection of the younger gen first if the flag is > 1153 // set. > 1154 virtual bool full_collects_younger_generations() const { > 1155 return UseCMSCompactAtFullCollection && !ScavengeBeforeFullGC; > 1156 } > > => > > 1123 virtual bool full_collects_younger_generations() const { > 1124 return !ScavengeBeforeFullGC; > 1125 } > > The complete removal of the comment is eliminating the information in > the final sentence, regarding the second conjunct (e.g. the code that > is being retained) being a hack. Presumably someone (long ago and > lost in the mists of pre-mercurial source control) thought that test > was a bit of a kludge and might deserve a rethink. > > Note that there is an identical comment on > TenuredGeneration::full_collects_younger_generations(), and the updated > implementation here is now identical to the implementation for > TenuredGeneration. Keeping the associated comments consistent for > these identical functions would be nice. Good point. I added the comment back. My motivation for removing the comment in the first place was just that I figured that the name of the method was pretty much saying the same thing as the comment. But you are right about the second conjunct and it seems better to just leave the comment in. > > ------------------------------------------------------------------------------ > > src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp > > 1248 // For debugging purposes, change the type of collection. > 1249 // If the rotation is not on the concurrent collection > 1250 // type, don't start a concurrent collection. > 1251 NOT_PRODUCT( > 1252 if (RotateCMSCollectionTypes) { > 1253 return _cmsGen->debug_concurrent_cycle(); > 1254 } > 1255 ) > > Code was changed, but the comment looks like it needs to be updated > too, since there is no longer a "concurrent collection type" in the > previous sense. Good catch. I updated the comment to: // For debugging purposes, change the type of collection. // Rotate between concurrent and stop-the-world full GCs. > > ------------------------------------------------------------------------------ > > src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp > > 1254 if (RotateCMSCollectionTypes && > 1255 (_cmsGen->debug_collection_type() != > 1256 ConcurrentMarkSweepGeneration::Concurrent_collection_type)) { > 1257 assert(_cmsGen->debug_collection_type() != > 1258 ConcurrentMarkSweepGeneration::Unknown_collection_type, > 1259 "Bad cms collection type"); > 1260 return false; > 1261 } > > => > > 1252 if (RotateCMSCollectionTypes) { > 1253 return _cmsGen->debug_concurrent_cycle(); > 1254 } > > I spent a large amount of time studying this change, because there are > behavioral changes here that looked suspicious. I think I've decided > it's ok, but RotateCMSCollectionTypes is now badly named, and was > never well described and had (and still has) unclear semantics. I > think I've puzzled out more or less what it means, but that required > reading a lot of code, and I'm still not confident that I know what > would happen if I used it, or why I might want to. Right. I agree that it is unclear if this code is needed. I have never been running with RotateCMSCollectionTypes but I did not want to remove it as part of this change. As you probably noticed the change I did was to go from three states (concurrent background, stw foreground and full serial GC) to two states (concurrent background and full serial GC). Since two states can be encoded with a bool I decided to simplify a bit a get rid of the ColletionType enum. It was easier for me to verify that I didn't break this if I could do that simplification. I don't think this functionality is used much and it is only available in debug builds. So, I would suggest that we just remove it. But I prefer not to remove it as part of this change. If you want to I can send out a small pre-patch to remove the RotateCMSCollectionTypes feature. However, I think it might be a better workflow for me to keep the changes as they are in the patch and then follow up with removing RotateCMSCollectionTypes. Let me know what you prefer and I'll file a JBS bug to track it if we decide to remove it. I do agree that the naming is bad. But if we decide to remove this functionality there is not much need to spend time on the naming discussion. Updated webrev: http://cr.openjdk.java.net/~brutisso/8064702/webrev.02/ Diff compared to last webrev: http://cr.openjdk.java.net/~brutisso/8064702/webrev.01-02.diff/ Thanks again for looking at this! Bengt > > ------------------------------------------------------------------------------ > From bengt.rutisson at oracle.com Thu Nov 13 09:44:02 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 10:44:02 +0100 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <546385BB.3080801@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <54637AF5.9080600@oracle.com> <54638012.3010202@oracle.com> <546385BB.3080801@oracle.com> Message-ID: <54647D62.5050202@oracle.com> Hi Evgeniya, On 2014-11-12 17:07, Evgeniya Stepanova wrote: > Hi Bengt, > > Please see comments inline > On 12.11.2014 19:43, Bengt Rutisson wrote: >> >> On 2014-11-12 16:21, Evgeniya Stepanova wrote: >>> Hi everyone! >>> >>> Since the decision was made to change only tests which fail because >>> of conflict for now (skip "selfish" tests), I post new webrev for >>> jdk part of the JDK-8019361 >>> : >>> http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ >> >> Thanks for updating the webrev! >> >> A couple of comments: >> >> MemoryTestAllGC.sh >> >> The test is run three times, once with no params, once with >> ParallelGC and once with CMS. So, I think the @requires should just >> be vm.gc == "null". Similarly to what was done for PendingAllGC.sh. >> > The third run (with CMS) is commented. Without this run UseParallelGC > is valid option > #runOne -XX:+UseConcMarkSweepGC MemoryTest 3 > (http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/test/java/lang/management/MemoryMXBean/MemoryTestAllGC.sh.frames.html) Right. I missed that it was commented out. Thanks for pointing that out. I went back and checked. That line was already commented out when the test was first added in 2001. Do you mind removing the CMS line to avoid more people doing the same mistake that I did? I don't think we need to keep a commented out test around for more than 13 years... ;) >> >> TestInputArgument.sh >> >> The changes here seem unrelated to @requires. >> > This test was changed after conversation with David Holmes (see > thread below) Ok. Good. >> >> EnqueuePollRace.java >> >> Can you explain why it is safe to remove -XX:+UseSerialGC for this test? >> >> > This test was modified after conversation with Filipp Zhinkin and > Mandy Chung (https://bugs.openjdk.java.net/browse/JDK-8051723) Ok. >> JpsHelper.java >> >> Can you explain why it is safe to remvoe -XX:+UseParallelGC for this >> test? >> > This test was changed after conversation with Katja Kantserova (see > thread below), GC flag is just an arbitrary chosen test flag Ok. I should clearly have read the other reviews more. In that case I guess the changes look good from my perspective. Thanks, Bengt >> >> When I use Aurora to check what tests that currently are considered >> known because of JDK-8019361 I get a pretty long list: >> >> http://aurora.ru.oracle.com/functional/faces/CRRules.xhtml?cr=JDK-8019361 >> >> Are the tests in webrev.03 the only tests that still fail? Have the >> others been fixed in other ways? > There would be 2 more changes in reviews in closed part :) > > Thanks, > Evgeniya Stepanova >> >> Thanks, >> Bengt >> >> >> >>> >>> Thanks, >>> Evgeniya Stepanova >>> On 07.11.2014 15:34, Evgeniya Stepanova wrote: >>>> David, Filipp, Katja >>>> >>>> Diff have been updated one more time: >>>> java/lang/management/RuntimeMXBean/TestInputArgument.sh and >>>> test/java/lang/ref/EnqueuePollRace.java have been changed >>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >>>> >>>> Thanks >>>> >>>> On 07.11.2014 9:37, David Holmes wrote: >>>>> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>>>>> New webrev: >>>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >>>>> >>>>> In: >>>>> >>>>> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >>>>> >>>>> the use of the gc options seems incidental - it's just picking two >>>>> innocuous options to use - similar to the JpsHelper case. You >>>>> could replace +UseParallelGC with something like >>>>> +UseFastJNIAccessors (platform independent and normally true). >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>>>>> Thanks a lot! >>>>>>> >>>>>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>>>>> Hi Katja, >>>>>>>> >>>>>>>> Ok, this seems to be a perfect solution. Thank you. I'll change >>>>>>>> the >>>>>>>> diff accordingly. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>>>>> Hi Dima, >>>>>>>>> >>>>>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>>>>> Hi Katja, >>>>>>>>>> >>>>>>>>>> You are right, there will be no conflict, because test >>>>>>>>>> ignores any >>>>>>>>>> external VM flags. >>>>>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>>>>> >>>>>>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>>>>> >>>>>>>>> This particular case is to test the defined flags are shown up as >>>>>>>>> expected. >>>>>>>>> >>>>>>>>> Evgeniya, >>>>>>>>> >>>>>>>>> would you mind to change JpsHelper.java instead? >>>>>>>>> >>>>>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>>>>> @@ -93,7 +93,7 @@ >>>>>>>>> /** >>>>>>>>> * VM arguments to start test application with >>>>>>>>> */ >>>>>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>> "-XX:+PrintGCDetails"}; >>>>>>>>> /** >>>>>>>>> * VM flag to start test application with >>>>>>>>> */ >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Katja >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> @requires will allow to modify tests to include external vm >>>>>>>>>> options without any risk of bumping into conflict and extend >>>>>>>>>> area >>>>>>>>>> of test applicability. >>>>>>>>>> >>>>>>>>>> But if you still believe, that @requires is not necessary - it's >>>>>>>>>> not a problem, tests could be kept as is. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> As David has pointed out these jps tests are not testing gc. >>>>>>>>>>> The >>>>>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>>>>>> should not be any conflicts either since these tests are >>>>>>>>>>> running >>>>>>>>>>> in driver mode: >>>>>>>>>>> >>>>>>>>>>> ... >>>>>>>>>>> * @run driver TestJpsJar >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>>> which means no flags from above are accepted. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Katja >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>> Hi David, >>>>>>>>>>>> >>>>>>>>>>>> tag added because tests contain string >>>>>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>>>>> >>>>>>>>>>>> and JpsHelper defines >>>>>>>>>>>> ... >>>>>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>>>> ... >>>>>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>>>>> if (testVmArgs == null) { >>>>>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>>>>> } >>>>>>>>>>>> return testVmArgs; >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>>>>>> cleanup-if we use for example SerialGC we must be sure that >>>>>>>>>>>> tests >>>>>>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>>>>>> Parallel GC. Plus there is no any sense to run test twice >>>>>>>>>>>> in TC >>>>>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>> >>>>>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part >>>>>>>>>>>>>> of the >>>>>>>>>>>>>> JDK-8019361 >>>>>>>>>>>>>> >>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>> another GC >>>>>>>>>>>>>> is set >>>>>>>>>>>>>> outside >>>>>>>>>>>>> >>>>>>>>>>>>> I don't see why you have done this for the >>>>>>>>>>>>> >>>>>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>>>>> >>>>>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>>>>> >>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>>>>> skip test >>>>>>>>>>>>>> if there is a conflict >>>>>>>>>>>>> >>>>>>>>>>>>> Just wondering: Does a skipped test get a .jtr file >>>>>>>>>>>>> showing it >>>>>>>>>>>>> was skipped; or does it only appear in the higher-level >>>>>>>>>>>>> jtreg log? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>> -XX:+UseConcMarkSweep and >>>>>>>>>>>>>> without >>>>>>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>>>>>> failed >>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>> >>>>>>>>>>>>>> // >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >>> -- >>> /Evgeniya Stepanova/ >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Thu Nov 13 10:00:48 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Thu, 13 Nov 2014 14:00:48 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <54647D62.5050202@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <54637AF5.9080600@oracle.com> <54638012.3010202@oracle.com> <546385BB.3080801@oracle.com> <54647D62.5050202@oracle.com> Message-ID: <54648150.8070607@oracle.com> Hi Bengt! Ok, I'll remove this string. Thank you for reviewing it one more time :) Evgeniya Stepanova On 13.11.2014 13:44, Bengt Rutisson wrote: > > Hi Evgeniya, > > On 2014-11-12 17:07, Evgeniya Stepanova wrote: >> Hi Bengt, >> >> Please see comments inline >> On 12.11.2014 19:43, Bengt Rutisson wrote: >>> >>> On 2014-11-12 16:21, Evgeniya Stepanova wrote: >>>> Hi everyone! >>>> >>>> Since the decision was made to change only tests which fail because >>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>> jdk part of the JDK-8019361 >>>> : >>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ >>> >>> Thanks for updating the webrev! >>> >>> A couple of comments: >>> >>> MemoryTestAllGC.sh >>> >>> The test is run three times, once with no params, once with >>> ParallelGC and once with CMS. So, I think the @requires should just >>> be vm.gc == "null". Similarly to what was done for PendingAllGC.sh. >>> >> The third run (with CMS) is commented. Without this run UseParallelGC >> is valid option >> #runOne -XX:+UseConcMarkSweepGC MemoryTest 3 >> (http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/test/java/lang/management/MemoryMXBean/MemoryTestAllGC.sh.frames.html) > > Right. I missed that it was commented out. Thanks for pointing that out. > > I went back and checked. That line was already commented out when the > test was first added in 2001. Do you mind removing the CMS line to > avoid more people doing the same mistake that I did? I don't think we > need to keep a commented out test around for more than 13 years... ;) > >>> >>> TestInputArgument.sh >>> >>> The changes here seem unrelated to @requires. >>> >> This test was changed after conversation with David Holmes (see >> thread below) > > Ok. Good. > >>> >>> EnqueuePollRace.java >>> >>> Can you explain why it is safe to remove -XX:+UseSerialGC for this test? >>> >>> >> This test was modified after conversation with Filipp Zhinkin and >> Mandy Chung (https://bugs.openjdk.java.net/browse/JDK-8051723) > > Ok. > >>> JpsHelper.java >>> >>> Can you explain why it is safe to remvoe -XX:+UseParallelGC for this >>> test? >>> >> This test was changed after conversation with Katja Kantserova (see >> thread below), GC flag is just an arbitrary chosen test flag > > Ok. > > I should clearly have read the other reviews more. > > In that case I guess the changes look good from my perspective. > > Thanks, > Bengt > >>> >>> When I use Aurora to check what tests that currently are considered >>> known because of JDK-8019361 I get a pretty long list: >>> >>> http://aurora.ru.oracle.com/functional/faces/CRRules.xhtml?cr=JDK-8019361 >>> >>> Are the tests in webrev.03 the only tests that still fail? Have the >>> others been fixed in other ways? >> There would be 2 more changes in reviews in closed part :) >> >> Thanks, >> Evgeniya Stepanova >>> >>> Thanks, >>> Bengt >>> >>> >>> >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 07.11.2014 15:34, Evgeniya Stepanova wrote: >>>>> David, Filipp, Katja >>>>> >>>>> Diff have been updated one more time: >>>>> java/lang/management/RuntimeMXBean/TestInputArgument.sh and >>>>> test/java/lang/ref/EnqueuePollRace.java have been changed >>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >>>>> >>>>> Thanks >>>>> >>>>> On 07.11.2014 9:37, David Holmes wrote: >>>>>> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>>>>>> New webrev: >>>>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >>>>>> >>>>>> In: >>>>>> >>>>>> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >>>>>> >>>>>> the use of the gc options seems incidental - it's just picking >>>>>> two innocuous options to use - similar to the JpsHelper case. You >>>>>> could replace +UseParallelGC with something like >>>>>> +UseFastJNIAccessors (platform independent and normally true). >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>>>>>> Thanks a lot! >>>>>>>> >>>>>>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>>>>>> Hi Katja, >>>>>>>>> >>>>>>>>> Ok, this seems to be a perfect solution. Thank you. I'll >>>>>>>>> change the >>>>>>>>> diff accordingly. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>>>>>> Hi Dima, >>>>>>>>>> >>>>>>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>>>>>> Hi Katja, >>>>>>>>>>> >>>>>>>>>>> You are right, there will be no conflict, because test >>>>>>>>>>> ignores any >>>>>>>>>>> external VM flags. >>>>>>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>>>>>> >>>>>>>>>>> Ignoring external options is bad thing, such "selfish" tests >>>>>>>>>>> are >>>>>>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>>>>>> >>>>>>>>>> This particular case is to test the defined flags are shown >>>>>>>>>> up as >>>>>>>>>> expected. >>>>>>>>>> >>>>>>>>>> Evgeniya, >>>>>>>>>> >>>>>>>>>> would you mind to change JpsHelper.java instead? >>>>>>>>>> >>>>>>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>>>>>> @@ -93,7 +93,7 @@ >>>>>>>>>> /** >>>>>>>>>> * VM arguments to start test application with >>>>>>>>>> */ >>>>>>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>> "-XX:+PrintGCDetails"}; >>>>>>>>>> /** >>>>>>>>>> * VM flag to start test application with >>>>>>>>>> */ >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Katja >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> @requires will allow to modify tests to include external vm >>>>>>>>>>> options without any risk of bumping into conflict and extend >>>>>>>>>>> area >>>>>>>>>>> of test applicability. >>>>>>>>>>> >>>>>>>>>>> But if you still believe, that @requires is not necessary - >>>>>>>>>>> it's >>>>>>>>>>> not a problem, tests could be kept as is. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>> >>>>>>>>>>>> As David has pointed out these jps tests are not testing >>>>>>>>>>>> gc. The >>>>>>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. >>>>>>>>>>>> There >>>>>>>>>>>> should not be any conflicts either since these tests are >>>>>>>>>>>> running >>>>>>>>>>>> in driver mode: >>>>>>>>>>>> >>>>>>>>>>>> ... >>>>>>>>>>>> * @run driver TestJpsJar >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>>> which means no flags from above are accepted. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Katja >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> tag added because tests contain string >>>>>>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>>>>>> >>>>>>>>>>>>> and JpsHelper defines >>>>>>>>>>>>> ... >>>>>>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>>>>> ... >>>>>>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>>>>>> if (testVmArgs == null) { >>>>>>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>>>>>> } >>>>>>>>>>>>> return testVmArgs; >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added >>>>>>>>>>>>> for >>>>>>>>>>>>> cleanup-if we use for example SerialGC we must be sure >>>>>>>>>>>>> that tests >>>>>>>>>>>>> passed with this GC, not with another one. Now you will >>>>>>>>>>>>> assume >>>>>>>>>>>>> that concrete test passed with Serial GC, but it run only >>>>>>>>>>>>> with >>>>>>>>>>>>> Parallel GC. Plus there is no any sense to run test twice >>>>>>>>>>>>> in TC >>>>>>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part >>>>>>>>>>>>>>> of the >>>>>>>>>>>>>>> JDK-8019361 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>>>>>> fix: >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>> another GC >>>>>>>>>>>>>>> is set >>>>>>>>>>>>>>> outside >>>>>>>>>>>>>> >>>>>>>>>>>>>> I don't see why you have done this for the >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>>>>>> >>>>>>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>> "requires" to >>>>>>>>>>>>>>> skip test >>>>>>>>>>>>>>> if there is a conflict >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just wondering: Does a skipped test get a .jtr file >>>>>>>>>>>>>> showing it >>>>>>>>>>>>>> was skipped; or does it only appear in the higher-level >>>>>>>>>>>>>> jtreg log? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and >>>>>>>>>>>>>>> without >>>>>>>>>>>>>>> any GC flag). Tests are being excluded as expected. No >>>>>>>>>>>>>>> tests >>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> // >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /Evgeniya Stepanova/ >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Nov 13 13:32:22 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 14:32:22 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <54638A9F.2030209@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> Message-ID: <5464B2E6.9070802@oracle.com> Hi Evgeniya, On 2014-11-12 17:28, Evgeniya Stepanova wrote: > Hi Dmitry, > > You are right - I've forgotten about copyrights > Copyrights and other issues you mentioned fixed. New webrev: > http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be good to add -XX:+UseG1GC to the @run tags and then use @requires vm.gc=="G1" | vm.gc == null. The change to test/gc/defnew/HeapChangeLogging.java is unrelated to the conflicting GC combinations. Should that really be part of this changeset? The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we really need @requires for them? Otherwise it look ok to me. Bengt > > Thanks > Evgeniya Stepanova > On 12.11.2014 18:23, Dmitry Fazunenko wrote: >> Hi Evgeniya, >> >> The fix looks good to me. >> >> I noticed the following minor things: >> - copyrights need to include the year of last change >> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >> files, but doesn't contain any changes >> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >> 'prohibitedVmOptions' >> >> Thanks, >> Dima >> >> >> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>> Hi everyone! >>> >>> Since the decision was made to change only tests that fail because >>> of conflict for now (skip "selfish" tests), I post new webrev for >>> hotspot part of the JDK-8019361 >>> : >>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>> >>> Thanks, >>> Evgeniya Stepanova >>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>> Nice plan! Please feel free to send me any feedback/questions >>>> regarding @requires >>>> >>>> Thanks, >>>> Dima >>>> >>>> >>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>> >>>>> Hi Dima, >>>>> >>>>> Thanks for the answers. I think the currently proposed patch is a >>>>> good start. We will have to evolve the @requires tag in the >>>>> future, but let's have that discussion separate from this review. >>>>> And we can start that discussion later when we have more >>>>> experience with the current version of @requires. >>>>> >>>>> Thanks for doing this! >>>>> Bengt >>>>> >>>>> >>>>> >>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>> Hi Bengt, >>>>>> >>>>>> That's great that we have very closed visions! >>>>>> >>>>>> The general comment: currently, jtreg doesn't support any sort of >>>>>> plugins, so you can't provide a VM specific handler of the >>>>>> @requires or another tag. This is very annoying limitation and we >>>>>> have to live with it. >>>>>> >>>>>> A few more comments inline. >>>>>> >>>>>> >>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>> >>>>>>> >>>>>>> Hi Dima, >>>>>>> >>>>>>> Answers inline. >>>>>>> >>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>> Hi Bengt, >>>>>>>> >>>>>>>> Thanks a lot for your detailed feedback, we appreciate it very >>>>>>>> much! >>>>>>>> >>>>>>>> See comments inline. >>>>>>>> >>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi Evgeniya, >>>>>>>>> >>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part >>>>>>>>>> of the JDK-8019361 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>> >>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>>>>>> another GC. >>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>> skip test if there is a conflict >>>>>>>>> >>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>> start sorting this out. >>>>>>>>> >>>>>>>>> First a general comment. The @requires tag has been developed >>>>>>>>> without much cooperation with the GC team. We did have a lot >>>>>>>>> of feedback when it was first presented a year ago, but it >>>>>>>>> does not seem like this feedback was incorporated into the >>>>>>>>> @requires that was eventually built. >>>>>>>> >>>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>>> But not everything is possible, sorry for that. >>>>>>> >>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>> been requesting this feature for 3 years and I was expecting us >>>>>>> to be able to influence the feature much more than was the case now. >>>>>> >>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>> >>>>>>>>> Some of the points I don't really like about the @requires tag >>>>>>>>> are: >>>>>>>>> >>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible >>>>>>>> to use: >>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>> >>>>>>>> The table with all vars available in jtreg: >>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>> >>>>>>> The problem with having this matching built in to JTreg is that >>>>>>> it makes it very hard to change. When we discussed this a year >>>>>>> ago I think we said that JTreg should only provide a means to >>>>>>> test against the command line and a hook for running some java >>>>>>> code in the @requires tag. That way we could put logic like this >>>>>>> in a test library that is under our control. This would make it >>>>>>> easy for us to change and also enables us to use different logic >>>>>>> for different versions. >>>>>> >>>>>> I would be glad to have own harness... >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> - the requirement should be per @run tag. Right now we have to >>>>>>>>> do what you did in this change and use vm.gc=null even when >>>>>>>>> some tests could actually have been run when a GC was specified. >>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>> well as test case support. >>>>>>> >>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>> >>>>>> Under test case support I mean ability to treat each @run as a >>>>>> separate test. Now >>>>>> >>>>>> @test >>>>>> @run -XX:g1RegSize=1m MyTest >>>>>> @run -XX:g1RegSize=2m MyTest >>>>>> @run -XX:g1RegSize=4m MyTest >>>>>> class MyTest { >>>>>> } >>>>>> >>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> - there are many tests that require more than just a specific >>>>>>>>> GC. Often there are other flags that can't be changed either >>>>>>>>> for the test to work properly. >>>>>>>> >>>>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>>>> conflicting options. >>>>>>>> If we address this issue and we are satisfied with solution, we >>>>>>>> could move further. >>>>>>> >>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>> Personally I would have preferred that the first step was a >>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>>> I'm not too happy about how the @requires tag turned out. But >>>>>>>>> assuming we have to use it the way it is now I guess the >>>>>>>>> proposed changeset looks good. >>>>>>>> >>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>> jtreg :) >>>>>>>> And thanks for reviewing it! >>>>>>> >>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>> looked at all tests. But if they now pass with the combinations >>>>>>> that we test with I guess they should be ok. >>>>>> >>>>>> Excellent! Thanks a lot! >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>> >>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>> >>>>>>> Ok. Thanks. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> I think some of the test, like >>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>>> >>>>>>>> These two tests ignore vm flags. >>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>> execute the tests when not needed. >>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>> tests 4 times, 1 should be enough. >>>>>>> >>>>>>> Do we really want to use the @requires functionality for this >>>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>>> want the tests to be run once I think Leonid's approach with >>>>>>> tests lists seems more suitable. >>>>>> >>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>> >>>>>> >>>>>>> But are you sure that this is the reason for the @requires in >>>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>>> doesn't fail today it should probably be changed so that it does >>>>>>> fail if it is run with the wrong GC. >>>>>> >>>>>> @requires - is not the silver bullet, but it's quite easy way to >>>>>> solve a lot of issues. >>>>>> >>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>> tests, which produce a new java process to ignore vm flags coming >>>>>> from outside. No @requires, no other mechanism could 100% protect >>>>>> a test from running with conflicting options, but this is not the >>>>>> goal. >>>>>> >>>>>> If one runs tests with an exotic option, like a new G2 collector, >>>>>> there shouldn't mass failures caused by options conflicts. But a >>>>>> few failures could be handled manually. >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> Similarly it looks to me like there are tests that will fail >>>>>>>>> if you run them with -XX:-UseParallelOldGC or >>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>> >>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>> about to push a changeset that removes them: >>>>>>>>> >>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>> okay, thank for letting us know. >>>>>>>> >>>>>>>>> >>>>>>>>> Is there some way of making sure that all tests are run at one >>>>>>>>> time or another. With this change there is a risk that some >>>>>>>>> tests are never run and always skipped. Will we somehow be >>>>>>>>> tracking what gets skipped and make sure that all tests are at >>>>>>>>> least run once with the correct GC so that it is not skipped >>>>>>>>> all the time? >>>>>>>> >>>>>>>> This is a very good question! >>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>> soon, after getting fix of: >>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>> >>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>> >>>>>>>> Other examples: >>>>>>>> >>>>>>>> /* >>>>>>>> *@ignore >>>>>>>> *@test >>>>>>>> */ >>>>>>>> ... >>>>>>>> >>>>>>>> /*@bug 4445555 >>>>>>>> *@test >>>>>>>> */ >>>>>>>> ... >>>>>>>> Such tests will never be run, because jtreg treats as test only >>>>>>>> files with @test on the first place... >>>>>>>> >>>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>>> very actively). But this subject for another discussion, not >>>>>>>> within RFR :) >>>>>>> >>>>>>> Right. Glad to hear that you are actively working on this! >>>>>> >>>>>> I was going to say "not very actively", but never mind, we know >>>>>> about this problem. With introducing @requires mechanism it will >>>>>> become more important! >>>>>> >>>>>> >>>>>> Thanks for your comments! >>>>>> >>>>>> -- Dima >>>>>> >>>>>> >>>>>>> >>>>>>> Bengt >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dima >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> -- >>> /Evgeniya Stepanova/ >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Thu Nov 13 12:49:49 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 13 Nov 2014 16:49:49 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B2E6.9070802@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> Message-ID: <5464A8ED.4070708@oracle.com> On 13.11.2014 17:32, Bengt Rutisson wrote: > > Hi Evgeniya, > > On 2014-11-12 17:28, Evgeniya Stepanova wrote: >> Hi Dmitry, >> >> You are right - I've forgotten about copyrights >> Copyrights and other issues you mentioned fixed. New webrev: >> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ > > > For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be > good to add -XX:+UseG1GC to the @run tags and then use @requires > vm.gc=="G1" | vm.gc == null. > > > The change to test/gc/defnew/HeapChangeLogging.java is unrelated to > the conflicting GC combinations. Should that really be part of this > changeset? > > > The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we > really need @requires for them? Yes, we do. These tests use TestShrinkAuxiliaryData class which implements its own mechanism to analyze VM options an skip if not applicable collector is given. @requires - allows to rely on jtreg. Driver mode is a kind of indicator, that the test will spawn its own java process. Thanks Dima > > > Otherwise it look ok to me. > > Bengt > > >> >> Thanks >> Evgeniya Stepanova >> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>> Hi Evgeniya, >>> >>> The fix looks good to me. >>> >>> I noticed the following minor things: >>> - copyrights need to include the year of last change >>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>> files, but doesn't contain any changes >>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >>> 'prohibitedVmOptions' >>> >>> Thanks, >>> Dima >>> >>> >>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>> Hi everyone! >>>> >>>> Since the decision was made to change only tests that fail because >>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>> hotspot part of the JDK-8019361 >>>> : >>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>> Nice plan! Please feel free to send me any feedback/questions >>>>> regarding @requires >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Dima, >>>>>> >>>>>> Thanks for the answers. I think the currently proposed patch is a >>>>>> good start. We will have to evolve the @requires tag in the >>>>>> future, but let's have that discussion separate from this review. >>>>>> And we can start that discussion later when we have more >>>>>> experience with the current version of @requires. >>>>>> >>>>>> Thanks for doing this! >>>>>> Bengt >>>>>> >>>>>> >>>>>> >>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> That's great that we have very closed visions! >>>>>>> >>>>>>> The general comment: currently, jtreg doesn't support any sort >>>>>>> of plugins, so you can't provide a VM specific handler of the >>>>>>> @requires or another tag. This is very annoying limitation and >>>>>>> we have to live with it. >>>>>>> >>>>>>> A few more comments inline. >>>>>>> >>>>>>> >>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi Dima, >>>>>>>> >>>>>>>> Answers inline. >>>>>>>> >>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>> Hi Bengt, >>>>>>>>> >>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it very >>>>>>>>> much! >>>>>>>>> >>>>>>>>> See comments inline. >>>>>>>>> >>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part >>>>>>>>>>> of the JDK-8019361 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>> set another GC. >>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>> skip test if there is a conflict >>>>>>>>>> >>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>> start sorting this out. >>>>>>>>>> >>>>>>>>>> First a general comment. The @requires tag has been developed >>>>>>>>>> without much cooperation with the GC team. We did have a lot >>>>>>>>>> of feedback when it was first presented a year ago, but it >>>>>>>>>> does not seem like this feedback was incorporated into the >>>>>>>>>> @requires that was eventually built. >>>>>>>>> >>>>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>>>> But not everything is possible, sorry for that. >>>>>>>> >>>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>>> been requesting this feature for 3 years and I was expecting us >>>>>>>> to be able to influence the feature much more than was the case >>>>>>>> now. >>>>>>> >>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>>> >>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>> tag are: >>>>>>>>>> >>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible >>>>>>>>> to use: >>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>> >>>>>>>>> The table with all vars available in jtreg: >>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>> >>>>>>>> The problem with having this matching built in to JTreg is that >>>>>>>> it makes it very hard to change. When we discussed this a year >>>>>>>> ago I think we said that JTreg should only provide a means to >>>>>>>> test against the command line and a hook for running some java >>>>>>>> code in the @requires tag. That way we could put logic like >>>>>>>> this in a test library that is under our control. This would >>>>>>>> make it easy for us to change and also enables us to use >>>>>>>> different logic for different versions. >>>>>>> >>>>>>> I would be glad to have own harness... >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> - the requirement should be per @run tag. Right now we have >>>>>>>>>> to do what you did in this change and use vm.gc=null even >>>>>>>>>> when some tests could actually have been run when a GC was >>>>>>>>>> specified. >>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>> well as test case support. >>>>>>>> >>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>> >>>>>>> Under test case support I mean ability to treat each @run as a >>>>>>> separate test. Now >>>>>>> >>>>>>> @test >>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>> class MyTest { >>>>>>> } >>>>>>> >>>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> - there are many tests that require more than just a specific >>>>>>>>>> GC. Often there are other flags that can't be changed either >>>>>>>>>> for the test to work properly. >>>>>>>>> >>>>>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>>>>> conflicting options. >>>>>>>>> If we address this issue and we are satisfied with solution, >>>>>>>>> we could move further. >>>>>>>> >>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>> Personally I would have preferred that the first step was a >>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>>>> I'm not too happy about how the @requires tag turned out. But >>>>>>>>>> assuming we have to use it the way it is now I guess the >>>>>>>>>> proposed changeset looks good. >>>>>>>>> >>>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>>> jtreg :) >>>>>>>>> And thanks for reviewing it! >>>>>>>> >>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>>> looked at all tests. But if they now pass with the combinations >>>>>>>> that we test with I guess they should be ok. >>>>>>> >>>>>>> Excellent! Thanks a lot! >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>> >>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>> >>>>>>>> Ok. Thanks. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think some of the test, like >>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>>>> >>>>>>>>> These two tests ignore vm flags. >>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>> execute the tests when not needed. >>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>>> tests 4 times, 1 should be enough. >>>>>>>> >>>>>>>> Do we really want to use the @requires functionality for this >>>>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>>>> want the tests to be run once I think Leonid's approach with >>>>>>>> tests lists seems more suitable. >>>>>>> >>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>> >>>>>>> >>>>>>>> But are you sure that this is the reason for the @requires in >>>>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>>>> doesn't fail today it should probably be changed so that it >>>>>>>> does fail if it is run with the wrong GC. >>>>>>> >>>>>>> @requires - is not the silver bullet, but it's quite easy way to >>>>>>> solve a lot of issues. >>>>>>> >>>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>>> tests, which produce a new java process to ignore vm flags >>>>>>> coming from outside. No @requires, no other mechanism could 100% >>>>>>> protect a test from running with conflicting options, but this >>>>>>> is not the goal. >>>>>>> >>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>> collector, there shouldn't mass failures caused by options >>>>>>> conflicts. But a few failures could be handled manually. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> Similarly it looks to me like there are tests that will fail >>>>>>>>>> if you run them with -XX:-UseParallelOldGC or >>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>> >>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>> >>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>> okay, thank for letting us know. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>> some tests are never run and always skipped. Will we somehow >>>>>>>>>> be tracking what gets skipped and make sure that all tests >>>>>>>>>> are at least run once with the correct GC so that it is not >>>>>>>>>> skipped all the time? >>>>>>>>> >>>>>>>>> This is a very good question! >>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>>> soon, after getting fix of: >>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>> >>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>> >>>>>>>>> Other examples: >>>>>>>>> >>>>>>>>> /* >>>>>>>>> *@ignore >>>>>>>>> *@test >>>>>>>>> */ >>>>>>>>> ... >>>>>>>>> >>>>>>>>> /*@bug 4445555 >>>>>>>>> *@test >>>>>>>>> */ >>>>>>>>> ... >>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>> only files with @test on the first place... >>>>>>>>> >>>>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>>>> very actively). But this subject for another discussion, not >>>>>>>>> within RFR :) >>>>>>>> >>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>> >>>>>>> I was going to say "not very actively", but never mind, we know >>>>>>> about this problem. With introducing @requires mechanism it will >>>>>>> become more important! >>>>>>> >>>>>>> >>>>>>> Thanks for your comments! >>>>>>> >>>>>>> -- Dima >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Bengt >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Nov 13 13:42:11 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 14:42:11 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464A8ED.4070708@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> Message-ID: <5464B533.9000100@oracle.com> On 2014-11-13 13:49, Dmitry Fazunenko wrote: > > On 13.11.2014 17:32, Bengt Rutisson wrote: >> >> Hi Evgeniya, >> >> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>> Hi Dmitry, >>> >>> You are right - I've forgotten about copyrights >>> Copyrights and other issues you mentioned fixed. New webrev: >>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >> >> >> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be >> good to add -XX:+UseG1GC to the @run tags and then use @requires >> vm.gc=="G1" | vm.gc == null. >> >> >> The change to test/gc/defnew/HeapChangeLogging.java is unrelated to >> the conflicting GC combinations. Should that really be part of this >> changeset? >> >> >> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >> really need @requires for them? > > Yes, we do. > These tests use TestShrinkAuxiliaryData class which implements its own > mechanism to analyze VM options an skip if not applicable collector is > given. @requires - allows to rely on jtreg. > > Driver mode is a kind of indicator, that the test will spawn its own > java process. I thought the point of @driver was that no external vmoptions were passed to such a test. Is that not the case? Bengt > > Thanks > Dima > >> >> >> Otherwise it look ok to me. >> >> Bengt >> >> >>> >>> Thanks >>> Evgeniya Stepanova >>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>> Hi Evgeniya, >>>> >>>> The fix looks good to me. >>>> >>>> I noticed the following minor things: >>>> - copyrights need to include the year of last change >>>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>>> files, but doesn't contain any changes >>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >>>> 'prohibitedVmOptions' >>>> >>>> Thanks, >>>> Dima >>>> >>>> >>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>> Hi everyone! >>>>> >>>>> Since the decision was made to change only tests that fail because >>>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>>> hotspot part of the JDK-8019361 >>>>> : >>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>> regarding @requires >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Dima, >>>>>>> >>>>>>> Thanks for the answers. I think the currently proposed patch is >>>>>>> a good start. We will have to evolve the @requires tag in the >>>>>>> future, but let's have that discussion separate from this >>>>>>> review. And we can start that discussion later when we have more >>>>>>> experience with the current version of @requires. >>>>>>> >>>>>>> Thanks for doing this! >>>>>>> Bengt >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>> Hi Bengt, >>>>>>>> >>>>>>>> That's great that we have very closed visions! >>>>>>>> >>>>>>>> The general comment: currently, jtreg doesn't support any sort >>>>>>>> of plugins, so you can't provide a VM specific handler of the >>>>>>>> @requires or another tag. This is very annoying limitation and >>>>>>>> we have to live with it. >>>>>>>> >>>>>>>> A few more comments inline. >>>>>>>> >>>>>>>> >>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Dima, >>>>>>>>> >>>>>>>>> Answers inline. >>>>>>>>> >>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>> Hi Bengt, >>>>>>>>>> >>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>> very much! >>>>>>>>>> >>>>>>>>>> See comments inline. >>>>>>>>>> >>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part >>>>>>>>>>>> of the JDK-8019361 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>>> set another GC. >>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" >>>>>>>>>>>> to skip test if there is a conflict >>>>>>>>>>> >>>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>>> start sorting this out. >>>>>>>>>>> >>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>> developed without much cooperation with the GC team. We did >>>>>>>>>>> have a lot of feedback when it was first presented a year >>>>>>>>>>> ago, but it does not seem like this feedback was >>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>> >>>>>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>>>>> But not everything is possible, sorry for that. >>>>>>>>> >>>>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>>>> been requesting this feature for 3 years and I was expecting >>>>>>>>> us to be able to influence the feature much more than was the >>>>>>>>> case now. >>>>>>>> >>>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>>>> >>>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>>> tag are: >>>>>>>>>>> >>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>> possible to use: >>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>> >>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>> >>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>> that it makes it very hard to change. When we discussed this a >>>>>>>>> year ago I think we said that JTreg should only provide a >>>>>>>>> means to test against the command line and a hook for running >>>>>>>>> some java code in the @requires tag. That way we could put >>>>>>>>> logic like this in a test library that is under our control. >>>>>>>>> This would make it easy for us to change and also enables us >>>>>>>>> to use different logic for different versions. >>>>>>>> >>>>>>>> I would be glad to have own harness... >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> - the requirement should be per @run tag. Right now we have >>>>>>>>>>> to do what you did in this change and use vm.gc=null even >>>>>>>>>>> when some tests could actually have been run when a GC was >>>>>>>>>>> specified. >>>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>>> well as test case support. >>>>>>>>> >>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>> >>>>>>>> Under test case support I mean ability to treat each @run as a >>>>>>>> separate test. Now >>>>>>>> >>>>>>>> @test >>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>> class MyTest { >>>>>>>> } >>>>>>>> >>>>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>> >>>>>>>>>> yes. conflicting GC is just the most popular problem caused >>>>>>>>>> by conflicting options. >>>>>>>>>> If we address this issue and we are satisfied with solution, >>>>>>>>>> we could move further. >>>>>>>>> >>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>> Personally I would have preferred that the first step was a >>>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>>>>> I'm not too happy about how the @requires tag turned out. >>>>>>>>>>> But assuming we have to use it the way it is now I guess the >>>>>>>>>>> proposed changeset looks good. >>>>>>>>>> >>>>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>>>> jtreg :) >>>>>>>>>> And thanks for reviewing it! >>>>>>>>> >>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>>>> looked at all tests. But if they now pass with the >>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>> >>>>>>>> Excellent! Thanks a lot! >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>> >>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>> >>>>>>>>> Ok. Thanks. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think some of the test, like >>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if >>>>>>>>>>> you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if >>>>>>>>>>> you run with -XX:-UseParNewGC. Could you test these two >>>>>>>>>>> cases too? >>>>>>>>>> >>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>>> execute the tests when not needed. >>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>>>> tests 4 times, 1 should be enough. >>>>>>>>> >>>>>>>>> Do we really want to use the @requires functionality for this >>>>>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>>>>> want the tests to be run once I think Leonid's approach with >>>>>>>>> tests lists seems more suitable. >>>>>>>> >>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>> >>>>>>>> >>>>>>>>> But are you sure that this is the reason for the @requires in >>>>>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>>>>> doesn't fail today it should probably be changed so that it >>>>>>>>> does fail if it is run with the wrong GC. >>>>>>>> >>>>>>>> @requires - is not the silver bullet, but it's quite easy way >>>>>>>> to solve a lot of issues. >>>>>>>> >>>>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>>>> tests, which produce a new java process to ignore vm flags >>>>>>>> coming from outside. No @requires, no other mechanism could >>>>>>>> 100% protect a test from running with conflicting options, but >>>>>>>> this is not the goal. >>>>>>>> >>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Similarly it looks to me like there are tests that will fail >>>>>>>>>>> if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>> >>>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>>> >>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>> okay, thank for letting us know. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>>> some tests are never run and always skipped. Will we somehow >>>>>>>>>>> be tracking what gets skipped and make sure that all tests >>>>>>>>>>> are at least run once with the correct GC so that it is not >>>>>>>>>>> skipped all the time? >>>>>>>>>> >>>>>>>>>> This is a very good question! >>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>>>> soon, after getting fix of: >>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>> >>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>> >>>>>>>>>> Other examples: >>>>>>>>>> >>>>>>>>>> /* >>>>>>>>>> *@ignore >>>>>>>>>> *@test >>>>>>>>>> */ >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>>> /*@bug 4445555 >>>>>>>>>> *@test >>>>>>>>>> */ >>>>>>>>>> ... >>>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>>> only files with @test on the first place... >>>>>>>>>> >>>>>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>>>>> very actively). But this subject for another discussion, not >>>>>>>>>> within RFR :) >>>>>>>>> >>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>> >>>>>>>> I was going to say "not very actively", but never mind, we know >>>>>>>> about this problem. With introducing @requires mechanism it >>>>>>>> will become more important! >>>>>>>> >>>>>>>> >>>>>>>> Thanks for your comments! >>>>>>>> >>>>>>>> -- Dima >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>> >>> -- >>> /Evgeniya Stepanova/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Thu Nov 13 12:56:35 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 13 Nov 2014 16:56:35 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B533.9000100@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> Message-ID: <5464AA83.4030306@oracle.com> On 13.11.2014 17:42, Bengt Rutisson wrote: > > On 2014-11-13 13:49, Dmitry Fazunenko wrote: >> >> On 13.11.2014 17:32, Bengt Rutisson wrote: >>> >>> Hi Evgeniya, >>> >>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>> Hi Dmitry, >>>> >>>> You are right - I've forgotten about copyrights >>>> Copyrights and other issues you mentioned fixed. New webrev: >>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>> >>> >>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be >>> good to add -XX:+UseG1GC to the @run tags and then use @requires >>> vm.gc=="G1" | vm.gc == null. >>> >>> >>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated to >>> the conflicting GC combinations. Should that really be part of this >>> changeset? >>> >>> >>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >>> really need @requires for them? >> >> Yes, we do. >> These tests use TestShrinkAuxiliaryData class which implements its >> own mechanism to analyze VM options an skip if not applicable >> collector is given. @requires - allows to rely on jtreg. >> >> Driver mode is a kind of indicator, that the test will spawn its own >> java process. > > I thought the point of @driver was that no external vmoptions were > passed to such a test. Is that not the case? In the driver mode VM is started without external VM flags. Those flags are passed to the tests via system property. The driver mode is a sort of shell to start something else. -- Dima > > Bengt > >> >> Thanks >> Dima >> >>> >>> >>> Otherwise it look ok to me. >>> >>> Bengt >>> >>> >>>> >>>> Thanks >>>> Evgeniya Stepanova >>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>> Hi Evgeniya, >>>>> >>>>> The fix looks good to me. >>>>> >>>>> I noticed the following minor things: >>>>> - copyrights need to include the year of last change >>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>>>> files, but doesn't contain any changes >>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >>>>> 'prohibitedVmOptions' >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>> Hi everyone! >>>>>> >>>>>> Since the decision was made to change only tests that fail >>>>>> because of conflict for now (skip "selfish" tests), I post new >>>>>> webrev for hotspot part of the JDK-8019361 >>>>>> : >>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>>> regarding @requires >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Dima, >>>>>>>> >>>>>>>> Thanks for the answers. I think the currently proposed patch is >>>>>>>> a good start. We will have to evolve the @requires tag in the >>>>>>>> future, but let's have that discussion separate from this >>>>>>>> review. And we can start that discussion later when we have >>>>>>>> more experience with the current version of @requires. >>>>>>>> >>>>>>>> Thanks for doing this! >>>>>>>> Bengt >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>> Hi Bengt, >>>>>>>>> >>>>>>>>> That's great that we have very closed visions! >>>>>>>>> >>>>>>>>> The general comment: currently, jtreg doesn't support any sort >>>>>>>>> of plugins, so you can't provide a VM specific handler of the >>>>>>>>> @requires or another tag. This is very annoying limitation and >>>>>>>>> we have to live with it. >>>>>>>>> >>>>>>>>> A few more comments inline. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Dima, >>>>>>>>>> >>>>>>>>>> Answers inline. >>>>>>>>>> >>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>>> very much! >>>>>>>>>>> >>>>>>>>>>> See comments inline. >>>>>>>>>>> >>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>> >>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>>>> set another GC. >>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" >>>>>>>>>>>>> to skip test if there is a conflict >>>>>>>>>>>> >>>>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>>>> start sorting this out. >>>>>>>>>>>> >>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>> developed without much cooperation with the GC team. We did >>>>>>>>>>>> have a lot of feedback when it was first presented a year >>>>>>>>>>>> ago, but it does not seem like this feedback was >>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>> >>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>> >>>>>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>>>>> been requesting this feature for 3 years and I was expecting >>>>>>>>>> us to be able to influence the feature much more than was the >>>>>>>>>> case now. >>>>>>>>> >>>>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>>>>> >>>>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>>>> tag are: >>>>>>>>>>>> >>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>> possible to use: >>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>> >>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>> >>>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>>> that it makes it very hard to change. When we discussed this >>>>>>>>>> a year ago I think we said that JTreg should only provide a >>>>>>>>>> means to test against the command line and a hook for running >>>>>>>>>> some java code in the @requires tag. That way we could put >>>>>>>>>> logic like this in a test library that is under our control. >>>>>>>>>> This would make it easy for us to change and also enables us >>>>>>>>>> to use different logic for different versions. >>>>>>>>> >>>>>>>>> I would be glad to have own harness... >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> - the requirement should be per @run tag. Right now we have >>>>>>>>>>>> to do what you did in this change and use vm.gc=null even >>>>>>>>>>>> when some tests could actually have been run when a GC was >>>>>>>>>>>> specified. >>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>>>> well as test case support. >>>>>>>>>> >>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>> >>>>>>>>> Under test case support I mean ability to treat each @run as a >>>>>>>>> separate test. Now >>>>>>>>> >>>>>>>>> @test >>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>> class MyTest { >>>>>>>>> } >>>>>>>>> >>>>>>>>> is always a single test. You can't exclude, or re-run a part >>>>>>>>> of it. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>> >>>>>>>>>>> yes. conflicting GC is just the most popular problem caused >>>>>>>>>>> by conflicting options. >>>>>>>>>>> If we address this issue and we are satisfied with solution, >>>>>>>>>>> we could move further. >>>>>>>>>> >>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>> Personally I would have preferred that the first step was a >>>>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>>> implementation of the @requires tag. I just want to say >>>>>>>>>>>> that I'm not too happy about how the @requires tag turned >>>>>>>>>>>> out. But assuming we have to use it the way it is now I >>>>>>>>>>>> guess the proposed changeset looks good. >>>>>>>>>>> >>>>>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>>>>> jtreg :) >>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>> >>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>>>>> looked at all tests. But if they now pass with the >>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>> >>>>>>>>> Excellent! Thanks a lot! >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests are >>>>>>>>>>>>> being excluded as expected. No tests failed because of the >>>>>>>>>>>>> conflict. >>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>> >>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>>> >>>>>>>>>> Ok. Thanks. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if >>>>>>>>>>>> you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if >>>>>>>>>>>> you run with -XX:-UseParNewGC. Could you test these two >>>>>>>>>>>> cases too? >>>>>>>>>>> >>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>>>> execute the tests when not needed. >>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>>>>> tests 4 times, 1 should be enough. >>>>>>>>>> >>>>>>>>>> Do we really want to use the @requires functionality for this >>>>>>>>>> purpose? It seems like a way of misusing @requires. If we >>>>>>>>>> just want the tests to be run once I think Leonid's approach >>>>>>>>>> with tests lists seems more suitable. >>>>>>>>> >>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>> >>>>>>>>> >>>>>>>>>> But are you sure that this is the reason for the @requires in >>>>>>>>>> this case? TestDefNewCMS does sound like a test that is >>>>>>>>>> DefNew specific. I don't see a reason to run it with ParNew. >>>>>>>>>> If it doesn't fail today it should probably be changed so >>>>>>>>>> that it does fail if it is run with the wrong GC. >>>>>>>>> >>>>>>>>> @requires - is not the silver bullet, but it's quite easy way >>>>>>>>> to solve a lot of issues. >>>>>>>>> >>>>>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>>>>> tests, which produce a new java process to ignore vm flags >>>>>>>>> coming from outside. No @requires, no other mechanism could >>>>>>>>> 100% protect a test from running with conflicting options, but >>>>>>>>> this is not the goal. >>>>>>>>> >>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>> >>>>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>>>> >>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>>>> some tests are never run and always skipped. Will we >>>>>>>>>>>> somehow be tracking what gets skipped and make sure that >>>>>>>>>>>> all tests are at least run once with the correct GC so that >>>>>>>>>>>> it is not skipped all the time? >>>>>>>>>>> >>>>>>>>>>> This is a very good question! >>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>>>>> soon, after getting fix of: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>> >>>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>> >>>>>>>>>>> Other examples: >>>>>>>>>>> >>>>>>>>>>> /* >>>>>>>>>>> *@ignore >>>>>>>>>>> *@test >>>>>>>>>>> */ >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>> *@test >>>>>>>>>>> */ >>>>>>>>>>> ... >>>>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>>>> only files with @test on the first place... >>>>>>>>>>> >>>>>>>>>>> So, making sure that tests do not disappear is important >>>>>>>>>>> SQE task, we know about that, we're thinking on solution >>>>>>>>>>> (may be very actively). But this subject for another >>>>>>>>>>> discussion, not within RFR :) >>>>>>>>>> >>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>> >>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>> know about this problem. With introducing @requires mechanism >>>>>>>>> it will become more important! >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks for your comments! >>>>>>>>> >>>>>>>>> -- Dima >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Nov 13 13:59:34 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 14:59:34 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464AA83.4030306@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> Message-ID: <5464B946.90806@oracle.com> On 2014-11-13 13:56, Dmitry Fazunenko wrote: > > On 13.11.2014 17:42, Bengt Rutisson wrote: >> >> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>> >>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>> >>>> Hi Evgeniya, >>>> >>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>> Hi Dmitry, >>>>> >>>>> You are right - I've forgotten about copyrights >>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>> >>>> >>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would >>>> be good to add -XX:+UseG1GC to the @run tags and then use >>>> @requires vm.gc=="G1" | vm.gc == null. >>>> >>>> >>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated to >>>> the conflicting GC combinations. Should that really be part of this >>>> changeset? >>>> >>>> >>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >>>> really need @requires for them? >>> >>> Yes, we do. >>> These tests use TestShrinkAuxiliaryData class which implements its >>> own mechanism to analyze VM options an skip if not applicable >>> collector is given. @requires - allows to rely on jtreg. >>> >>> Driver mode is a kind of indicator, that the test will spawn its own >>> java process. >> >> I thought the point of @driver was that no external vmoptions were >> passed to such a test. Is that not the case? > > In the driver mode VM is started without external VM flags. Those > flags are passed to the tests via system property. > The driver mode is a sort of shell to start something else. Right. So, why would you need @requires on the TestShrinkAuxiliaryDataXX tests because the utility TestShrinkAuxiliaryData picks up the vm flags through Utils.getTestJavaOpts(). What's the point in running this in a driver mode when they anyway pick up the vm flags? I'm asking because adding @requires to these tests means that we will run them less often than we do now. So, I'm a bit worried that we reduce the amount of testing we do. Bengt > > -- Dima > > >> >> Bengt >> >>> >>> Thanks >>> Dima >>> >>>> >>>> >>>> Otherwise it look ok to me. >>>> >>>> Bengt >>>> >>>> >>>>> >>>>> Thanks >>>>> Evgeniya Stepanova >>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>> Hi Evgeniya, >>>>>> >>>>>> The fix looks good to me. >>>>>> >>>>>> I noticed the following minor things: >>>>>> - copyrights need to include the year of last change >>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>>>>> files, but doesn't contain any changes >>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>> variable 'prohibitedVmOptions' >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>> Hi everyone! >>>>>>> >>>>>>> Since the decision was made to change only tests that fail >>>>>>> because of conflict for now (skip "selfish" tests), I post new >>>>>>> webrev for hotspot part of the JDK-8019361 >>>>>>> : >>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>>>> regarding @requires >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dima >>>>>>>> >>>>>>>> >>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi Dima, >>>>>>>>> >>>>>>>>> Thanks for the answers. I think the currently proposed patch >>>>>>>>> is a good start. We will have to evolve the @requires tag in >>>>>>>>> the future, but let's have that discussion separate from this >>>>>>>>> review. And we can start that discussion later when we have >>>>>>>>> more experience with the current version of @requires. >>>>>>>>> >>>>>>>>> Thanks for doing this! >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>> Hi Bengt, >>>>>>>>>> >>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>> >>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>> sort of plugins, so you can't provide a VM specific handler >>>>>>>>>> of the @requires or another tag. This is very annoying >>>>>>>>>> limitation and we have to live with it. >>>>>>>>>> >>>>>>>>>> A few more comments inline. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Dima, >>>>>>>>>>> >>>>>>>>>>> Answers inline. >>>>>>>>>>> >>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>> >>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>>>> very much! >>>>>>>>>>>> >>>>>>>>>>>> See comments inline. >>>>>>>>>>>> >>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>> >>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>>>>> set another GC. >>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" >>>>>>>>>>>>>> to skip test if there is a conflict >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>>>>> start sorting this out. >>>>>>>>>>>>> >>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>> developed without much cooperation with the GC team. We >>>>>>>>>>>>> did have a lot of feedback when it was first presented a >>>>>>>>>>>>> year ago, but it does not seem like this feedback was >>>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>>> >>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>> >>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>> expecting us to be able to influence the feature much more >>>>>>>>>>> than was the case now. >>>>>>>>>> >>>>>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>>>> forward and I won't object to it. But I am pretty >>>>>>>>>>>>> convinced that we will soon run in to the limitations of >>>>>>>>>>>>> the current @requires implementation and we will have to >>>>>>>>>>>>> redo this work. >>>>>>>>>>>>> >>>>>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>>>>> tag are: >>>>>>>>>>>>> >>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. >>>>>>>>>>>>> It would have been better to just "require" any command >>>>>>>>>>>>> line flag. >>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>> possible to use: >>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>> >>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>> >>>>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>>>> that it makes it very hard to change. When we discussed this >>>>>>>>>>> a year ago I think we said that JTreg should only provide a >>>>>>>>>>> means to test against the command line and a hook for >>>>>>>>>>> running some java code in the @requires tag. That way we >>>>>>>>>>> could put logic like this in a test library that is under >>>>>>>>>>> our control. This would make it easy for us to change and >>>>>>>>>>> also enables us to use different logic for different versions. >>>>>>>>>> >>>>>>>>>> I would be glad to have own harness... >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>> have to do what you did in this change and use vm.gc=null >>>>>>>>>>>>> even when some tests could actually have been run when a >>>>>>>>>>>>> GC was specified. >>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>>>>> well as test case support. >>>>>>>>>>> >>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> Under test case support I mean ability to treat each @run as >>>>>>>>>> a separate test. Now >>>>>>>>>> >>>>>>>>>> @test >>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>> class MyTest { >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> is always a single test. You can't exclude, or re-run a part >>>>>>>>>> of it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>> >>>>>>>>>>>> yes. conflicting GC is just the most popular problem caused >>>>>>>>>>>> by conflicting options. >>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>> solution, we could move further. >>>>>>>>>>> >>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>> Personally I would have preferred that the first step was a >>>>>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>>>> implementation of the @requires tag. I just want to say >>>>>>>>>>>>> that I'm not too happy about how the @requires tag turned >>>>>>>>>>>>> out. But assuming we have to use it the way it is now I >>>>>>>>>>>>> guess the proposed changeset looks good. >>>>>>>>>>>> >>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>> about jtreg :) >>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>> >>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have >>>>>>>>>>> not looked at all tests. But if they now pass with the >>>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>>> >>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests are >>>>>>>>>>>>>> being excluded as expected. No tests failed because of >>>>>>>>>>>>>> the conflict. >>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>> >>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>>>> >>>>>>>>>>> Ok. Thanks. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if >>>>>>>>>>>>> you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if >>>>>>>>>>>>> you run with -XX:-UseParNewGC. Could you test these two >>>>>>>>>>>>> cases too? >>>>>>>>>>>> >>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>>>>> execute the tests when not needed. >>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>> >>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>> this purpose? It seems like a way of misusing @requires. If >>>>>>>>>>> we just want the tests to be run once I think Leonid's >>>>>>>>>>> approach with tests lists seems more suitable. >>>>>>>>>> >>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> But are you sure that this is the reason for the @requires >>>>>>>>>>> in this case? TestDefNewCMS does sound like a test that is >>>>>>>>>>> DefNew specific. I don't see a reason to run it with ParNew. >>>>>>>>>>> If it doesn't fail today it should probably be changed so >>>>>>>>>>> that it does fail if it is run with the wrong GC. >>>>>>>>>> >>>>>>>>>> @requires - is not the silver bullet, but it's quite easy way >>>>>>>>>> to solve a lot of issues. >>>>>>>>>> >>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>> "selfish" tests, which produce a new java process to ignore >>>>>>>>>> vm flags coming from outside. No @requires, no other >>>>>>>>>> mechanism could 100% protect a test from running with >>>>>>>>>> conflicting options, but this is not the goal. >>>>>>>>>> >>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>>> >>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>>>>> >>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>>>>> some tests are never run and always skipped. Will we >>>>>>>>>>>>> somehow be tracking what gets skipped and make sure that >>>>>>>>>>>>> all tests are at least run once with the correct GC so >>>>>>>>>>>>> that it is not skipped all the time? >>>>>>>>>>>> >>>>>>>>>>>> This is a very good question! >>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will >>>>>>>>>>>> do soon, after getting fix of: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>> >>>>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>> >>>>>>>>>>>> Other examples: >>>>>>>>>>>> >>>>>>>>>>>> /* >>>>>>>>>>>> *@ignore >>>>>>>>>>>> *@test >>>>>>>>>>>> */ >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>> *@test >>>>>>>>>>>> */ >>>>>>>>>>>> ... >>>>>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>>>>> only files with @test on the first place... >>>>>>>>>>>> >>>>>>>>>>>> So, making sure that tests do not disappear is important >>>>>>>>>>>> SQE task, we know about that, we're thinking on solution >>>>>>>>>>>> (may be very actively). But this subject for another >>>>>>>>>>>> discussion, not within RFR :) >>>>>>>>>>> >>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>> >>>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>>> know about this problem. With introducing @requires mechanism >>>>>>>>>> it will become more important! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks for your comments! >>>>>>>>>> >>>>>>>>>> -- Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Dima >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Bengt >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Nov 13 14:29:00 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 15:29:00 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B946.90806@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> Message-ID: > 13 nov 2014 kl. 14:59 skrev Bengt Rutisson : > > >> On 2014-11-13 13:56, Dmitry Fazunenko wrote: >> >>> On 13.11.2014 17:42, Bengt Rutisson wrote: >>> >>>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>> >>>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>> >>>>> Hi Evgeniya, >>>>> >>>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>> Hi Dmitry, >>>>>> >>>>>> You are right - I've forgotten about copyrights >>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>> >>>>> >>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be good to add -XX:+UseG1GC to the @run tags and then use @requires vm.gc=="G1" | vm.gc == null. >>>>> >>>>> >>>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated to the conflicting GC combinations. Should that really be part of this changeset? >>>>> >>>>> >>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we really need @requires for them? >>>> >>>> Yes, we do. >>>> These tests use TestShrinkAuxiliaryData class which implements its own mechanism to analyze VM options an skip if not applicable collector is given. @requires - allows to rely on jtreg. >>>> >>>> Driver mode is a kind of indicator, that the test will spawn its own java process. >>> >>> I thought the point of @driver was that no external vmoptions were passed to such a test. Is that not the case? >> >> In the driver mode VM is started without external VM flags. Those flags are passed to the tests via system property. >> The driver mode is a sort of shell to start something else. > > Right. So, why would you need @requires on the TestShrinkAuxiliaryDataXX tests because the utility TestShrinkAuxiliaryData picks up the vm flags through Utils.getTestJavaOpts(). What's the point in running this in a driver mode when they anyway pick up the vm flags? The first sentence in the above paragraph makes more sense if you remove the words "why would". Bengt > > I'm asking because adding @requires to these tests means that we will run them less often than we do now. So, I'm a bit worried that we reduce the amount of testing we do. > > Bengt > >> >> -- Dima >> >> >>> >>> Bengt >>> >>>> >>>> Thanks >>>> Dima >>>> >>>>> >>>>> >>>>> Otherwise it look ok to me. >>>>> >>>>> Bengt >>>>> >>>>> >>>>>> >>>>>> Thanks >>>>>> Evgeniya Stepanova >>>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> The fix looks good to me. >>>>>>> >>>>>>> I noticed the following minor things: >>>>>>> - copyrights need to include the year of last change >>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated files, but doesn't contain any changes >>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable 'prohibitedVmOptions' >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>> Hi everyone! >>>>>>>> >>>>>>>> Since the decision was made to change only tests that fail because of conflict for now (skip "selfish" tests), I post new webrev for hotspot part of the JDK-8019361: >>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>> Nice plan! Please feel free to send me any feedback/questions regarding @requires >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Dima, >>>>>>>>>> >>>>>>>>>> Thanks for the answers. I think the currently proposed patch is a good start. We will have to evolve the @requires tag in the future, but let's have that discussion separate from this review. And we can start that discussion later when we have more experience with the current version of @requires. >>>>>>>>>> >>>>>>>>>> Thanks for doing this! >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>> >>>>>>>>>>> The general comment: currently, jtreg doesn't support any sort of plugins, so you can't provide a VM specific handler of the @requires or another tag. This is very annoying limitation and we have to live with it. >>>>>>>>>>> >>>>>>>>>>> A few more comments inline. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Dima, >>>>>>>>>>>> >>>>>>>>>>>> Answers inline. >>>>>>>>>>>> >>>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>>>>>>>>>>> >>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>> >>>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of the JDK-8019361 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set another GC. >>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to skip test if there is a conflict >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for fixing this! It is really great that we finally start sorting this out. >>>>>>>>>>>>>> >>>>>>>>>>>>>> First a general comment. The @requires tag has been developed without much cooperation with the GC team. We did have a lot of feedback when it was first presented a year ago, but it does not seem like this feedback was incorporated into the @requires that was eventually built. >>>>>>>>>>>>> >>>>>>>>>>>>> We tried to implement as much developer's wishes as possible. But not everything is possible, sorry for that. >>>>>>>>>>>> >>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we have been requesting this feature for 3 years and I was expecting us to be able to influence the feature much more than was the case now. >>>>>>>>>>> >>>>>>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think this change that gets proposed now is a big step forward and I won't object to it. But I am pretty convinced that we will soon run in to the limitations of the current @requires implementation and we will have to redo this work. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Some of the points I don't really like about the @requires tag are: >>>>>>>>>>>>>> >>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It would have been better to just "require" any command line flag. >>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible to use: >>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>> >>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>> >>>>>>>>>>>> The problem with having this matching built in to JTreg is that it makes it very hard to change. When we discussed this a year ago I think we said that JTreg should only provide a means to test against the command line and a hook for running some java code in the @requires tag. That way we could put logic like this in a test library that is under our control. This would make it easy for us to change and also enables us to use different logic for different versions. >>>>>>>>>>> >>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we have to do what you did in this change and use vm.gc=null even when some tests could actually have been run when a GC was specified. >>>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, as well as test case support. >>>>>>>>>>>> >>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> Under test case support I mean ability to treat each @run as a separate test. Now >>>>>>>>>>> >>>>>>>>>>> @test >>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>> class MyTest { >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - there are many tests that require more than just a specific GC. Often there are other flags that can't be changed either for the test to work properly. >>>>>>>>>>>>> >>>>>>>>>>>>> yes. conflicting GC is just the most popular problem caused by conflicting options. >>>>>>>>>>>>> If we address this issue and we are satisfied with solution, we could move further. >>>>>>>>>>>> >>>>>>>>>>>> Yes, I agree that taking one step at the time is good. Personally I would have preferred that the first step was a "just run the command line as specified in the @run tag" step. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Maybe this is not the right place to discuss the current implementation of the @requires tag. I just want to say that I'm not too happy about how the @requires tag turned out. But assuming we have to use it the way it is now I guess the proposed changeset looks good. >>>>>>>>>>>>> >>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>> >>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not looked at all tests. But if they now pass with the combinations that we test with I guess they should be ok. >>>>>>>>>>> >>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and without any GC flag). Tests are being excluded as expected. No tests failed because of the conflict. >>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>> >>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>>>>> >>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think some of the test, like test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run with -XX:+UseParNewGC. Others, like test/gc/startup_warnings/TestParNewCMS.java, will fail if you run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>>>>>>>> >>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>> Add @requires here is not necessary, but it will allow not execute the tests when not needed. >>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these tests 4 times, 1 should be enough. >>>>>>>>>>>> >>>>>>>>>>>> Do we really want to use the @requires functionality for this purpose? It seems like a way of misusing @requires. If we just want the tests to be run once I think Leonid's approach with tests lists seems more suitable. >>>>>>>>>>> >>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> But are you sure that this is the reason for the @requires in this case? TestDefNewCMS does sound like a test that is DefNew specific. I don't see a reason to run it with ParNew. If it doesn't fail today it should probably be changed so that it does fail if it is run with the wrong GC. >>>>>>>>>>> >>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy way to solve a lot of issues. >>>>>>>>>>> >>>>>>>>>>> I hope, @requires will allow to reduce the number of "selfish" tests, which produce a new java process to ignore vm flags coming from outside. No @requires, no other mechanism could 100% protect a test from running with conflicting options, but this is not the goal. >>>>>>>>>>> >>>>>>>>>>> If one runs tests with an exotic option, like a new G2 collector, there shouldn't mass failures caused by options conflicts. But a few failures could be handled manually. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Similarly it looks to me like there are tests that will fail if you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm about to push a changeset that removes them: >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Is there some way of making sure that all tests are run at one time or another. With this change there is a risk that some tests are never run and always skipped. Will we somehow be tracking what gets skipped and make sure that all tests are at least run once with the correct GC so that it is not skipped all the time? >>>>>>>>>>>>> >>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do soon, after getting fix of: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>> >>>>>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>> >>>>>>>>>>>>> Other examples: >>>>>>>>>>>>> >>>>>>>>>>>>> /* >>>>>>>>>>>>> *@ignore >>>>>>>>>>>>> *@test >>>>>>>>>>>>> */ >>>>>>>>>>>>> ... >>>>>>>>>>>>> >>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>> *@test >>>>>>>>>>>>> */ >>>>>>>>>>>>> ... >>>>>>>>>>>>> Such tests will never be run, because jtreg treats as test only files with @test on the first place... >>>>>>>>>>>>> >>>>>>>>>>>>> So, making sure that tests do not disappear is important SQE task, we know about that, we're thinking on solution (may be very actively). But this subject for another discussion, not within RFR :) >>>>>>>>>>>> >>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>> >>>>>>>>>>> I was going to say "not very actively", but never mind, we know about this problem. With introducing @requires mechanism it will become more important! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks for your comments! >>>>>>>>>>> >>>>>>>>>>> -- Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Dima >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>> >>>>>>>> -- >>>>>>>> Evgeniya Stepanova >>>>>> >>>>>> -- >>>>>> Evgeniya Stepanova > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Thu Nov 13 14:33:10 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Thu, 13 Nov 2014 18:33:10 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B2E6.9070802@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> Message-ID: <5464C126.3020003@oracle.com> Hi Bengt, On 13.11.2014 17:32, Bengt Rutisson wrote: > > Hi Evgeniya, > > On 2014-11-12 17:28, Evgeniya Stepanova wrote: >> Hi Dmitry, >> >> You are right - I've forgotten about copyrights >> Copyrights and other issues you mentioned fixed. New webrev: >> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ > > > For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be > good to add -XX:+UseG1GC to the @run tags and then use @requires > vm.gc=="G1" | vm.gc == null. > It is a good idea, I'll change the test > The change to test/gc/defnew/HeapChangeLogging.java is unrelated to > the conflicting GC combinations. Should that really be part of this > changeset? > This change is from previous webrev. Test is "selfish" and I've removed @requires from it, but left comments change that Jon pointed to me. So this change is really not related to the @requires. I'll remove them if it is confuse you. > > The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we > really need @requires for them? > > > Otherwise it look ok to me. > > Bengt > > >> >> Thanks >> Evgeniya Stepanova >> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>> Hi Evgeniya, >>> >>> The fix looks good to me. >>> >>> I noticed the following minor things: >>> - copyrights need to include the year of last change >>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>> files, but doesn't contain any changes >>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >>> 'prohibitedVmOptions' >>> >>> Thanks, >>> Dima >>> >>> >>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>> Hi everyone! >>>> >>>> Since the decision was made to change only tests that fail because >>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>> hotspot part of the JDK-8019361 >>>> : >>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>> Nice plan! Please feel free to send me any feedback/questions >>>>> regarding @requires >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Dima, >>>>>> >>>>>> Thanks for the answers. I think the currently proposed patch is a >>>>>> good start. We will have to evolve the @requires tag in the >>>>>> future, but let's have that discussion separate from this review. >>>>>> And we can start that discussion later when we have more >>>>>> experience with the current version of @requires. >>>>>> >>>>>> Thanks for doing this! >>>>>> Bengt >>>>>> >>>>>> >>>>>> >>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> That's great that we have very closed visions! >>>>>>> >>>>>>> The general comment: currently, jtreg doesn't support any sort >>>>>>> of plugins, so you can't provide a VM specific handler of the >>>>>>> @requires or another tag. This is very annoying limitation and >>>>>>> we have to live with it. >>>>>>> >>>>>>> A few more comments inline. >>>>>>> >>>>>>> >>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi Dima, >>>>>>>> >>>>>>>> Answers inline. >>>>>>>> >>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>> Hi Bengt, >>>>>>>>> >>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it very >>>>>>>>> much! >>>>>>>>> >>>>>>>>> See comments inline. >>>>>>>>> >>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part >>>>>>>>>>> of the JDK-8019361 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>> set another GC. >>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>> skip test if there is a conflict >>>>>>>>>> >>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>> start sorting this out. >>>>>>>>>> >>>>>>>>>> First a general comment. The @requires tag has been developed >>>>>>>>>> without much cooperation with the GC team. We did have a lot >>>>>>>>>> of feedback when it was first presented a year ago, but it >>>>>>>>>> does not seem like this feedback was incorporated into the >>>>>>>>>> @requires that was eventually built. >>>>>>>>> >>>>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>>>> But not everything is possible, sorry for that. >>>>>>>> >>>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>>> been requesting this feature for 3 years and I was expecting us >>>>>>>> to be able to influence the feature much more than was the case >>>>>>>> now. >>>>>>> >>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>>> >>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>> tag are: >>>>>>>>>> >>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible >>>>>>>>> to use: >>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>> >>>>>>>>> The table with all vars available in jtreg: >>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>> >>>>>>>> The problem with having this matching built in to JTreg is that >>>>>>>> it makes it very hard to change. When we discussed this a year >>>>>>>> ago I think we said that JTreg should only provide a means to >>>>>>>> test against the command line and a hook for running some java >>>>>>>> code in the @requires tag. That way we could put logic like >>>>>>>> this in a test library that is under our control. This would >>>>>>>> make it easy for us to change and also enables us to use >>>>>>>> different logic for different versions. >>>>>>> >>>>>>> I would be glad to have own harness... >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> - the requirement should be per @run tag. Right now we have >>>>>>>>>> to do what you did in this change and use vm.gc=null even >>>>>>>>>> when some tests could actually have been run when a GC was >>>>>>>>>> specified. >>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>> well as test case support. >>>>>>>> >>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>> >>>>>>> Under test case support I mean ability to treat each @run as a >>>>>>> separate test. Now >>>>>>> >>>>>>> @test >>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>> class MyTest { >>>>>>> } >>>>>>> >>>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> - there are many tests that require more than just a specific >>>>>>>>>> GC. Often there are other flags that can't be changed either >>>>>>>>>> for the test to work properly. >>>>>>>>> >>>>>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>>>>> conflicting options. >>>>>>>>> If we address this issue and we are satisfied with solution, >>>>>>>>> we could move further. >>>>>>>> >>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>> Personally I would have preferred that the first step was a >>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>>>> I'm not too happy about how the @requires tag turned out. But >>>>>>>>>> assuming we have to use it the way it is now I guess the >>>>>>>>>> proposed changeset looks good. >>>>>>>>> >>>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>>> jtreg :) >>>>>>>>> And thanks for reviewing it! >>>>>>>> >>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>>> looked at all tests. But if they now pass with the combinations >>>>>>>> that we test with I guess they should be ok. >>>>>>> >>>>>>> Excellent! Thanks a lot! >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>> >>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>> >>>>>>>> Ok. Thanks. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think some of the test, like >>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>>>> >>>>>>>>> These two tests ignore vm flags. >>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>> execute the tests when not needed. >>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>>> tests 4 times, 1 should be enough. >>>>>>>> >>>>>>>> Do we really want to use the @requires functionality for this >>>>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>>>> want the tests to be run once I think Leonid's approach with >>>>>>>> tests lists seems more suitable. >>>>>>> >>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>> >>>>>>> >>>>>>>> But are you sure that this is the reason for the @requires in >>>>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>>>> doesn't fail today it should probably be changed so that it >>>>>>>> does fail if it is run with the wrong GC. >>>>>>> >>>>>>> @requires - is not the silver bullet, but it's quite easy way to >>>>>>> solve a lot of issues. >>>>>>> >>>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>>> tests, which produce a new java process to ignore vm flags >>>>>>> coming from outside. No @requires, no other mechanism could 100% >>>>>>> protect a test from running with conflicting options, but this >>>>>>> is not the goal. >>>>>>> >>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>> collector, there shouldn't mass failures caused by options >>>>>>> conflicts. But a few failures could be handled manually. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> Similarly it looks to me like there are tests that will fail >>>>>>>>>> if you run them with -XX:-UseParallelOldGC or >>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>> >>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>> >>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>> okay, thank for letting us know. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>> some tests are never run and always skipped. Will we somehow >>>>>>>>>> be tracking what gets skipped and make sure that all tests >>>>>>>>>> are at least run once with the correct GC so that it is not >>>>>>>>>> skipped all the time? >>>>>>>>> >>>>>>>>> This is a very good question! >>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>>> soon, after getting fix of: >>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>> >>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>> >>>>>>>>> Other examples: >>>>>>>>> >>>>>>>>> /* >>>>>>>>> *@ignore >>>>>>>>> *@test >>>>>>>>> */ >>>>>>>>> ... >>>>>>>>> >>>>>>>>> /*@bug 4445555 >>>>>>>>> *@test >>>>>>>>> */ >>>>>>>>> ... >>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>> only files with @test on the first place... >>>>>>>>> >>>>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>>>> very actively). But this subject for another discussion, not >>>>>>>>> within RFR :) >>>>>>>> >>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>> >>>>>>> I was going to say "not very actively", but never mind, we know >>>>>>> about this problem. With introducing @requires mechanism it will >>>>>>> become more important! >>>>>>> >>>>>>> >>>>>>> Thanks for your comments! >>>>>>> >>>>>>> -- Dima >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Bengt >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Thu Nov 13 13:57:20 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 13 Nov 2014 17:57:20 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B946.90806@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> Message-ID: <5464B8C0.6050300@oracle.com> On 13.11.2014 17:59, Bengt Rutisson wrote: > > On 2014-11-13 13:56, Dmitry Fazunenko wrote: >> >> On 13.11.2014 17:42, Bengt Rutisson wrote: >>> >>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>> >>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>> >>>>> Hi Evgeniya, >>>>> >>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>> Hi Dmitry, >>>>>> >>>>>> You are right - I've forgotten about copyrights >>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>> >>>>> >>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would >>>>> be good to add -XX:+UseG1GC to the @run tags and then use >>>>> @requires vm.gc=="G1" | vm.gc == null. >>>>> >>>>> >>>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated >>>>> to the conflicting GC combinations. Should that really be part of >>>>> this changeset? >>>>> >>>>> >>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >>>>> really need @requires for them? >>>> >>>> Yes, we do. >>>> These tests use TestShrinkAuxiliaryData class which implements its >>>> own mechanism to analyze VM options an skip if not applicable >>>> collector is given. @requires - allows to rely on jtreg. >>>> >>>> Driver mode is a kind of indicator, that the test will spawn its >>>> own java process. >>> >>> I thought the point of @driver was that no external vmoptions were >>> passed to such a test. Is that not the case? >> >> In the driver mode VM is started without external VM flags. Those >> flags are passed to the tests via system property. >> The driver mode is a sort of shell to start something else. > > Right. So, why would you need @requires on the > TestShrinkAuxiliaryDataXX tests because the utility > TestShrinkAuxiliaryData picks up the vm flags through > Utils.getTestJavaOpts(). What's the point in running this in a driver > mode when they anyway pick up the vm flags? TestShrinkAuxiliaryData implemented a workaround awaiting for @requires to appear in jtreg. Frankly speaking, the driver mode doesn't bring a lot of value, it's rather confusing and obligate developers to be more careful. If a class just spawns another java process with a real test, it's a big deal to run this class with or without external options. But there is no guarantee, that people will not start run real tests in driver mode... > > I'm asking because adding @requires to these tests means that we will > run them less often than we do now. So, I'm a bit worried that we > reduce the amount of testing we do. Don't worry about it. We want to run more tests, believe me. -- Dima > > Bengt > >> >> -- Dima >> >> >>> >>> Bengt >>> >>>> >>>> Thanks >>>> Dima >>>> >>>>> >>>>> >>>>> Otherwise it look ok to me. >>>>> >>>>> Bengt >>>>> >>>>> >>>>>> >>>>>> Thanks >>>>>> Evgeniya Stepanova >>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> The fix looks good to me. >>>>>>> >>>>>>> I noticed the following minor things: >>>>>>> - copyrights need to include the year of last change >>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among >>>>>>> updated files, but doesn't contain any changes >>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>>> variable 'prohibitedVmOptions' >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>> Hi everyone! >>>>>>>> >>>>>>>> Since the decision was made to change only tests that fail >>>>>>>> because of conflict for now (skip "selfish" tests), I post new >>>>>>>> webrev for hotspot part of the JDK-8019361 >>>>>>>> : >>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>>>>> regarding @requires >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Dima, >>>>>>>>>> >>>>>>>>>> Thanks for the answers. I think the currently proposed patch >>>>>>>>>> is a good start. We will have to evolve the @requires tag in >>>>>>>>>> the future, but let's have that discussion separate from this >>>>>>>>>> review. And we can start that discussion later when we have >>>>>>>>>> more experience with the current version of @requires. >>>>>>>>>> >>>>>>>>>> Thanks for doing this! >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>> >>>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>>> sort of plugins, so you can't provide a VM specific handler >>>>>>>>>>> of the @requires or another tag. This is very annoying >>>>>>>>>>> limitation and we have to live with it. >>>>>>>>>>> >>>>>>>>>>> A few more comments inline. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Dima, >>>>>>>>>>>> >>>>>>>>>>>> Answers inline. >>>>>>>>>>>> >>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>>>>> very much! >>>>>>>>>>>>> >>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>> >>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>> jtreg set another GC. >>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>> "requires" to skip test if there is a conflict >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for fixing this! It is really great that we >>>>>>>>>>>>>> finally start sorting this out. >>>>>>>>>>>>>> >>>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>>> developed without much cooperation with the GC team. We >>>>>>>>>>>>>> did have a lot of feedback when it was first presented a >>>>>>>>>>>>>> year ago, but it does not seem like this feedback was >>>>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>>>> >>>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>>> >>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>>> expecting us to be able to influence the feature much more >>>>>>>>>>>> than was the case now. >>>>>>>>>>> >>>>>>>>>>> My personal hope: @requires will address ~90% of existing >>>>>>>>>>> issues. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>>>>> forward and I won't object to it. But I am pretty >>>>>>>>>>>>>> convinced that we will soon run in to the limitations of >>>>>>>>>>>>>> the current @requires implementation and we will have to >>>>>>>>>>>>>> redo this work. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Some of the points I don't really like about the >>>>>>>>>>>>>> @requires tag are: >>>>>>>>>>>>>> >>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. >>>>>>>>>>>>>> It would have been better to just "require" any command >>>>>>>>>>>>>> line flag. >>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>>> possible to use: >>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>> >>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>> >>>>>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>>>>> that it makes it very hard to change. When we discussed >>>>>>>>>>>> this a year ago I think we said that JTreg should only >>>>>>>>>>>> provide a means to test against the command line and a hook >>>>>>>>>>>> for running some java code in the @requires tag. That way >>>>>>>>>>>> we could put logic like this in a test library that is >>>>>>>>>>>> under our control. This would make it easy for us to change >>>>>>>>>>>> and also enables us to use different logic for different >>>>>>>>>>>> versions. >>>>>>>>>>> >>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>>> have to do what you did in this change and use vm.gc=null >>>>>>>>>>>>>> even when some tests could actually have been run when a >>>>>>>>>>>>>> GC was specified. >>>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, >>>>>>>>>>>>> as well as test case support. >>>>>>>>>>>> >>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> Under test case support I mean ability to treat each @run as >>>>>>>>>>> a separate test. Now >>>>>>>>>>> >>>>>>>>>>> @test >>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>> class MyTest { >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> is always a single test. You can't exclude, or re-run a part >>>>>>>>>>> of it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>>> >>>>>>>>>>>>> yes. conflicting GC is just the most popular problem >>>>>>>>>>>>> caused by conflicting options. >>>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>>> solution, we could move further. >>>>>>>>>>>> >>>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>>> Personally I would have preferred that the first step was a >>>>>>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>>>>> implementation of the @requires tag. I just want to say >>>>>>>>>>>>>> that I'm not too happy about how the @requires tag turned >>>>>>>>>>>>>> out. But assuming we have to use it the way it is now I >>>>>>>>>>>>>> guess the proposed changeset looks good. >>>>>>>>>>>>> >>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>>> about jtreg :) >>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>> >>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have >>>>>>>>>>>> not looked at all tests. But if they now pass with the >>>>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>>>> >>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests >>>>>>>>>>>>>>> are being excluded as expected. No tests failed because >>>>>>>>>>>>>>> of the conflict. >>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>> >>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>>>>> >>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if >>>>>>>>>>>>>> you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if >>>>>>>>>>>>>> you run with -XX:-UseParNewGC. Could you test these two >>>>>>>>>>>>>> cases too? >>>>>>>>>>>>> >>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>>>>>> execute the tests when not needed. >>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>>> >>>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>>> this purpose? It seems like a way of misusing @requires. If >>>>>>>>>>>> we just want the tests to be run once I think Leonid's >>>>>>>>>>>> approach with tests lists seems more suitable. >>>>>>>>>>> >>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> But are you sure that this is the reason for the @requires >>>>>>>>>>>> in this case? TestDefNewCMS does sound like a test that is >>>>>>>>>>>> DefNew specific. I don't see a reason to run it with >>>>>>>>>>>> ParNew. If it doesn't fail today it should probably be >>>>>>>>>>>> changed so that it does fail if it is run with the wrong GC. >>>>>>>>>>> >>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy >>>>>>>>>>> way to solve a lot of issues. >>>>>>>>>>> >>>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>>> "selfish" tests, which produce a new java process to ignore >>>>>>>>>>> vm flags coming from outside. No @requires, no other >>>>>>>>>>> mechanism could 100% protect a test from running with >>>>>>>>>>> conflicting options, but this is not the goal. >>>>>>>>>>> >>>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. >>>>>>>>>>>>>> I'm about to push a changeset that removes them: >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Is there some way of making sure that all tests are run >>>>>>>>>>>>>> at one time or another. With this change there is a risk >>>>>>>>>>>>>> that some tests are never run and always skipped. Will we >>>>>>>>>>>>>> somehow be tracking what gets skipped and make sure that >>>>>>>>>>>>>> all tests are at least run once with the correct GC so >>>>>>>>>>>>>> that it is not skipped all the time? >>>>>>>>>>>>> >>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will >>>>>>>>>>>>> do soon, after getting fix of: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>> >>>>>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>> >>>>>>>>>>>>> Other examples: >>>>>>>>>>>>> >>>>>>>>>>>>> /* >>>>>>>>>>>>> *@ignore >>>>>>>>>>>>> *@test >>>>>>>>>>>>> */ >>>>>>>>>>>>> ... >>>>>>>>>>>>> >>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>> *@test >>>>>>>>>>>>> */ >>>>>>>>>>>>> ... >>>>>>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>>>>>> only files with @test on the first place... >>>>>>>>>>>>> >>>>>>>>>>>>> So, making sure that tests do not disappear is important >>>>>>>>>>>>> SQE task, we know about that, we're thinking on solution >>>>>>>>>>>>> (may be very actively). But this subject for another >>>>>>>>>>>>> discussion, not within RFR :) >>>>>>>>>>>> >>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>> >>>>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>>>> know about this problem. With introducing @requires >>>>>>>>>>> mechanism it will become more important! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks for your comments! >>>>>>>>>>> >>>>>>>>>>> -- Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Dima >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.helin at oracle.com Thu Nov 13 16:01:33 2014 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 13 Nov 2014 17:01:33 +0100 Subject: RFR: 8064721: The card tables only ever need two covering regions In-Reply-To: <46B47E05-306E-4380-AC2F-A586AEE9BBC4@oracle.com> References: <5463ADA1.9070804@oracle.com> <46B47E05-306E-4380-AC2F-A586AEE9BBC4@oracle.com> Message-ID: <5464D5DD.7010300@oracle.com> On 2014-11-12 22:30, Kim Barrett wrote: > On Nov 12, 2014, at 1:57 PM, Erik Helin wrote: >> >> this patch removes the max_covering_regions argument from the BarrierSet constructor, since all the code using a BarrierSet only needs two covering regions. Having the maximum number of covering regions as a constant is desirable because it makes it much easier to reason about the code in all the various card tables (CardTableRS, CardTableModRefBS, G1SATBCardTableModRefBS, CardTableExtension). >> >> [?] > >> Webrev: >> http://cr.openjdk.java.net/~ehelin/8064721/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8064721 > > Thanks for the thorough explanation in the covering email for this > RFR. Thanks for taking your time to review! > ------------------------------------------------------------------------------ > > src/share/vm/memory/barrierSet.hpp > 52 // Some barrier sets create tables whose elements correspond to parts of > 53 // the heap; the CardTableModRefBS is an example. Such barrier sets will > 54 // normally reserve space for such tables, and commit parts of the table > 55 // "covering" parts of the heap that are committed. At most one covered > 56 // region per generation is needed. > 57 static const int _max_covered_regions = 2; > > Is this (now) constant slated for later elimination? Sorry, I don't really follow, why would we remove this constant? Do you mean we should "hard code" 2 at all places where _max_covered_regions is used in CardTableModRefBS? Or that we should just move the constant down the inheritance chain to CardTableModRefBS where it really belongs? > ------------------------------------------------------------------------------ > > src/share/vm/memory/cardTableRS.hpp > 86 // Number of generations (including permgen). > 87 static const int _regions_to_iterate = 3; > > And from the covering email: > The other variable that was made static const is > CardTableRS::_regions_to_iterate. This was previously set to > max_covered_regions - 1 by the framework collectors, so 4 - 1 = 3. I > also added a comment explaining why it needs to be 3 and not 2. > > Sorry, but I'm not finding that comment on line 86 very informative, > especially in light of past removal of permgen. > > I don't yet understand why this isn't 1 if it used to be > max_covered_regions - 1 and max_covered_regions is now a constant 2. > But I suspect that value is wrong, based on its usage. So, CardTableRS is used by the framework collectors (and G1) and is set up via GenCollectedHeap::initialize which calls collector_policy()->create_rem_set(...) which creates a new CardTableRS. The framework collectors (CMS, ParNew, DefNew, Serial) passed 4 as max_covered_regions, G1 passes 2. This is why _regions_to_iterate must be max(4-1, 2-1) = 3. > Maybe it should instead be 2 because the now removed increment of > max_covered_regions by 2 was really a workaround for the calculation > here, and needed this to include permgen (e.g. be 3). (This actually > seems kind of plausible, since the offending increment had a comment > indicating it was a workaround for a needed cardtable fix.) Yes, 2 should be enough, but as you correctly suspect, CardTableRS was not updated when PermGen was removed. The comment above the declaration of _last_cur_val_in_gen reads: // An array that contains, for each generation, the card table value last // used as the current value for a younger_refs_do iteration of that // portion of the table. (The perm gen is index 0; other gens are at // their level plus 1. They youngest gen is in the table, but will // always have the value "clean_card".) jbyte* _last_cur_val_in_gen; So CMS (which will get index 2) needs to access index 2 in this array, therefore _regions_to_iterate must be 3 (permgen + young gen + old gen). If we remove all traces of PermGen in CardTableRS, we can probably use 2, but thas is another change. This is why I tried to add the comment, to help someone understand why we use the number 3 for _regions_to_iterate. // Number of generations (including permgen). static const int _regions_to_iterate = 3; Do you think I should rephrase this comment? > _regions_to_iterate is used as a bound on the iteration through the > _last_cur_val_in_gen table, whose size is > GenCollectedHeap::max_gens+1. I'm not sure what the +1 is for; I > wonder if this might be another permgen leftover. The +1 here is because PermGen gets index 0, so every generation in GenCollectedHeap will get their generation index + 1. I don't think GenCollectedHeap::max_gens included perm gen in the count, PermGen was never part of GenCollectedHeap::_gens (I believe it was created separately). > In any case, it's not apparent to me why 3 is the right value now. I > think 3 will work, but I think there may be further followup work to > be done in this vicinity. I would be fine with having that be spawned > off as a separate task, rather than being rolled into here; the > analysis of this change set is already complicated enough. Since I didn't trust my reasoning that I outlined in the review request email, I run a lot of tests :) The JVM crashes for values 1 and 2, so we better use 3. As I mentioned above, if someone gets rid of all the PermGen legacy in CardTableRS, we can probably use 2 as the value. > But perhaps there is there a plan to change the value later, or > eliminate this constant altogether? If so, it would be helpful to > mention that. > > ------------------------------------------------------------------------------ > > src/share/vm/memory/modRefBarrierSet.hpp > 40 ModRefBarrierSet() : BarrierSet() { _kind = BarrierSet::ModRef; } > > Not sure why this change was made; the default construction of > BarrierSet was already implicit. Not that I object to the change, > just surprised by it. I didn't completely remember when in C++ a parent constructor was automatically called, so I decided to add a call to BarrierSet for the default constructor. If it is implicit then I can revert this part of the patch if you like (although I prefer the the explicit variant). > If any change is going to be made here, _kind should be initialized > via a member initializer rather than assignment in the constructor > body. Yes, if you agree that we should keep the explicit constructor call to BarrierSet then I will definitely update the patch to use _kind in the initialization list. > ------------------------------------------------------------------------------ > > src/share/vm/runtime/vmStructs.cpp > removal of > 476 nonstatic_field(BarrierSet, _max_covered_regions, int) \ > > This is removing an entry that was being made available to Java code. > Are we sure there aren't any users of it? [I did a grep over a full > JDK tree and found no occurrences of "max_covered_regions" outside > hotspot C++ code, so there don't appear to be any Java uses in the > JDK. Could there be uses outside that tree?] It might be safer to > change it to a static_field, rather than removing it entirely. Hm, > and does removal (or even change?) of this require CCC approval? The entries in vmStructs are not supported, the only supported user of contents in vmStrcuts is the SA agent (hotspot/agent). We do not need a CCC request for changing vmStructs, as long the SA agent works, any change is fine. I also grepped for any usage of "max_covered_regions" in the SA code and didn't find any references (I also run the SA tests and they all pass), so the "max_covered_regions" field can safely be removed. Thanks, Erik From kim.barrett at oracle.com Thu Nov 13 16:11:35 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Nov 2014 11:11:35 -0500 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <54645C75.2030707@oracle.com> References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> <54638F87.30809@oracle.com> <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> <54645C75.2030707@oracle.com> Message-ID: <9F2FBF39-EDEF-4872-9665-8B107E2C4DA7@oracle.com> On Nov 13, 2014, at 2:23 AM, Bengt Rutisson wrote: > >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp >> >> 1248 // For debugging purposes, change the type of collection. >> 1249 // If the rotation is not on the concurrent collection >> 1250 // type, don't start a concurrent collection. >> 1251 NOT_PRODUCT( >> 1252 if (RotateCMSCollectionTypes) { >> 1253 return _cmsGen->debug_concurrent_cycle(); >> 1254 } >> 1255 ) >> >> Code was changed, but the comment looks like it needs to be updated >> too, since there is no longer a "concurrent collection type" in the >> previous sense. > > Good catch. I updated the comment to: > > // For debugging purposes, change the type of collection. > // Rotate between concurrent and stop-the-world full GCs. Looks good. >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp >> >> 1254 if (RotateCMSCollectionTypes && >> 1255 (_cmsGen->debug_collection_type() != >> 1256 ConcurrentMarkSweepGeneration::Concurrent_collection_type)) { >> 1257 assert(_cmsGen->debug_collection_type() != >> 1258 ConcurrentMarkSweepGeneration::Unknown_collection_type, >> 1259 "Bad cms collection type"); >> 1260 return false; >> 1261 } >> >> => >> >> 1252 if (RotateCMSCollectionTypes) { >> 1253 return _cmsGen->debug_concurrent_cycle(); >> 1254 } >> >> I spent a large amount of time studying this change, because there are >> behavioral changes here that looked suspicious. I think I've decided >> it's ok, but RotateCMSCollectionTypes is now badly named, and was >> never well described and had (and still has) unclear semantics. I >> think I've puzzled out more or less what it means, but that required >> reading a lot of code, and I'm still not confident that I know what >> would happen if I used it, or why I might want to. > > Right. I agree that it is unclear if this code is needed. I have never been running with RotateCMSCollectionTypes but I did not want to remove it as part of this change. > > As you probably noticed the change I did was to go from three states (concurrent background, stw foreground and full serial GC) to two states (concurrent background and full serial GC). Since two states can be encoded with a bool I decided to simplify a bit a get rid of the ColletionType enum. It was easier for me to verify that I didn't break this if I could do that simplification. > > I don't think this functionality is used much and it is only available in debug builds. So, I would suggest that we just remove it. But I prefer not to remove it as part of this change. If you want to I can send out a small pre-patch to remove the RotateCMSCollectionTypes feature. However, I think it might be a better workflow for me to keep the changes as they are in the patch and then follow up with removing RotateCMSCollectionTypes. Let me know what you prefer and I'll file a JBS bug to track it if we decide to remove it. > > I do agree that the naming is bad. But if we decide to remove this functionality there is not much need to spend time on the naming discussion. My inclination would be to remove it. I think we have way too many options whose purpose is unclear and which are never being tested, so who knows whether they actually do what they were intended for (whatever that was), or do anything useful at all. (Witness the option I recently removed that has not worked during it?s entire lifetime in the mercurial repository.) > Updated webrev: > http://cr.openjdk.java.net/~brutisso/8064702/webrev.02/ > > Diff compared to last webrev: > http://cr.openjdk.java.net/~brutisso/8064702/webrev.01-02.diff/ New webrev looks good. From andrey.x.zakharov at oracle.com Thu Nov 13 17:05:49 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Thu, 13 Nov 2014 21:05:49 +0400 Subject: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent Message-ID: <5464E4ED.4040003@oracle.com> Hi, team. Excuse me for disturbing you, here is simple fix for subj. I've added explicit -XX:-ExplicitGCInvokesConcurrent to invocations Its fixes at least twu test bugs: https://bugs.openjdk.java.net/browse/JDK-8062957 https://bugs.openjdk.java.net/browse/JDK-8064716 webrev: http://cr.openjdk.java.net/~azakharov/8064716/webrev/ Thanks. From jesper.wilhelmsson at oracle.com Thu Nov 13 18:09:57 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 13 Nov 2014 19:09:57 +0100 Subject: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent In-Reply-To: <5464E4ED.4040003@oracle.com> References: <5464E4ED.4040003@oracle.com> Message-ID: <5464F3F5.1020203@oracle.com> Hi Andrey, Is this the preferred way over using @requires? /Jesper Andrey Zakharov skrev 13/11/14 18:05: > Hi, team. Excuse me for disturbing you, here is simple fix for subj. > I've added explicit -XX:-ExplicitGCInvokesConcurrent to invocations > > Its fixes at least twu test bugs: > https://bugs.openjdk.java.net/browse/JDK-8062957 > https://bugs.openjdk.java.net/browse/JDK-8064716 > > webrev: > http://cr.openjdk.java.net/~azakharov/8064716/webrev/ > > > Thanks. > From andrey.x.zakharov at oracle.com Thu Nov 13 17:11:38 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Thu, 13 Nov 2014 21:11:38 +0400 Subject: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent In-Reply-To: <5464F3F5.1020203@oracle.com> References: <5464E4ED.4040003@oracle.com> <5464F3F5.1020203@oracle.com> Message-ID: <5464E64A.4040307@oracle.com> Simplest way is the preferred way to solve issues, right? On 13.11.2014 22:09, Jesper Wilhelmsson wrote: > Hi Andrey, > > Is this the preferred way over using @requires? > /Jesper > > Andrey Zakharov skrev 13/11/14 18:05: >> Hi, team. Excuse me for disturbing you, here is simple fix for subj. >> I've added explicit -XX:-ExplicitGCInvokesConcurrent to invocations >> >> Its fixes at least twu test bugs: >> https://bugs.openjdk.java.net/browse/JDK-8062957 >> https://bugs.openjdk.java.net/browse/JDK-8064716 >> >> webrev: >> http://cr.openjdk.java.net/~azakharov/8064716/webrev/ >> >> >> Thanks. >> > From jesper.wilhelmsson at oracle.com Thu Nov 13 18:17:56 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 13 Nov 2014 19:17:56 +0100 Subject: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent In-Reply-To: <5464E64A.4040307@oracle.com> References: <5464E4ED.4040003@oracle.com> <5464F3F5.1020203@oracle.com> <5464E64A.4040307@oracle.com> Message-ID: <5464F5D4.9090600@oracle.com> That depends.. Will this work for all cases where -XX:+ExplicitGCInvokesConcurrent is added to the command line from different frameworks etc? /Jesper Andrey Zakharov skrev 13/11/14 18:11: > Simplest way is the preferred way to solve issues, right? > > On 13.11.2014 22:09, Jesper Wilhelmsson wrote: >> Hi Andrey, >> >> Is this the preferred way over using @requires? >> /Jesper >> >> Andrey Zakharov skrev 13/11/14 18:05: >>> Hi, team. Excuse me for disturbing you, here is simple fix for subj. >>> I've added explicit -XX:-ExplicitGCInvokesConcurrent to invocations >>> >>> Its fixes at least twu test bugs: >>> https://bugs.openjdk.java.net/browse/JDK-8062957 >>> https://bugs.openjdk.java.net/browse/JDK-8064716 >>> >>> webrev: >>> http://cr.openjdk.java.net/~azakharov/8064716/webrev/ >>> >>> >>> Thanks. >>> >> > From kim.barrett at oracle.com Thu Nov 13 19:33:41 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Nov 2014 14:33:41 -0500 Subject: RFR: 8064721: The card tables only ever need two covering regions In-Reply-To: <5464D5DD.7010300@oracle.com> References: <5463ADA1.9070804@oracle.com> <46B47E05-306E-4380-AC2F-A586AEE9BBC4@oracle.com> <5464D5DD.7010300@oracle.com> Message-ID: <51F872EF-EFDD-494F-8412-6BABA60CAAC5@oracle.com> On Nov 13, 2014, at 11:01 AM, Erik Helin wrote: > > On 2014-11-12 22:30, Kim Barrett wrote: >> src/share/vm/memory/barrierSet.hpp >> 52 // Some barrier sets create tables whose elements correspond to parts of >> 53 // the heap; the CardTableModRefBS is an example. Such barrier sets will >> 54 // normally reserve space for such tables, and commit parts of the table >> 55 // "covering" parts of the heap that are committed. At most one covered >> 56 // region per generation is needed. >> 57 static const int _max_covered_regions = 2; >> >> Is this (now) constant slated for later elimination? >> > Sorry, I don't really follow, why would we remove this constant? Do > you mean we should "hard code" 2 at all places where > _max_covered_regions is used in CardTableModRefBS? Or that we should > just move the constant down the inheritance chain to CardTableModRefBS > where it really belongs? There's the location question, of course. But there is also the question of whether this is any different from the maximum number of generations, e.g. use that constant rather than a separate one, or derived from that constant. (Can't depend on maximum number of generations right now, of course, since the cleanup and simplification of number of generations is still in progress.) >> src/share/vm/memory/cardTableRS.hpp >> 86 // Number of generations (including permgen). >> 87 static const int _regions_to_iterate = 3; >> [...] >> > // The perm gen is index 0; other gens are at > // their level plus 1. > [...] > Do you think I should rephrase this comment? > [...] > As I mentioned above, if someone gets rid of all the PermGen legacy in > CardTableRS, we can probably use 2 as the value. I'd missed that perm gen was index 0 for this. So 3 it is, at least for now. And yes, I think the comment needs more explanation; something along the lines of Number of generations, plus one for lingering PermGen issues in CardTableRS. Otherwise someone is likely to come along later, see "permgen", think they can subtract one and update the comment, and forget to do adequate testing... And perhaps someday this too would be replaced by direct reference to a maximum number of generations constant. And there needs to be a new bug report to clean up the lingering permgen stuff in CardTableRS. >> src/share/vm/memory/modRefBarrierSet.hpp >> 40 ModRefBarrierSet() : BarrierSet() { _kind = BarrierSet::ModRef; } >> >> Not sure why this change was made; the default construction of >> BarrierSet was already implicit. Not that I object to the change, >> just surprised by it. > I didn't completely remember when in C++ a parent constructor was > automatically called, so I decided to add a call to BarrierSet for the > default constructor. If it is implicit then I can revert this part of > the patch if you like (although I prefer the the explicit variant). If there's no explicit constructor call for a base class, a call to its default constructor gets implicitly inserted at the appropriate point in the member initializer list. I don't have a strong preference either way for explicit vs implicit calls to base class default constructors. In this case I probably would have left it implicit, but have no objection to explicit. But if there needed to be explicit calls to other base class constructors (due to multiple inheritance) then I'd probably make them all explicit. But the initialization of _kind really should be in the member initialization list, and since you are touching the code anyway? >> src/share/vm/runtime/vmStructs.cpp >> removal of >> 476 nonstatic_field(BarrierSet, _max_covered_regions, int) \ >> >> [...] >> > The entries in vmStructs are not supported ? OK, so no issue here. From kim.barrett at oracle.com Thu Nov 13 19:37:24 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Nov 2014 14:37:24 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <545CADC9.1010208@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <545CADC9.1010208@oracle.com> Message-ID: On Nov 7, 2014, at 6:32 AM, Bengt Rutisson wrote: > I don't think a thread can ever see a stale true value. We only reset > after a GC and there is a lot of things happening, including several > locks being taken and released, before we start a new GC. So, as far > as I can tell this does not require any more barriers. I may be > missing something though. If I am, can you give a more specific > example, Kim? > I can't point to a specific problem path yet. However, I'm rather paranoid around use of atomics instead of locks, and even more so with intentional races. And since I can't easily show that all paths are indeed safe... However, I think we should defer this part of the discussion until the final locations of the flag references have been ironed out; see other emails. From kim.barrett at oracle.com Thu Nov 13 21:28:50 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Nov 2014 16:28:50 -0500 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <9F2FBF39-EDEF-4872-9665-8B107E2C4DA7@oracle.com> References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> <54638F87.30809@oracle.com> <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> <54645C75.2030707@oracle.com> <9F2FBF39-EDEF-4872-9665-8B107E2C4DA7@oracle.com> Message-ID: <80D56232-2E5B-40EB-95F6-D79CBCF3D9AF@oracle.com> On Nov 13, 2014, at 11:11 AM, Kim Barrett wrote: > > On Nov 13, 2014, at 2:23 AM, Bengt Rutisson wrote: >>> ------------------------------------------------------------------------------ >>> >>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp >>> >>> 1254 if (RotateCMSCollectionTypes && >>> 1255 (_cmsGen->debug_collection_type() != >>> 1256 ConcurrentMarkSweepGeneration::Concurrent_collection_type)) { >>> 1257 assert(_cmsGen->debug_collection_type() != >>> 1258 ConcurrentMarkSweepGeneration::Unknown_collection_type, >>> 1259 "Bad cms collection type"); >>> 1260 return false; >>> 1261 } >>> >>> => >>> >>> 1252 if (RotateCMSCollectionTypes) { >>> 1253 return _cmsGen->debug_concurrent_cycle(); >>> 1254 } >>> >>> I spent a large amount of time studying this change, because there are >>> behavioral changes here that looked suspicious. I think I've decided >>> it's ok, but RotateCMSCollectionTypes is now badly named, and was >>> never well described and had (and still has) unclear semantics. I >>> think I've puzzled out more or less what it means, but that required >>> reading a lot of code, and I'm still not confident that I know what >>> would happen if I used it, or why I might want to. >> >> Right. I agree that it is unclear if this code is needed. I have never been running with RotateCMSCollectionTypes but I did not want to remove it as part of this change. >> >> [?] >> >> I do agree that the naming is bad. But if we decide to remove this functionality there is not much need to spend time on the naming discussion. > > My inclination would be to remove it. [?] Just to be clear, I?m ok with deferring removal to a later task. Please file a bug report? From kim.barrett at oracle.com Thu Nov 13 22:13:15 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Nov 2014 17:13:15 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <54642CFC.7020609@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> Message-ID: <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> On Nov 12, 2014, at 11:01 PM, Jon Masamitsu wrote: > > When any promotion failure occurs the concurrent CMS collections is > abandoned and a full collection is done instead. There is some > cleanup that is done to get the heap back into a state where the > full collection can be used. When any promotion failure occurs the concurrent CMS collections is abandoned and a full collection is done instead. There is some cleanup that is done to get the heap back into a state where the full collection can be used. That's what I suspected. Is the same true for other collectors that use par_promote()? (I think that's just ParallelOld?) I'm going to guess so for now. In that case I wonder what is really going on and whether the proposed change is really the best solution. I've not run the reproducer program from the bug report and examined its behavior, so my current understanding of the problem is based on the proposed change and the discussion surrounding it. At a high level, when promotion fails for some object it may take a long time for the survivor processing to come to a halt and allow follow-on work to occur. Based on the proposed change, part of the delay is that there continue to be promotion attempts, and those attempts are serialized while each, in turn, re-attempts work that will (probably?) fail. This is addressed through the introduction of the new flag, setting it at promotion failure, and checking it on entry to the lock context to fail fast and avoid the re-attempted work. However, during discussion of the proposed change it was stated that, while checking the new flag on lock context entry is an improvement, there is still a significant delay with just that change, and that this is due to contention for the lock. This is the rationale for introducing flag checks outside the lock context. This indicates that even after promotion fails the threads involved continue to make *many* promotion attempts. That leads me to wonder why we continue slamming par_promote() when we're just going to later abandon the collection in progress and start over with a full GC. Why don't we just stop calling par_promote() at all once we've had a promotion failure? The obvious first place to look is the caller(s) of par_promote(), e.g. ParNewGeneration::copy_to_survivor_space[_XXX_undo](). Promotion failure is handled there by setting an already existing flag and doing some additional work. It seems like it might work to just change the calls to par_promote() there to be conditional on that existing flag. This would make the checks of the proposed new flag outside the lock context redundant. If it is acceptable to have a few lingering wasted calls occur (which seems likely to me) then this conditionalization of calls to par_promote() would entirely eliminate the need for the proposed new flag. It seems like an even better approach would be to arrange to terminate the iteration over surviving objects early once promotion failure has occurred, rather than continuing to perform work that will ultimately be discarded. That looks to me like a much harder change to make though. (For one thing, I'm not sure all the work being done would actually be discarded. And there is cleanup to be done to restore the heap to a state ready for the full GC, and that might be complicated by having the earlier iteration terminate early.) Summing up, I currently think conditionalizing the calls to par_promote() on the existing _promotion_failed flag is a simpler and better solution than the proposed introduction of a new flag. (There may need to be some barriers added to support this; if so, similar barriers would be needed for the proposed new flag if it is to be accessed outside the lock context.) From bengt.rutisson at oracle.com Fri Nov 14 07:05:27 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 08:05:27 +0100 Subject: RFR (XS): -8064865: Remove the debug funciontality RotateCMSCollectionTypes for CMS Message-ID: <5465A9B7.2060104@oracle.com> Hi all, Can I have a couple of reviews for this small change to remove a debug flag from CMS? http://cr.openjdk.java.net/~brutisso/8064865/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8064865 Background: During the review of "8064702: Remove the CMS foreground collector" it was discussed how to handle the RotateCMSCollectionTypes feature. It was suggested to remove this feature in a separate change. This is the change to remove that feature. From the bug description: The RotateCMSCollectionTypes flag is used in CMS to rotate between the three different kinds of collections of the old generation that CMS has; background collection, foreground collection and full compacting serial GC. The functionality is only available in non-product builds. With JDK-8064702 the foreground collector is being removed. This means that there are only two types left. The RotateCMSCollectionTypes flag is not being automatically tested and it is very rarely used. Rather than maintaining it we should remove this functionality. If we need this for debugging in the future it is a pretty simple patch to add back. But for the most part it should be possible to debug in other ways, by for example using ExplicitGCInvokesConcurrent flag and similar. Thanks, Bengt From bengt.rutisson at oracle.com Fri Nov 14 07:08:17 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 08:08:17 +0100 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <80D56232-2E5B-40EB-95F6-D79CBCF3D9AF@oracle.com> References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> <54638F87.30809@oracle.com> <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> <54645C75.2030707@oracle.com> <9F2FBF39-EDEF-4872-9665-8B107E2C4DA7@oracle.com> <80D56232-2E5B-40EB-95F6-D79CBCF3D9AF@oracle.com> Message-ID: <5465AA61.5040409@oracle.com> Hi Kim, On 2014-11-13 22:28, Kim Barrett wrote: > On Nov 13, 2014, at 11:11 AM, Kim Barrett wrote: >> On Nov 13, 2014, at 2:23 AM, Bengt Rutisson wrote: >>>> ------------------------------------------------------------------------------ >>>> >>>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp >>>> >>>> 1254 if (RotateCMSCollectionTypes && >>>> 1255 (_cmsGen->debug_collection_type() != >>>> 1256 ConcurrentMarkSweepGeneration::Concurrent_collection_type)) { >>>> 1257 assert(_cmsGen->debug_collection_type() != >>>> 1258 ConcurrentMarkSweepGeneration::Unknown_collection_type, >>>> 1259 "Bad cms collection type"); >>>> 1260 return false; >>>> 1261 } >>>> >>>> => >>>> >>>> 1252 if (RotateCMSCollectionTypes) { >>>> 1253 return _cmsGen->debug_concurrent_cycle(); >>>> 1254 } >>>> >>>> I spent a large amount of time studying this change, because there are >>>> behavioral changes here that looked suspicious. I think I've decided >>>> it's ok, but RotateCMSCollectionTypes is now badly named, and was >>>> never well described and had (and still has) unclear semantics. I >>>> think I've puzzled out more or less what it means, but that required >>>> reading a lot of code, and I'm still not confident that I know what >>>> would happen if I used it, or why I might want to. >>> Right. I agree that it is unclear if this code is needed. I have never been running with RotateCMSCollectionTypes but I did not want to remove it as part of this change. >>> >>> [?] >>> >>> I do agree that the naming is bad. But if we decide to remove this functionality there is not much need to spend time on the naming discussion. >> My inclination would be to remove it. [?] > Just to be clear, I?m ok with deferring removal to a later task. Please file a bug report? Great, thanks! I filed a bug (JDK-8064865) and sent out a review request. Thanks again for looking at this and thanks for the review! Bengt > From bengt.rutisson at oracle.com Fri Nov 14 07:09:24 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 08:09:24 +0100 Subject: RFR (XS): -8064865: Remove the debug funciontality RotateCMSCollectionTypes for CMS In-Reply-To: <5465A9B7.2060104@oracle.com> References: <5465A9B7.2060104@oracle.com> Message-ID: <5465AAA4.9060702@oracle.com> On 2014-11-14 08:05, Bengt Rutisson wrote: > > Hi all, > > Can I have a couple of reviews for this small change to remove a debug > flag from CMS? > > http://cr.openjdk.java.net/~brutisso/8064865/webrev.00/ I should have pointed out that this patch is based on top of the patch for JDK-8064702 that is currently out for review. Bengt > > https://bugs.openjdk.java.net/browse/JDK-8064865 > > Background: > > During the review of "8064702: Remove the CMS foreground collector" it > was discussed how to handle the RotateCMSCollectionTypes feature. It > was suggested to remove this feature in a separate change. This is the > change to remove that feature. > > From the bug description: > > The RotateCMSCollectionTypes flag is used in CMS to rotate between the > three different kinds of collections of the old generation that CMS > has; background collection, foreground collection and full compacting > serial GC. > > The functionality is only available in non-product builds. > > With JDK-8064702 the foreground collector is being removed. This means > that there are only two types left. The RotateCMSCollectionTypes flag > is not being automatically tested and it is very rarely used. Rather > than maintaining it we should remove this functionality. If we need > this for debugging in the future it is a pretty simple patch to add > back. But for the most part it should be possible to debug in other > ways, by for example using ExplicitGCInvokesConcurrent flag and similar. > > Thanks, > Bengt From bengt.rutisson at oracle.com Fri Nov 14 08:46:59 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 09:46:59 +0100 Subject: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent In-Reply-To: <5464F5D4.9090600@oracle.com> References: <5464E4ED.4040003@oracle.com> <5464F3F5.1020203@oracle.com> <5464E64A.4040307@oracle.com> <5464F5D4.9090600@oracle.com> Message-ID: <5465C183.5020107@oracle.com> On 2014-11-13 19:17, Jesper Wilhelmsson wrote: > That depends.. Will this work for all cases where > -XX:+ExplicitGCInvokesConcurrent is added to the command line from > different frameworks etc? Jesper, the command line flags specified in the @run tag are *appened* to whatever the framework passes in. So, with Andrey's change the -XX:-ExplicitGCInvokesConcurrent in the @run tag will always override any -XX:+ExplicitGCInvokesConcurrent passed in from the framework. Note that this is a "normal" hotspot flag (not specially treated like the GC selection flags) so there is not conflict in specifying the same flag multiple times on the command line. The rule "last one wins" applies. Andrey, the changes look good. One minor thing is that I would prefer the @run tag in TestHumongousShrinkHeap.java to be a single line. I think it is a bit hard to read when the @run tag gets split up over multiple lines: 30 * @run main/othervm -XX:-ExplicitGCInvokesConcurrent -XX:MinHeapFreeRatio=10 31 * -XX:MaxHeapFreeRatio=12 -XX:+UseG1GC -XX:G1HeapRegionSize=1M -verbose:gc 32 * TestHumongousShrinkHeap Then a general question. We need this change because we some days rotate in -XX:+ExplicitGCInvokesConcurrent in our nightly testing. Is that really a good idea? (I'm sure it was requested from the GC team at some point, but maybe it is time to re-evaluate that decision.) A different way of handling these failures would be to stop rotating in that flag in our testing. I am not sure it really improves our quality and apparently it causes issues. Maybe we should just add a few tests that explicitly test with -XX:+ExplicitGCInvokesConcurrent? Thanks, Bengt > > /Jesper > > Andrey Zakharov skrev 13/11/14 18:11: >> Simplest way is the preferred way to solve issues, right? >> >> On 13.11.2014 22:09, Jesper Wilhelmsson wrote: >>> Hi Andrey, >>> >>> Is this the preferred way over using @requires? >>> /Jesper >>> >>> Andrey Zakharov skrev 13/11/14 18:05: >>>> Hi, team. Excuse me for disturbing you, here is simple fix for subj. >>>> I've added explicit -XX:-ExplicitGCInvokesConcurrent to invocations >>>> >>>> Its fixes at least twu test bugs: >>>> https://bugs.openjdk.java.net/browse/JDK-8062957 >>>> https://bugs.openjdk.java.net/browse/JDK-8064716 >>>> >>>> webrev: >>>> http://cr.openjdk.java.net/~azakharov/8064716/webrev/ >>>> >>>> >>>> Thanks. >>>> >>> >> > From bengt.rutisson at oracle.com Fri Nov 14 09:16:36 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 10:16:36 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B8C0.6050300@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> <5464B8C0.6050300@oracle.com> Message-ID: <5465C874.10504@oracle.com> On 2014-11-13 14:57, Dmitry Fazunenko wrote: > > On 13.11.2014 17:59, Bengt Rutisson wrote: >> >> On 2014-11-13 13:56, Dmitry Fazunenko wrote: >>> >>> On 13.11.2014 17:42, Bengt Rutisson wrote: >>>> >>>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>>> >>>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Evgeniya, >>>>>> >>>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>>> Hi Dmitry, >>>>>>> >>>>>>> You are right - I've forgotten about copyrights >>>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>>> >>>>>> >>>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would >>>>>> be good to add -XX:+UseG1GC to the @run tags and then use >>>>>> @requires vm.gc=="G1" | vm.gc == null. >>>>>> >>>>>> >>>>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated >>>>>> to the conflicting GC combinations. Should that really be part of >>>>>> this changeset? >>>>>> >>>>>> >>>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >>>>>> really need @requires for them? >>>>> >>>>> Yes, we do. >>>>> These tests use TestShrinkAuxiliaryData class which implements its >>>>> own mechanism to analyze VM options an skip if not applicable >>>>> collector is given. @requires - allows to rely on jtreg. >>>>> >>>>> Driver mode is a kind of indicator, that the test will spawn its >>>>> own java process. >>>> >>>> I thought the point of @driver was that no external vmoptions were >>>> passed to such a test. Is that not the case? >>> >>> In the driver mode VM is started without external VM flags. Those >>> flags are passed to the tests via system property. >>> The driver mode is a sort of shell to start something else. >> >> Right. So, why would you need @requires on the >> TestShrinkAuxiliaryDataXX tests because the utility >> TestShrinkAuxiliaryData picks up the vm flags through >> Utils.getTestJavaOpts(). What's the point in running this in a driver >> mode when they anyway pick up the vm flags? > > TestShrinkAuxiliaryData implemented a workaround awaiting for > @requires to appear in jtreg. > > Frankly speaking, the driver mode doesn't bring a lot of value, it's > rather confusing and obligate developers to be more careful. If a > class just spawns another java process with a real test, it's a big > deal to run this class with or without external options. But there is > no guarantee, that people will not start run real tests in driver mode... Ok. So, do we want to keep "driver" for this test or not? > >> >> I'm asking because adding @requires to these tests means that we will >> run them less often than we do now. So, I'm a bit worried that we >> reduce the amount of testing we do. > > Don't worry about it. > We want to run more tests, believe me. Sure, but adding @requires means that we run the test less often. The TestShrinkAuxiliaryData tests were for example run every week in PIT testing but with the @requires tag they will only be run every 4th week since PIT testing is rotating which GC it runs tests with. Bengt > > -- Dima > > >> >> Bengt >> >>> >>> -- Dima >>> >>> >>>> >>>> Bengt >>>> >>>>> >>>>> Thanks >>>>> Dima >>>>> >>>>>> >>>>>> >>>>>> Otherwise it look ok to me. >>>>>> >>>>>> Bengt >>>>>> >>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> Evgeniya Stepanova >>>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> The fix looks good to me. >>>>>>>> >>>>>>>> I noticed the following minor things: >>>>>>>> - copyrights need to include the year of last change >>>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among >>>>>>>> updated files, but doesn't contain any changes >>>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>>>> variable 'prohibitedVmOptions' >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dima >>>>>>>> >>>>>>>> >>>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>>> Hi everyone! >>>>>>>>> >>>>>>>>> Since the decision was made to change only tests that fail >>>>>>>>> because of conflict for now (skip "selfish" tests), I post new >>>>>>>>> webrev for hotspot part of the JDK-8019361 >>>>>>>>> : >>>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>>>>>> regarding @requires >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Dima, >>>>>>>>>>> >>>>>>>>>>> Thanks for the answers. I think the currently proposed patch >>>>>>>>>>> is a good start. We will have to evolve the @requires tag in >>>>>>>>>>> the future, but let's have that discussion separate from >>>>>>>>>>> this review. And we can start that discussion later when we >>>>>>>>>>> have more experience with the current version of @requires. >>>>>>>>>>> >>>>>>>>>>> Thanks for doing this! >>>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>> >>>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>>> >>>>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>>>> sort of plugins, so you can't provide a VM specific handler >>>>>>>>>>>> of the @requires or another tag. This is very annoying >>>>>>>>>>>> limitation and we have to live with it. >>>>>>>>>>>> >>>>>>>>>>>> A few more comments inline. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>> >>>>>>>>>>>>> Answers inline. >>>>>>>>>>>>> >>>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>>>>>> very much! >>>>>>>>>>>>>> >>>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>>> fix: >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>>> jtreg set another GC. >>>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>>> "requires" to skip test if there is a conflict >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for fixing this! It is really great that we >>>>>>>>>>>>>>> finally start sorting this out. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>>>> developed without much cooperation with the GC team. We >>>>>>>>>>>>>>> did have a lot of feedback when it was first presented a >>>>>>>>>>>>>>> year ago, but it does not seem like this feedback was >>>>>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>>>> >>>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>>>> expecting us to be able to influence the feature much more >>>>>>>>>>>>> than was the case now. >>>>>>>>>>>> >>>>>>>>>>>> My personal hope: @requires will address ~90% of existing >>>>>>>>>>>> issues. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>>>>>> forward and I won't object to it. But I am pretty >>>>>>>>>>>>>>> convinced that we will soon run in to the limitations of >>>>>>>>>>>>>>> the current @requires implementation and we will have to >>>>>>>>>>>>>>> redo this work. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Some of the points I don't really like about the >>>>>>>>>>>>>>> @requires tag are: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. >>>>>>>>>>>>>>> It would have been better to just "require" any command >>>>>>>>>>>>>>> line flag. >>>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>>>> possible to use: >>>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>>> >>>>>>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>>>>>> that it makes it very hard to change. When we discussed >>>>>>>>>>>>> this a year ago I think we said that JTreg should only >>>>>>>>>>>>> provide a means to test against the command line and a >>>>>>>>>>>>> hook for running some java code in the @requires tag. That >>>>>>>>>>>>> way we could put logic like this in a test library that is >>>>>>>>>>>>> under our control. This would make it easy for us to >>>>>>>>>>>>> change and also enables us to use different logic for >>>>>>>>>>>>> different versions. >>>>>>>>>>>> >>>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>>>> have to do what you did in this change and use >>>>>>>>>>>>>>> vm.gc=null even when some tests could actually have been >>>>>>>>>>>>>>> run when a GC was specified. >>>>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, >>>>>>>>>>>>>> as well as test case support. >>>>>>>>>>>>> >>>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>>> >>>>>>>>>>>> Under test case support I mean ability to treat each @run >>>>>>>>>>>> as a separate test. Now >>>>>>>>>>>> >>>>>>>>>>>> @test >>>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>>> class MyTest { >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> is always a single test. You can't exclude, or re-run a >>>>>>>>>>>> part of it. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>>>> >>>>>>>>>>>>>> yes. conflicting GC is just the most popular problem >>>>>>>>>>>>>> caused by conflicting options. >>>>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>>>> solution, we could move further. >>>>>>>>>>>>> >>>>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>>>> Personally I would have preferred that the first step was >>>>>>>>>>>>> a "just run the command line as specified in the @run tag" >>>>>>>>>>>>> step. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>>>>>> implementation of the @requires tag. I just want to say >>>>>>>>>>>>>>> that I'm not too happy about how the @requires tag >>>>>>>>>>>>>>> turned out. But assuming we have to use it the way it is >>>>>>>>>>>>>>> now I guess the proposed changeset looks good. >>>>>>>>>>>>>> >>>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>>>> about jtreg :) >>>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>>> >>>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have >>>>>>>>>>>>> not looked at all tests. But if they now pass with the >>>>>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>>>>> >>>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests >>>>>>>>>>>>>>>> are being excluded as expected. No tests failed because >>>>>>>>>>>>>>>> of the conflict. >>>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>>> >>>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will >>>>>>>>>>>>>> submit) >>>>>>>>>>>>> >>>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail >>>>>>>>>>>>>>> if you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail >>>>>>>>>>>>>>> if you run with -XX:-UseParNewGC. Could you test these >>>>>>>>>>>>>>> two cases too? >>>>>>>>>>>>>> >>>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>>> Add @requires here is not necessary, but it will allow >>>>>>>>>>>>>> not execute the tests when not needed. >>>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>>>> >>>>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>>>> this purpose? It seems like a way of misusing @requires. >>>>>>>>>>>>> If we just want the tests to be run once I think Leonid's >>>>>>>>>>>>> approach with tests lists seems more suitable. >>>>>>>>>>>> >>>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> But are you sure that this is the reason for the @requires >>>>>>>>>>>>> in this case? TestDefNewCMS does sound like a test that is >>>>>>>>>>>>> DefNew specific. I don't see a reason to run it with >>>>>>>>>>>>> ParNew. If it doesn't fail today it should probably be >>>>>>>>>>>>> changed so that it does fail if it is run with the wrong GC. >>>>>>>>>>>> >>>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy >>>>>>>>>>>> way to solve a lot of issues. >>>>>>>>>>>> >>>>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>>>> "selfish" tests, which produce a new java process to ignore >>>>>>>>>>>> vm flags coming from outside. No @requires, no other >>>>>>>>>>>> mechanism could 100% protect a test from running with >>>>>>>>>>>> conflicting options, but this is not the goal. >>>>>>>>>>>> >>>>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. >>>>>>>>>>>>>>> I'm about to push a changeset that removes them: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is there some way of making sure that all tests are run >>>>>>>>>>>>>>> at one time or another. With this change there is a risk >>>>>>>>>>>>>>> that some tests are never run and always skipped. Will >>>>>>>>>>>>>>> we somehow be tracking what gets skipped and make sure >>>>>>>>>>>>>>> that all tests are at least run once with the correct GC >>>>>>>>>>>>>>> so that it is not skipped all the time? >>>>>>>>>>>>>> >>>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will >>>>>>>>>>>>>> do soon, after getting fix of: >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>>> >>>>>>>>>>>>>> And yes, tracking tests which are not run is important >>>>>>>>>>>>>> thing. >>>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Other examples: >>>>>>>>>>>>>> >>>>>>>>>>>>>> /* >>>>>>>>>>>>>> *@ignore >>>>>>>>>>>>>> *@test >>>>>>>>>>>>>> */ >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> >>>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>>> *@test >>>>>>>>>>>>>> */ >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> Such tests will never be run, because jtreg treats as >>>>>>>>>>>>>> test only files with @test on the first place... >>>>>>>>>>>>>> >>>>>>>>>>>>>> So, making sure that tests do not disappear is important >>>>>>>>>>>>>> SQE task, we know about that, we're thinking on solution >>>>>>>>>>>>>> (may be very actively). But this subject for another >>>>>>>>>>>>>> discussion, not within RFR :) >>>>>>>>>>>>> >>>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>>> >>>>>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>>>>> know about this problem. With introducing @requires >>>>>>>>>>>> mechanism it will become more important! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your comments! >>>>>>>>>>>> >>>>>>>>>>>> -- Dima >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Bengt >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Dima >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /Evgeniya Stepanova/ >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Fri Nov 14 09:55:07 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 10:55:07 +0100 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> Message-ID: <5465D17B.3030209@oracle.com> Hi Kim, On 2014-11-13 23:13, Kim Barrett wrote: > On Nov 12, 2014, at 11:01 PM, Jon Masamitsu wrote: >> When any promotion failure occurs the concurrent CMS collections is >> abandoned and a full collection is done instead. There is some >> cleanup that is done to get the heap back into a state where the >> full collection can be used. > When any promotion failure occurs the concurrent CMS collections is > abandoned and a full collection is done instead. There is some > cleanup that is done to get the heap back into a state where the > full collection can be used. > > That's what I suspected. Is the same true for other collectors that > use par_promote()? (I think that's just ParallelOld?) I'm going to > guess so for now. > > In that case I wonder what is really going on and whether the proposed > change is really the best solution. > > I've not run the reproducer program from the bug report and examined > its behavior, so my current understanding of the problem is based on > the proposed change and the discussion surrounding it. > > At a high level, when promotion fails for some object it may take a > long time for the survivor processing to come to a halt and allow > follow-on work to occur. > > Based on the proposed change, part of the delay is that there continue > to be promotion attempts, and those attempts are serialized while > each, in turn, re-attempts work that will (probably?) fail. This is > addressed through the introduction of the new flag, setting it at > promotion failure, and checking it on entry to the lock context to > fail fast and avoid the re-attempted work. > > However, during discussion of the proposed change it was stated that, > while checking the new flag on lock context entry is an improvement, > there is still a significant delay with just that change, and that > this is due to contention for the lock. This is the rationale for > introducing flag checks outside the lock context. > > This indicates that even after promotion fails the threads involved > continue to make *many* promotion attempts. That leads me to wonder > why we continue slamming par_promote() when we're just going to later > abandon the collection in progress and start over with a full GC. Why > don't we just stop calling par_promote() at all once we've had a > promotion failure? > > The obvious first place to look is the caller(s) of par_promote(), > e.g. ParNewGeneration::copy_to_survivor_space[_XXX_undo](). Promotion > failure is handled there by setting an already existing flag and doing > some additional work. It seems like it might work to just change the > calls to par_promote() there to be conditional on that existing flag. > This would make the checks of the proposed new flag outside the lock > context redundant. If it is acceptable to have a few lingering wasted > calls occur (which seems likely to me) then this conditionalization of > calls to par_promote() would entirely eliminate the need for the > proposed new flag. Do you mean something like this? diff --git a/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp b/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp --- a/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp +++ b/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp @@ -1196,8 +1196,10 @@ return real_forwardee(old); } + if (!_promotion_failed) { new_obj = _next_gen->par_promote(par_scan_state->thread_num(), old, m, sz); + } if (new_obj == NULL) { // promotion failed, forward to self I tried this with the reproducer that Jungwoo has. It gives the same good results as the originally suggested patch. Without my patch above I get a 113 seconds long pause when we hit the promotion failure. #7: [GC (Allocation Failure) #7: [ParNew (promotion failed): 838912K->943744K(943744K), 110,3634457 secs]#8: [CMS#6: [CMS-concurrent-sweep: 0,666/111,030 secs] [Times: user=164,88 sys=33,27 real=111,04 secs] (concurrent mode failure): 1048335K->1048575K(1048576K), 2,6529365 secs] 1567474K->1170279K(1992320K), [Metaspace: 2454K->2454K(1056768K)], 113,0165630 secs] [Times: user=166,87 sys=33,24 real=113,02 secs] With the above patch the pause is 5 seconds: #7: [GC (Allocation Failure) #7: [ParNew#6: [CMS-concurrent-abortable-preclean: 0,006/0,306 secs] [Times: user=0,53 sys=0,07 real=0,31 secs] (promotion failed): 838912K->943744K(943744K), 2,5426524 secs]#8: [CMS (concurrent mode failure): 1048381K->1048575K(1048576K), 2,3803190 secs] 1609214K->1170277K(1992320K), [Metaspace: 2454K->2454K(1056768K)], 4,9231005 secs] [Times: user=11,31 sys=0,91 real=4,92 secs] So, I think it is fair to say that it is unnecessary to add the new state variable. I also agree that the existing flag has the same race behavior as the new one that was proposed. While I agree with Kim that racing on a variable is always scary I think it is pretty safe in this case. We only race on setting it from false to true. And if anybody sees a stale false value they will just take a more expensive path in the code. Nothing will break. > > It seems like an even better approach would be to arrange to terminate > the iteration over surviving objects early once promotion failure has > occurred, rather than continuing to perform work that will ultimately > be discarded. That looks to me like a much harder change to make > though. (For one thing, I'm not sure all the work being done would > actually be discarded. And there is cleanup to be done to restore the > heap to a state ready for the full GC, and that might be complicated > by having the earlier iteration terminate early.) Unfortunately it is not possible to stop iteration over all objects in the young gen. We still need to scan them all to update their references. Even objects we have not yet scanned may have references to objects that we moved before we got the promotion failure. So we can not immediately abandon scanning and go to to a full GC when we hit a promotion failure. First we need to make sure that all objects have their references updated. Thanks, Bengt > > Summing up, I currently think conditionalizing the calls to > par_promote() on the existing _promotion_failed flag is a simpler and > better solution than the proposed introduction of a new flag. (There > may need to be some barriers added to support this; if so, similar > barriers would be needed for the proposed new flag if it is to be > accessed outside the lock context.) > > From dmitry.fazunenko at oracle.com Fri Nov 14 10:02:07 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Fri, 14 Nov 2014 14:02:07 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5465C874.10504@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> <5464B8C0.6050300@oracle.com> <5465C874.10504@oracle.com> Message-ID: <5465D31F.2040002@oracle.com> Hi Bengt, On 14.11.2014 13:16, Bengt Rutisson wrote: > > On 2014-11-13 14:57, Dmitry Fazunenko wrote: >> >> On 13.11.2014 17:59, Bengt Rutisson wrote: >>> >>> On 2014-11-13 13:56, Dmitry Fazunenko wrote: >>>> >>>> On 13.11.2014 17:42, Bengt Rutisson wrote: >>>>> >>>>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>>>> >>>>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>>>> Hi Dmitry, >>>>>>>> >>>>>>>> You are right - I've forgotten about copyrights >>>>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>>>> >>>>>>> >>>>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it >>>>>>> would be good to add -XX:+UseG1GC to the @run tags and then use >>>>>>> @requires vm.gc=="G1" | vm.gc == null. >>>>>>> >>>>>>> >>>>>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated >>>>>>> to the conflicting GC combinations. Should that really be part >>>>>>> of this changeset? >>>>>>> >>>>>>> >>>>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do >>>>>>> we really need @requires for them? >>>>>> >>>>>> Yes, we do. >>>>>> These tests use TestShrinkAuxiliaryData class which implements >>>>>> its own mechanism to analyze VM options an skip if not applicable >>>>>> collector is given. @requires - allows to rely on jtreg. >>>>>> >>>>>> Driver mode is a kind of indicator, that the test will spawn its >>>>>> own java process. >>>>> >>>>> I thought the point of @driver was that no external vmoptions were >>>>> passed to such a test. Is that not the case? >>>> >>>> In the driver mode VM is started without external VM flags. Those >>>> flags are passed to the tests via system property. >>>> The driver mode is a sort of shell to start something else. >>> >>> Right. So, why would you need @requires on the >>> TestShrinkAuxiliaryDataXX tests because the utility >>> TestShrinkAuxiliaryData picks up the vm flags through >>> Utils.getTestJavaOpts(). What's the point in running this in a >>> driver mode when they anyway pick up the vm flags? >> >> TestShrinkAuxiliaryData implemented a workaround awaiting for >> @requires to appear in jtreg. >> >> Frankly speaking, the driver mode doesn't bring a lot of value, it's >> rather confusing and obligate developers to be more careful. If a >> class just spawns another java process with a real test, it's a big >> deal to run this class with or without external options. But there is >> no guarantee, that people will not start run real tests in driver mode... > > Ok. So, do we want to keep "driver" for this test or not? I believe yes: TestShrinkAuxiliaryData - is a real test TestShrinkAuxiliaryDataXX - are drivers that run TestShrinkAuxiliaryData with various options So, this is a good example of usage the 'driver' concept. > >> >>> >>> I'm asking because adding @requires to these tests means that we >>> will run them less often than we do now. So, I'm a bit worried that >>> we reduce the amount of testing we do. >> >> Don't worry about it. >> We want to run more tests, believe me. > > Sure, but adding @requires means that we run the test less often. The > TestShrinkAuxiliaryData tests were for example run every week in PIT > testing but with the @requires tag they will only be run every 4th > week since PIT testing is rotating which GC it runs tests with. We don't specify GC for PIT testing, so it will be executed every week. In promotion testing we specify two GC, so the test will be executed every 2nd week. In nightly testing it will be executed every day. Thanks, Dima > > Bengt > >> >> -- Dima >> >> >>> >>> Bengt >>> >>>> >>>> -- Dima >>>> >>>> >>>>> >>>>> Bengt >>>>> >>>>>> >>>>>> Thanks >>>>>> Dima >>>>>> >>>>>>> >>>>>>> >>>>>>> Otherwise it look ok to me. >>>>>>> >>>>>>> Bengt >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> Evgeniya Stepanova >>>>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>>>> Hi Evgeniya, >>>>>>>>> >>>>>>>>> The fix looks good to me. >>>>>>>>> >>>>>>>>> I noticed the following minor things: >>>>>>>>> - copyrights need to include the year of last change >>>>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among >>>>>>>>> updated files, but doesn't contain any changes >>>>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>>>>> variable 'prohibitedVmOptions' >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>>>> Hi everyone! >>>>>>>>>> >>>>>>>>>> Since the decision was made to change only tests that fail >>>>>>>>>> because of conflict for now (skip "selfish" tests), I post >>>>>>>>>> new webrev for hotspot part of the JDK-8019361 >>>>>>>>>> : >>>>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>>>> Nice plan! Please feel free to send me any >>>>>>>>>>> feedback/questions regarding @requires >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Dima, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for the answers. I think the currently proposed >>>>>>>>>>>> patch is a good start. We will have to evolve the @requires >>>>>>>>>>>> tag in the future, but let's have that discussion separate >>>>>>>>>>>> from this review. And we can start that discussion later >>>>>>>>>>>> when we have more experience with the current version of >>>>>>>>>>>> @requires. >>>>>>>>>>>> >>>>>>>>>>>> Thanks for doing this! >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>> >>>>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>>>> >>>>>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>>>>> sort of plugins, so you can't provide a VM specific >>>>>>>>>>>>> handler of the @requires or another tag. This is very >>>>>>>>>>>>> annoying limitation and we have to live with it. >>>>>>>>>>>>> >>>>>>>>>>>>> A few more comments inline. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Answers inline. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate >>>>>>>>>>>>>>> it very much! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>>>> fix: >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>>>> jtreg set another GC. >>>>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>>>> "requires" to skip test if there is a conflict >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for fixing this! It is really great that we >>>>>>>>>>>>>>>> finally start sorting this out. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>>>>> developed without much cooperation with the GC team. We >>>>>>>>>>>>>>>> did have a lot of feedback when it was first presented >>>>>>>>>>>>>>>> a year ago, but it does not seem like this feedback was >>>>>>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>>>>> expecting us to be able to influence the feature much >>>>>>>>>>>>>> more than was the case now. >>>>>>>>>>>>> >>>>>>>>>>>>> My personal hope: @requires will address ~90% of existing >>>>>>>>>>>>> issues. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think this change that gets proposed now is a big >>>>>>>>>>>>>>>> step forward and I won't object to it. But I am pretty >>>>>>>>>>>>>>>> convinced that we will soon run in to the limitations >>>>>>>>>>>>>>>> of the current @requires implementation and we will >>>>>>>>>>>>>>>> have to redo this work. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Some of the points I don't really like about the >>>>>>>>>>>>>>>> @requires tag are: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than >>>>>>>>>>>>>>>> helping. It would have been better to just "require" >>>>>>>>>>>>>>>> any command line flag. >>>>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>>>>> possible to use: >>>>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>>>> >>>>>>>>>>>>>> The problem with having this matching built in to JTreg >>>>>>>>>>>>>> is that it makes it very hard to change. When we >>>>>>>>>>>>>> discussed this a year ago I think we said that JTreg >>>>>>>>>>>>>> should only provide a means to test against the command >>>>>>>>>>>>>> line and a hook for running some java code in the >>>>>>>>>>>>>> @requires tag. That way we could put logic like this in a >>>>>>>>>>>>>> test library that is under our control. This would make >>>>>>>>>>>>>> it easy for us to change and also enables us to use >>>>>>>>>>>>>> different logic for different versions. >>>>>>>>>>>>> >>>>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>>>>> have to do what you did in this change and use >>>>>>>>>>>>>>>> vm.gc=null even when some tests could actually have >>>>>>>>>>>>>>>> been run when a GC was specified. >>>>>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, >>>>>>>>>>>>>>> as well as test case support. >>>>>>>>>>>>>> >>>>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>>>> >>>>>>>>>>>>> Under test case support I mean ability to treat each @run >>>>>>>>>>>>> as a separate test. Now >>>>>>>>>>>>> >>>>>>>>>>>>> @test >>>>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>>>> class MyTest { >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> is always a single test. You can't exclude, or re-run a >>>>>>>>>>>>> part of it. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> yes. conflicting GC is just the most popular problem >>>>>>>>>>>>>>> caused by conflicting options. >>>>>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>>>>> solution, we could move further. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>>>>> Personally I would have preferred that the first step was >>>>>>>>>>>>>> a "just run the command line as specified in the @run >>>>>>>>>>>>>> tag" step. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Maybe this is not the right place to discuss the >>>>>>>>>>>>>>>> current implementation of the @requires tag. I just >>>>>>>>>>>>>>>> want to say that I'm not too happy about how the >>>>>>>>>>>>>>>> @requires tag turned out. But assuming we have to use >>>>>>>>>>>>>>>> it the way it is now I guess the proposed changeset >>>>>>>>>>>>>>>> looks good. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>>>>> about jtreg :) >>>>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>>>> >>>>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have >>>>>>>>>>>>>> not looked at all tests. But if they now pass with the >>>>>>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>>>>>> >>>>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests >>>>>>>>>>>>>>>>> are being excluded as expected. No tests failed >>>>>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will >>>>>>>>>>>>>>> submit) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail >>>>>>>>>>>>>>>> if you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail >>>>>>>>>>>>>>>> if you run with -XX:-UseParNewGC. Could you test these >>>>>>>>>>>>>>>> two cases too? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>>>> Add @requires here is not necessary, but it will allow >>>>>>>>>>>>>>> not execute the tests when not needed. >>>>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>>>>> this purpose? It seems like a way of misusing @requires. >>>>>>>>>>>>>> If we just want the tests to be run once I think Leonid's >>>>>>>>>>>>>> approach with tests lists seems more suitable. >>>>>>>>>>>>> >>>>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> But are you sure that this is the reason for the >>>>>>>>>>>>>> @requires in this case? TestDefNewCMS does sound like a >>>>>>>>>>>>>> test that is DefNew specific. I don't see a reason to run >>>>>>>>>>>>>> it with ParNew. If it doesn't fail today it should >>>>>>>>>>>>>> probably be changed so that it does fail if it is run >>>>>>>>>>>>>> with the wrong GC. >>>>>>>>>>>>> >>>>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy >>>>>>>>>>>>> way to solve a lot of issues. >>>>>>>>>>>>> >>>>>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>>>>> "selfish" tests, which produce a new java process to >>>>>>>>>>>>> ignore vm flags coming from outside. No @requires, no >>>>>>>>>>>>> other mechanism could 100% protect a test from running >>>>>>>>>>>>> with conflicting options, but this is not the goal. >>>>>>>>>>>>> >>>>>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. >>>>>>>>>>>>>>>> I'm about to push a changeset that removes them: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Is there some way of making sure that all tests are run >>>>>>>>>>>>>>>> at one time or another. With this change there is a >>>>>>>>>>>>>>>> risk that some tests are never run and always skipped. >>>>>>>>>>>>>>>> Will we somehow be tracking what gets skipped and make >>>>>>>>>>>>>>>> sure that all tests are at least run once with the >>>>>>>>>>>>>>>> correct GC so that it is not skipped all the time? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it >>>>>>>>>>>>>>> will do soon, after getting fix of: >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> And yes, tracking tests which are not run is important >>>>>>>>>>>>>>> thing. >>>>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Other examples: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> /* >>>>>>>>>>>>>>> *@ignore >>>>>>>>>>>>>>> *@test >>>>>>>>>>>>>>> */ >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>>>> *@test >>>>>>>>>>>>>>> */ >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> Such tests will never be run, because jtreg treats as >>>>>>>>>>>>>>> test only files with @test on the first place... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So, making sure that tests do not disappear is >>>>>>>>>>>>>>> important SQE task, we know about that, we're thinking >>>>>>>>>>>>>>> on solution (may be very actively). But this subject >>>>>>>>>>>>>>> for another discussion, not within RFR :) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>>>> >>>>>>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>>>>>> know about this problem. With introducing @requires >>>>>>>>>>>>> mechanism it will become more important! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your comments! >>>>>>>>>>>>> >>>>>>>>>>>>> -- Dima >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Dima >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Fri Nov 14 11:47:58 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 12:47:58 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5465D31F.2040002@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> <5464B8C0.6050300@oracle.com> <5465C874.10504@oracle.com> <5465D31F.2040002@oracle.com> Message-ID: <5465EBEE.3010900@oracle.com> On 2014-11-14 11:02, Dmitry Fazunenko wrote: > Hi Bengt, > > On 14.11.2014 13:16, Bengt Rutisson wrote: >> >> On 2014-11-13 14:57, Dmitry Fazunenko wrote: >>> >>> On 13.11.2014 17:59, Bengt Rutisson wrote: >>>> >>>> On 2014-11-13 13:56, Dmitry Fazunenko wrote: >>>>> >>>>> On 13.11.2014 17:42, Bengt Rutisson wrote: >>>>>> >>>>>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>>>>> >>>>>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>>>>> Hi Dmitry, >>>>>>>>> >>>>>>>>> You are right - I've forgotten about copyrights >>>>>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>>>>> >>>>>>>> >>>>>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it >>>>>>>> would be good to add -XX:+UseG1GC to the @run tags and then >>>>>>>> use @requires vm.gc=="G1" | vm.gc == null. >>>>>>>> >>>>>>>> >>>>>>>> The change to test/gc/defnew/HeapChangeLogging.java is >>>>>>>> unrelated to the conflicting GC combinations. Should that >>>>>>>> really be part of this changeset? >>>>>>>> >>>>>>>> >>>>>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do >>>>>>>> we really need @requires for them? >>>>>>> >>>>>>> Yes, we do. >>>>>>> These tests use TestShrinkAuxiliaryData class which implements >>>>>>> its own mechanism to analyze VM options an skip if not >>>>>>> applicable collector is given. @requires - allows to rely on jtreg. >>>>>>> >>>>>>> Driver mode is a kind of indicator, that the test will spawn its >>>>>>> own java process. >>>>>> >>>>>> I thought the point of @driver was that no external vmoptions >>>>>> were passed to such a test. Is that not the case? >>>>> >>>>> In the driver mode VM is started without external VM flags. Those >>>>> flags are passed to the tests via system property. >>>>> The driver mode is a sort of shell to start something else. >>>> >>>> Right. So, why would you need @requires on the >>>> TestShrinkAuxiliaryDataXX tests because the utility >>>> TestShrinkAuxiliaryData picks up the vm flags through >>>> Utils.getTestJavaOpts(). What's the point in running this in a >>>> driver mode when they anyway pick up the vm flags? >>> >>> TestShrinkAuxiliaryData implemented a workaround awaiting for >>> @requires to appear in jtreg. >>> >>> Frankly speaking, the driver mode doesn't bring a lot of value, it's >>> rather confusing and obligate developers to be more careful. If a >>> class just spawns another java process with a real test, it's a big >>> deal to run this class with or without external options. But there >>> is no guarantee, that people will not start run real tests in driver >>> mode... >> >> Ok. So, do we want to keep "driver" for this test or not? > > I believe yes: > TestShrinkAuxiliaryData - is a real test > TestShrinkAuxiliaryDataXX - are drivers that run > TestShrinkAuxiliaryData with various options > > So, this is a good example of usage the 'driver' concept. Ok. Sounds good. > >> >>> >>>> >>>> I'm asking because adding @requires to these tests means that we >>>> will run them less often than we do now. So, I'm a bit worried that >>>> we reduce the amount of testing we do. >>> >>> Don't worry about it. >>> We want to run more tests, believe me. >> >> Sure, but adding @requires means that we run the test less often. The >> TestShrinkAuxiliaryData tests were for example run every week in PIT >> testing but with the @requires tag they will only be run every 4th >> week since PIT testing is rotating which GC it runs tests with. > We don't specify GC for PIT testing, so it will be executed every week. > In promotion testing we specify two GC, so the test will be executed > every 2nd week. > In nightly testing it will be executed every day. Good. Thanks for clarifying. I obviously had PIT and promotion testing mixed up. In that case I think you can consider this change reviewed from my side. Thanks for putting up with all the questions! Bengt > > Thanks, > Dima > >> >> Bengt >> >>> >>> -- Dima >>> >>> >>>> >>>> Bengt >>>> >>>>> >>>>> -- Dima >>>>> >>>>> >>>>>> >>>>>> Bengt >>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> Dima >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Otherwise it look ok to me. >>>>>>>> >>>>>>>> Bengt >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Evgeniya Stepanova >>>>>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> The fix looks good to me. >>>>>>>>>> >>>>>>>>>> I noticed the following minor things: >>>>>>>>>> - copyrights need to include the year of last change >>>>>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among >>>>>>>>>> updated files, but doesn't contain any changes >>>>>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>>>>>> variable 'prohibitedVmOptions' >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi everyone! >>>>>>>>>>> >>>>>>>>>>> Since the decision was made to change only tests that fail >>>>>>>>>>> because of conflict for now (skip "selfish" tests), I post >>>>>>>>>>> new webrev for hotspot part of the JDK-8019361 >>>>>>>>>>> : >>>>>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>>>>> Nice plan! Please feel free to send me any >>>>>>>>>>>> feedback/questions regarding @requires >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Dima >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the answers. I think the currently proposed >>>>>>>>>>>>> patch is a good start. We will have to evolve the >>>>>>>>>>>>> @requires tag in the future, but let's have that >>>>>>>>>>>>> discussion separate from this review. And we can start >>>>>>>>>>>>> that discussion later when we have more experience with >>>>>>>>>>>>> the current version of @requires. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for doing this! >>>>>>>>>>>>> Bengt >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>>> >>>>>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>>>>> >>>>>>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>>>>>> sort of plugins, so you can't provide a VM specific >>>>>>>>>>>>>> handler of the @requires or another tag. This is very >>>>>>>>>>>>>> annoying limitation and we have to live with it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> A few more comments inline. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Answers inline. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate >>>>>>>>>>>>>>>> it very much! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Please review changes for 8062537, the >>>>>>>>>>>>>>>>>> OpenJDK/hotspot part of the JDK-8019361 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>>>>> fix: >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>>>>> jtreg set another GC. >>>>>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>>>>> "requires" to skip test if there is a conflict >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for fixing this! It is really great that we >>>>>>>>>>>>>>>>> finally start sorting this out. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>>>>>> developed without much cooperation with the GC team. >>>>>>>>>>>>>>>>> We did have a lot of feedback when it was first >>>>>>>>>>>>>>>>> presented a year ago, but it does not seem like this >>>>>>>>>>>>>>>>> feedback was incorporated into the @requires that was >>>>>>>>>>>>>>>>> eventually built. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>>>>>> expecting us to be able to influence the feature much >>>>>>>>>>>>>>> more than was the case now. >>>>>>>>>>>>>> >>>>>>>>>>>>>> My personal hope: @requires will address ~90% of existing >>>>>>>>>>>>>> issues. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think this change that gets proposed now is a big >>>>>>>>>>>>>>>>> step forward and I won't object to it. But I am pretty >>>>>>>>>>>>>>>>> convinced that we will soon run in to the limitations >>>>>>>>>>>>>>>>> of the current @requires implementation and we will >>>>>>>>>>>>>>>>> have to redo this work. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Some of the points I don't really like about the >>>>>>>>>>>>>>>>> @requires tag are: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than >>>>>>>>>>>>>>>>> helping. It would have been better to just "require" >>>>>>>>>>>>>>>>> any command line flag. >>>>>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>>>>>> possible to use: >>>>>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The problem with having this matching built in to JTreg >>>>>>>>>>>>>>> is that it makes it very hard to change. When we >>>>>>>>>>>>>>> discussed this a year ago I think we said that JTreg >>>>>>>>>>>>>>> should only provide a means to test against the command >>>>>>>>>>>>>>> line and a hook for running some java code in the >>>>>>>>>>>>>>> @requires tag. That way we could put logic like this in >>>>>>>>>>>>>>> a test library that is under our control. This would >>>>>>>>>>>>>>> make it easy for us to change and also enables us to use >>>>>>>>>>>>>>> different logic for different versions. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>>>>>> have to do what you did in this change and use >>>>>>>>>>>>>>>>> vm.gc=null even when some tests could actually have >>>>>>>>>>>>>>>>> been run when a GC was specified. >>>>>>>>>>>>>>>> it would be great, but it will unlikely happen in >>>>>>>>>>>>>>>> jtreg, as well as test case support. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Under test case support I mean ability to treat each @run >>>>>>>>>>>>>> as a separate test. Now >>>>>>>>>>>>>> >>>>>>>>>>>>>> @test >>>>>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>>>>> class MyTest { >>>>>>>>>>>>>> } >>>>>>>>>>>>>> >>>>>>>>>>>>>> is always a single test. You can't exclude, or re-run a >>>>>>>>>>>>>> part of it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> yes. conflicting GC is just the most popular problem >>>>>>>>>>>>>>>> caused by conflicting options. >>>>>>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>>>>>> solution, we could move further. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>>>>>> Personally I would have preferred that the first step >>>>>>>>>>>>>>> was a "just run the command line as specified in the >>>>>>>>>>>>>>> @run tag" step. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Maybe this is not the right place to discuss the >>>>>>>>>>>>>>>>> current implementation of the @requires tag. I just >>>>>>>>>>>>>>>>> want to say that I'm not too happy about how the >>>>>>>>>>>>>>>>> @requires tag turned out. But assuming we have to use >>>>>>>>>>>>>>>>> it the way it is now I guess the proposed changeset >>>>>>>>>>>>>>>>> looks good. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>>>>>> about jtreg :) >>>>>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I >>>>>>>>>>>>>>> have not looked at all tests. But if they now pass with >>>>>>>>>>>>>>> the combinations that we test with I guess they should >>>>>>>>>>>>>>> be ok. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests >>>>>>>>>>>>>>>>>> are being excluded as expected. No tests failed >>>>>>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will >>>>>>>>>>>>>>>> submit) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail >>>>>>>>>>>>>>>>> if you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail >>>>>>>>>>>>>>>>> if you run with -XX:-UseParNewGC. Could you test these >>>>>>>>>>>>>>>>> two cases too? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>>>>> Add @requires here is not necessary, but it will allow >>>>>>>>>>>>>>>> not execute the tests when not needed. >>>>>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>>>>>> this purpose? It seems like a way of misusing @requires. >>>>>>>>>>>>>>> If we just want the tests to be run once I think >>>>>>>>>>>>>>> Leonid's approach with tests lists seems more suitable. >>>>>>>>>>>>>> >>>>>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> But are you sure that this is the reason for the >>>>>>>>>>>>>>> @requires in this case? TestDefNewCMS does sound like a >>>>>>>>>>>>>>> test that is DefNew specific. I don't see a reason to >>>>>>>>>>>>>>> run it with ParNew. If it doesn't fail today it should >>>>>>>>>>>>>>> probably be changed so that it does fail if it is run >>>>>>>>>>>>>>> with the wrong GC. >>>>>>>>>>>>>> >>>>>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy >>>>>>>>>>>>>> way to solve a lot of issues. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>>>>>> "selfish" tests, which produce a new java process to >>>>>>>>>>>>>> ignore vm flags coming from outside. No @requires, no >>>>>>>>>>>>>> other mechanism could 100% protect a test from running >>>>>>>>>>>>>> with conflicting options, but this is not the goal. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>>>>>> collector, there shouldn't mass failures caused by >>>>>>>>>>>>>> options conflicts. But a few failures could be handled >>>>>>>>>>>>>> manually. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Similarly it looks to me like there are tests that >>>>>>>>>>>>>>>>> will fail if you run them with -XX:-UseParallelOldGC >>>>>>>>>>>>>>>>> or -XX:+UseParallelOldGC. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. >>>>>>>>>>>>>>>>> I'm about to push a changeset that removes them: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Is there some way of making sure that all tests are >>>>>>>>>>>>>>>>> run at one time or another. With this change there is >>>>>>>>>>>>>>>>> a risk that some tests are never run and always >>>>>>>>>>>>>>>>> skipped. Will we somehow be tracking what gets skipped >>>>>>>>>>>>>>>>> and make sure that all tests are at least run once >>>>>>>>>>>>>>>>> with the correct GC so that it is not skipped all the >>>>>>>>>>>>>>>>> time? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it >>>>>>>>>>>>>>>> will do soon, after getting fix of: >>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> And yes, tracking tests which are not run is important >>>>>>>>>>>>>>>> thing. >>>>>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Other examples: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> /* >>>>>>>>>>>>>>>> *@ignore >>>>>>>>>>>>>>>> *@test >>>>>>>>>>>>>>>> */ >>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>>>>> *@test >>>>>>>>>>>>>>>> */ >>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>> Such tests will never be run, because jtreg treats as >>>>>>>>>>>>>>>> test only files with @test on the first place... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> So, making sure that tests do not disappear is >>>>>>>>>>>>>>>> important SQE task, we know about that, we're thinking >>>>>>>>>>>>>>>> on solution (may be very actively). But this subject >>>>>>>>>>>>>>>> for another discussion, not within RFR :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>>>>> >>>>>>>>>>>>>> I was going to say "not very actively", but never mind, >>>>>>>>>>>>>> we know about this problem. With introducing @requires >>>>>>>>>>>>>> mechanism it will become more important! >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your comments! >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- Dima >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Dima >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /Evgeniya Stepanova/ >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Fri Nov 14 12:09:12 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 14 Nov 2014 13:09:12 +0100 Subject: RFR: JDK-8064473 - Improved handling of age during object copy in G1 In-Reply-To: <54623886.4060602@oracle.com> References: <54623886.4060602@oracle.com> Message-ID: <1415966952.4679.1.camel@oracle.com> Hi, On Tue, 2014-11-11 at 17:25 +0100, Staffan Friberg wrote: > Hi, > > Small update to object copying in G1 to reduce the number of times age > has to be read and also make sure the markOop can be trusted by reading > and saving it prior doing the forwarded check. This change is broken out > from the PLAB event RFE. > > bug: https://bugs.openjdk.java.net/browse/JDK-8064473 > webrev: http://cr.openjdk.java.net/~sfriberg/JDK-8064473/webrev.01 Looks good except for a minor nit: the comment at line 203 // in this case, we have to install the mark word first, // otherwise obj looks to be forwarded (the old mark word, should start with a capital letter. I also created an RFE to fix up markOopDesc to allow nicer method names than "oop_decode_pointer", https://bugs.openjdk.java.net/browse/JDK-8064885 Thanks, Thomas From thomas.schatzl at oracle.com Fri Nov 14 12:12:07 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 14 Nov 2014 13:12:07 +0100 Subject: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent In-Reply-To: <5465C183.5020107@oracle.com> References: <5464E4ED.4040003@oracle.com> <5464F3F5.1020203@oracle.com> <5464E64A.4040307@oracle.com> <5464F5D4.9090600@oracle.com> <5465C183.5020107@oracle.com> Message-ID: <1415967127.4679.2.camel@oracle.com> Hi, On Fri, 2014-11-14 at 09:46 +0100, Bengt Rutisson wrote: > On 2014-11-13 19:17, Jesper Wilhelmsson wrote: > > That depends.. Will this work for all cases where > > -XX:+ExplicitGCInvokesConcurrent is added to the command line from > > different frameworks etc? > > Jesper, the command line flags specified in the @run tag are *appened* > to whatever the framework passes in. So, with Andrey's change the > -XX:-ExplicitGCInvokesConcurrent in the @run tag will always override > any -XX:+ExplicitGCInvokesConcurrent passed in from the framework. Note > that this is a "normal" hotspot flag (not specially treated like the GC > selection flags) so there is not conflict in specifying the same flag > multiple times on the command line. The rule "last one wins" applies. > > Andrey, the changes look good. > > One minor thing is that I would prefer the @run tag in > TestHumongousShrinkHeap.java to be a single line. I think it is a bit > hard to read when the @run tag gets split up over multiple lines: I agree with Bengt, and would also prefer if the @run tag contents were not split into multiple lines. Thanks, Thomas From andreas.eriksson at oracle.com Fri Nov 14 12:28:24 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Fri, 14 Nov 2014 13:28:24 +0100 Subject: RFR (S): Forward-port: 7176220 'Full GC' events miss date stamp information occasionally Message-ID: <5465F568.5090702@oracle.com> Hi, This fix was actually pushed to jdk7/hsx24 over a year ago by me. IIRC the push to jdk8/hsx25 was delayed at the time, over fears that it would conflict with the big metaspace change and/or JFR work that was about to get pushed. Unfortunately I then forgot all about it. I would like to push this fix to jdk9 (and then jdk8) now. Bug: https://bugs.openjdk.java.net/browse/JDK-7176220 Summary: The problem is that CMS sometimes misses datestamps for time trace printouts. The fix is to move the datestamp print logic into the class that handles time trace printouts, GCTraceTime. This makes it consistent with timestamps, which are already printed from there. Webrev for jdk9: http://cr.openjdk.java.net/~aeriksso/7176220/webrev.00/ And for reference: hsx24 review thread: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2013-May/007127.html hsx24 changeset: http://hg.openjdk.java.net/hsx/hsx24/hotspot/rev/aa01d1f5fcd9 Thanks, Andreas From bengt.rutisson at oracle.com Fri Nov 14 12:33:11 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 13:33:11 +0100 Subject: RFR (S): Forward-port: 7176220 'Full GC' events miss date stamp information occasionally In-Reply-To: <5465F568.5090702@oracle.com> References: <5465F568.5090702@oracle.com> Message-ID: <5465F687.1010501@oracle.com> Hi Andreas, Looks good. Thanks for fixing this! Bengt On 2014-11-14 13:28, Andreas Eriksson wrote: > Hi, > > This fix was actually pushed to jdk7/hsx24 over a year ago by me. > IIRC the push to jdk8/hsx25 was delayed at the time, over fears that > it would conflict with the big metaspace change and/or JFR work that > was about to get pushed. > Unfortunately I then forgot all about it. > > I would like to push this fix to jdk9 (and then jdk8) now. > > Bug: https://bugs.openjdk.java.net/browse/JDK-7176220 > > Summary: > The problem is that CMS sometimes misses datestamps for time trace > printouts. > The fix is to move the datestamp print logic into the class that > handles time trace printouts, GCTraceTime. > This makes it consistent with timestamps, which are already printed > from there. > > Webrev for jdk9: > http://cr.openjdk.java.net/~aeriksso/7176220/webrev.00/ > > And for reference: > hsx24 review thread: > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2013-May/007127.html > hsx24 changeset: > http://hg.openjdk.java.net/hsx/hsx24/hotspot/rev/aa01d1f5fcd9 > > Thanks, > Andreas From thomas.schatzl at oracle.com Fri Nov 14 12:48:24 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 14 Nov 2014 13:48:24 +0100 Subject: RFR (S): Forward-port: 7176220 'Full GC' events miss date stamp information occasionally In-Reply-To: <5465F568.5090702@oracle.com> References: <5465F568.5090702@oracle.com> Message-ID: <1415969304.4679.12.camel@oracle.com> Hi, On Fri, 2014-11-14 at 13:28 +0100, Andreas Eriksson wrote: > Hi, > > This fix was actually pushed to jdk7/hsx24 over a year ago by me. > IIRC the push to jdk8/hsx25 was delayed at the time, over fears that it > would conflict with the big metaspace change and/or JFR work that was > about to get pushed. > Unfortunately I then forgot all about it. > > I would like to push this fix to jdk9 (and then jdk8) now. Looks good. Thanks, Thomas From bengt.rutisson at oracle.com Fri Nov 14 12:42:09 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 13:42:09 +0100 Subject: RFR: JDK-8064473 - Improved handling of age during object copy in G1 In-Reply-To: <1415966952.4679.1.camel@oracle.com> References: <54623886.4060602@oracle.com> <1415966952.4679.1.camel@oracle.com> Message-ID: <5465F8A1.6070401@oracle.com> Hi Staffan, On 2014-11-14 13:09, Thomas Schatzl wrote: > Hi, > > On Tue, 2014-11-11 at 17:25 +0100, Staffan Friberg wrote: >> Hi, >> >> Small update to object copying in G1 to reduce the number of times age >> has to be read and also make sure the markOop can be trusted by reading >> and saving it prior doing the forwarded check. This change is broken out >> from the PLAB event RFE. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8064473 >> webrev: http://cr.openjdk.java.net/~sfriberg/JDK-8064473/webrev.01 > Looks good except for a minor nit: the comment at line 203 > > // in this case, we have to install the mark word first, > // otherwise obj looks to be forwarded (the old mark word, > > should start with a capital letter. > > I also created an RFE to fix up markOopDesc to allow nicer method names than "oop_decode_pointer", https://bugs.openjdk.java.net/browse/JDK-8064885 Looks good to me to modulo Thomas comment about the comment. Bengt > > Thanks, > Thomas > > From andrey.x.zakharov at oracle.com Fri Nov 14 12:52:17 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Fri, 14 Nov 2014 04:52:17 -0800 (PST) Subject: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent Message-ID: <95e2b378-d615-420b-9600-7bf09016d921@default> Thanks for review, guys. About rotating options in nighlies is a good idea because it can reveals more product issues. As this flag is "normal" and it can be used within product by everyone we should test all cases with this flag too. IMHO. I'll fix comment. Thanks. ----- Original Message ----- From: bengt.rutisson at oracle.com To: jesper.wilhelmsson at oracle.com, andrey.x.zakharov at oracle.com, hotspot-gc-dev at openjdk.java.net Sent: Friday, November 14, 2014 11:55:56 AM GMT +04:00 Abu Dhabi / Muscat Subject: Re: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent On 2014-11-13 19:17, Jesper Wilhelmsson wrote: > That depends.. Will this work for all cases where > -XX:+ExplicitGCInvokesConcurrent is added to the command line from > different frameworks etc? Jesper, the command line flags specified in the @run tag are *appened* to whatever the framework passes in. So, with Andrey's change the -XX:-ExplicitGCInvokesConcurrent in the @run tag will always override any -XX:+ExplicitGCInvokesConcurrent passed in from the framework. Note that this is a "normal" hotspot flag (not specially treated like the GC selection flags) so there is not conflict in specifying the same flag multiple times on the command line. The rule "last one wins" applies. Andrey, the changes look good. One minor thing is that I would prefer the @run tag in TestHumongousShrinkHeap.java to be a single line. I think it is a bit hard to read when the @run tag gets split up over multiple lines: 30 * @run main/othervm -XX:-ExplicitGCInvokesConcurrent -XX:MinHeapFreeRatio=10 31 * -XX:MaxHeapFreeRatio=12 -XX:+UseG1GC -XX:G1HeapRegionSize=1M -verbose:gc 32 * TestHumongousShrinkHeap Then a general question. We need this change because we some days rotate in -XX:+ExplicitGCInvokesConcurrent in our nightly testing. Is that really a good idea? (I'm sure it was requested from the GC team at some point, but maybe it is time to re-evaluate that decision.) A different way of handling these failures would be to stop rotating in that flag in our testing. I am not sure it really improves our quality and apparently it causes issues. Maybe we should just add a few tests that explicitly test with -XX:+ExplicitGCInvokesConcurrent? Thanks, Bengt > > /Jesper > > Andrey Zakharov skrev 13/11/14 18:11: >> Simplest way is the preferred way to solve issues, right? >> >> On 13.11.2014 22:09, Jesper Wilhelmsson wrote: >>> Hi Andrey, >>> >>> Is this the preferred way over using @requires? >>> /Jesper >>> >>> Andrey Zakharov skrev 13/11/14 18:05: >>>> Hi, team. Excuse me for disturbing you, here is simple fix for subj. >>>> I've added explicit -XX:-ExplicitGCInvokesConcurrent to invocations >>>> >>>> Its fixes at least twu test bugs: >>>> https://bugs.openjdk.java.net/browse/JDK-8062957 >>>> https://bugs.openjdk.java.net/browse/JDK-8064716 >>>> >>>> webrev: >>>> http://cr.openjdk.java.net/~azakharov/8064716/webrev/ >>>> >>>> >>>> Thanks. >>>> >>> >> > From bengt.rutisson at oracle.com Fri Nov 14 12:46:22 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 13:46:22 +0100 Subject: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent In-Reply-To: <95e2b378-d615-420b-9600-7bf09016d921@default> References: <95e2b378-d615-420b-9600-7bf09016d921@default> Message-ID: <5465F99E.6020909@oracle.com> On 2014-11-14 13:52, Andrey Zakharov wrote: > Thanks for review, guys. > About rotating options in nighlies is a good idea because it can reveals more product issues. > As this flag is "normal" and it can be used within product by everyone we should test all cases with this flag too. IMHO. I did not mean that we should stop testing with ExplicitGCInvokesConcurrent. I was suggesting that we should add test for this functionality rather than relying on the nightly testing to be configured the right way. Either way that can be a separate discussion topic, so I'm fine with the currently proposed change. Bengt > > > I'll fix comment. > > Thanks. > > ----- Original Message ----- > From: bengt.rutisson at oracle.com > To: jesper.wilhelmsson at oracle.com, andrey.x.zakharov at oracle.com, hotspot-gc-dev at openjdk.java.net > Sent: Friday, November 14, 2014 11:55:56 AM GMT +04:00 Abu Dhabi / Muscat > Subject: Re: [RFR] 8064716: TestHumongousShrinkHeap.java can not be run with -XX:+ExplicitGCInvokesConcurrent > > > On 2014-11-13 19:17, Jesper Wilhelmsson wrote: >> That depends.. Will this work for all cases where >> -XX:+ExplicitGCInvokesConcurrent is added to the command line from >> different frameworks etc? > Jesper, the command line flags specified in the @run tag are *appened* > to whatever the framework passes in. So, with Andrey's change the > -XX:-ExplicitGCInvokesConcurrent in the @run tag will always override > any -XX:+ExplicitGCInvokesConcurrent passed in from the framework. Note > that this is a "normal" hotspot flag (not specially treated like the GC > selection flags) so there is not conflict in specifying the same flag > multiple times on the command line. The rule "last one wins" applies. > > Andrey, the changes look good. > > One minor thing is that I would prefer the @run tag in > TestHumongousShrinkHeap.java to be a single line. I think it is a bit > hard to read when the @run tag gets split up over multiple lines: > > 30 * @run main/othervm -XX:-ExplicitGCInvokesConcurrent > -XX:MinHeapFreeRatio=10 > 31 * -XX:MaxHeapFreeRatio=12 -XX:+UseG1GC -XX:G1HeapRegionSize=1M > -verbose:gc > 32 * TestHumongousShrinkHeap > > Then a general question. We need this change because we some days rotate > in -XX:+ExplicitGCInvokesConcurrent in our nightly testing. Is that > really a good idea? (I'm sure it was requested from the GC team at some > point, but maybe it is time to re-evaluate that decision.) A different > way of handling these failures would be to stop rotating in that flag in > our testing. I am not sure it really improves our quality and apparently > it causes issues. Maybe we should just add a few tests that explicitly > test with -XX:+ExplicitGCInvokesConcurrent? > > Thanks, > Bengt > >> /Jesper >> >> Andrey Zakharov skrev 13/11/14 18:11: >>> Simplest way is the preferred way to solve issues, right? >>> >>> On 13.11.2014 22:09, Jesper Wilhelmsson wrote: >>>> Hi Andrey, >>>> >>>> Is this the preferred way over using @requires? >>>> /Jesper >>>> >>>> Andrey Zakharov skrev 13/11/14 18:05: >>>>> Hi, team. Excuse me for disturbing you, here is simple fix for subj. >>>>> I've added explicit -XX:-ExplicitGCInvokesConcurrent to invocations >>>>> >>>>> Its fixes at least twu test bugs: >>>>> https://bugs.openjdk.java.net/browse/JDK-8062957 >>>>> https://bugs.openjdk.java.net/browse/JDK-8064716 >>>>> >>>>> webrev: >>>>> http://cr.openjdk.java.net/~azakharov/8064716/webrev/ >>>>> >>>>> >>>>> Thanks. >>>>> From andreas.eriksson at oracle.com Fri Nov 14 13:04:47 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Fri, 14 Nov 2014 14:04:47 +0100 Subject: RFR (S): Forward-port: 7176220 'Full GC' events miss date stamp information occasionally In-Reply-To: <1415969304.4679.12.camel@oracle.com> References: <5465F568.5090702@oracle.com> <1415969304.4679.12.camel@oracle.com> Message-ID: <5465FDEF.1060400@oracle.com> Thanks! - Andreas On 2014-11-14 13:48, Thomas Schatzl wrote: > Hi, > > On Fri, 2014-11-14 at 13:28 +0100, Andreas Eriksson wrote: >> Hi, >> >> This fix was actually pushed to jdk7/hsx24 over a year ago by me. >> IIRC the push to jdk8/hsx25 was delayed at the time, over fears that it >> would conflict with the big metaspace change and/or JFR work that was >> about to get pushed. >> Unfortunately I then forgot all about it. >> >> I would like to push this fix to jdk9 (and then jdk8) now. > Looks good. > > Thanks, > Thomas > > From andreas.eriksson at oracle.com Fri Nov 14 13:05:15 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Fri, 14 Nov 2014 14:05:15 +0100 Subject: RFR (S): Forward-port: 7176220 'Full GC' events miss date stamp information occasionally In-Reply-To: <5465F687.1010501@oracle.com> References: <5465F568.5090702@oracle.com> <5465F687.1010501@oracle.com> Message-ID: <5465FE0B.6010701@oracle.com> Thanks for the review! - Andreas On 2014-11-14 13:33, Bengt Rutisson wrote: > > > Hi Andreas, > > Looks good. > > Thanks for fixing this! > > Bengt > > On 2014-11-14 13:28, Andreas Eriksson wrote: >> Hi, >> >> This fix was actually pushed to jdk7/hsx24 over a year ago by me. >> IIRC the push to jdk8/hsx25 was delayed at the time, over fears that >> it would conflict with the big metaspace change and/or JFR work that >> was about to get pushed. >> Unfortunately I then forgot all about it. >> >> I would like to push this fix to jdk9 (and then jdk8) now. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-7176220 >> >> Summary: >> The problem is that CMS sometimes misses datestamps for time trace >> printouts. >> The fix is to move the datestamp print logic into the class that >> handles time trace printouts, GCTraceTime. >> This makes it consistent with timestamps, which are already printed >> from there. >> >> Webrev for jdk9: >> http://cr.openjdk.java.net/~aeriksso/7176220/webrev.00/ >> >> And for reference: >> hsx24 review thread: >> http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2013-May/007127.html >> hsx24 changeset: >> http://hg.openjdk.java.net/hsx/hsx24/hotspot/rev/aa01d1f5fcd9 >> >> Thanks, >> Andreas > From staffan.friberg at oracle.com Fri Nov 14 13:14:42 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Fri, 14 Nov 2014 14:14:42 +0100 Subject: RFR: JDK-8064473 - Improved handling of age during object copy in G1 In-Reply-To: <5465F8A1.6070401@oracle.com> References: <54623886.4060602@oracle.com> <1415966952.4679.1.camel@oracle.com> <5465F8A1.6070401@oracle.com> Message-ID: <54660042.1090800@oracle.com> Thank you for the reviews. Will fix the comment. Would either of you be OK to sponsor and push the change? Regards, Staffan On 11/14/2014 01:42 PM, Bengt Rutisson wrote: > > Hi Staffan, > > On 2014-11-14 13:09, Thomas Schatzl wrote: >> Hi, >> >> On Tue, 2014-11-11 at 17:25 +0100, Staffan Friberg wrote: >>> Hi, >>> >>> Small update to object copying in G1 to reduce the number of times age >>> has to be read and also make sure the markOop can be trusted by reading >>> and saving it prior doing the forwarded check. This change is broken >>> out >>> from the PLAB event RFE. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8064473 >>> webrev: http://cr.openjdk.java.net/~sfriberg/JDK-8064473/webrev.01 >> Looks good except for a minor nit: the comment at line 203 >> >> // in this case, we have to install the mark word first, >> // otherwise obj looks to be forwarded (the old mark word, >> >> should start with a capital letter. >> >> I also created an RFE to fix up markOopDesc to allow nicer method >> names than "oop_decode_pointer", >> https://bugs.openjdk.java.net/browse/JDK-8064885 > > Looks good to me to modulo Thomas comment about the comment. > > Bengt > >> >> Thanks, >> Thomas >> >> > From erik.helin at oracle.com Fri Nov 14 13:48:55 2014 From: erik.helin at oracle.com (Erik Helin) Date: Fri, 14 Nov 2014 14:48:55 +0100 Subject: RFR: 8064721: The card tables only ever need two covering regions In-Reply-To: <51F872EF-EFDD-494F-8412-6BABA60CAAC5@oracle.com> References: <5463ADA1.9070804@oracle.com> <46B47E05-306E-4380-AC2F-A586AEE9BBC4@oracle.com> <5464D5DD.7010300@oracle.com> <51F872EF-EFDD-494F-8412-6BABA60CAAC5@oracle.com> Message-ID: <54660847.2060307@oracle.com> All, new webrevs available at: - full: http://cr.openjdk.java.net/~ehelin/8064721/webrev.01/ - incremental: http://cr.openjdk.java.net/~ehelin/8064721/webrev.00-01/ See comments inline. On 2014-11-13 20:33, Kim Barrett wrote: > On Nov 13, 2014, at 11:01 AM, Erik Helin wrote: >> >> On 2014-11-12 22:30, Kim Barrett wrote: >>> src/share/vm/memory/barrierSet.hpp >>> 52 // Some barrier sets create tables whose elements correspond to parts of >>> 53 // the heap; the CardTableModRefBS is an example. Such barrier sets will >>> 54 // normally reserve space for such tables, and commit parts of the table >>> 55 // "covering" parts of the heap that are committed. At most one covered >>> 56 // region per generation is needed. >>> 57 static const int _max_covered_regions = 2; >>> >>> Is this (now) constant slated for later elimination? >>> >> Sorry, I don't really follow, why would we remove this constant? Do >> you mean we should "hard code" 2 at all places where >> _max_covered_regions is used in CardTableModRefBS? Or that we should >> just move the constant down the inheritance chain to CardTableModRefBS >> where it really belongs? > > There's the location question, of course. But there is also the > question of whether this is any different from the maximum number of > generations, e.g. use that constant rather than a separate one, or > derived from that constant. (Can't depend on maximum number of > generations right now, of course, since the cleanup and simplification > of number of generations is still in progress.) Ah ok, now I see. Yes, once the generations cleanup has been pushed, we should use GenCollectedHeap::_max_gens. >>> src/share/vm/memory/cardTableRS.hpp >>> 86 // Number of generations (including permgen). >>> 87 static const int _regions_to_iterate = 3; >>> [...] >>> >> // The perm gen is index 0; other gens are at >> // their level plus 1. >> [...] >> Do you think I should rephrase this comment? >> [...] >> As I mentioned above, if someone gets rid of all the PermGen legacy in >> CardTableRS, we can probably use 2 as the value. > > I'd missed that perm gen was index 0 for this. So 3 it is, at least > for now. And yes, I think the comment needs more explanation; > something along the lines of > > Number of generations, plus one for lingering PermGen issues in > CardTableRS. > > Otherwise someone is likely to come along later, see "permgen", > think they can subtract one and update the comment, and forget to do > adequate testing... Agree, I've updated the patch. > And perhaps someday this too would be replaced by direct reference to > a maximum number of generations constant. > > And there needs to be a new bug report to clean up the lingering > permgen stuff in CardTableRS. Agree, I've filed JDK-8064876 - "Remove all remnants of PermGen in CardTableRS". >>> src/share/vm/memory/modRefBarrierSet.hpp >>> 40 ModRefBarrierSet() : BarrierSet() { _kind = BarrierSet::ModRef; } >>> >>> Not sure why this change was made; the default construction of >>> BarrierSet was already implicit. Not that I object to the change, >>> just surprised by it. > >> I didn't completely remember when in C++ a parent constructor was >> automatically called, so I decided to add a call to BarrierSet for the >> default constructor. If it is implicit then I can revert this part of >> the patch if you like (although I prefer the the explicit variant). > > If there's no explicit constructor call for a base class, a call to > its default constructor gets implicitly inserted at the appropriate > point in the member initializer list. > > I don't have a strong preference either way for explicit vs implicit > calls to base class default constructors. In this case I probably > would have left it implicit, but have no objection to explicit. But > if there needed to be explicit calls to other base class constructors > (due to multiple inheritance) then I'd probably make them all > explicit. > > But the initialization of _kind really should be in the member > initialization list, and since you are touching the code anyway? _kind is actually a member of BarrierSet, not ModRefBarrierSet, so it can't be set in member initialization list of a ModRefBarrierSet constructor. The proper way to clean this up would be to have the BarrierSet constructor take a parameter of type BarrierSet::Name to set _kind. In order to keep this patch focused on only cleaning up max_covered_regions, I reverted the patch to use the implicit call to the BarrierSet constructor in modRefBarrierSet.cpp. Thanks, Erik >>> src/share/vm/runtime/vmStructs.cpp >>> removal of >>> 476 nonstatic_field(BarrierSet, _max_covered_regions, int) \ >>> >>> [...] >>> >> The entries in vmStructs are not supported ? > > OK, so no issue here. > > From kim.barrett at oracle.com Fri Nov 14 20:25:52 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 14 Nov 2014 15:25:52 -0500 Subject: RFR (XS): -8064865: Remove the debug funciontality RotateCMSCollectionTypes for CMS In-Reply-To: <5465A9B7.2060104@oracle.com> References: <5465A9B7.2060104@oracle.com> Message-ID: On Nov 14, 2014, at 2:05 AM, Bengt Rutisson wrote: > > Can I have a couple of reviews for this small change to remove a debug flag from CMS? > > http://cr.openjdk.java.net/~brutisso/8064865/webrev.00/ > > https://bugs.openjdk.java.net/browse/JDK-8064865 Looks good to me. From kim.barrett at oracle.com Fri Nov 14 21:01:29 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 14 Nov 2014 16:01:29 -0500 Subject: RFR: 8064721: The card tables only ever need two covering regions In-Reply-To: <54660847.2060307@oracle.com> References: <5463ADA1.9070804@oracle.com> <46B47E05-306E-4380-AC2F-A586AEE9BBC4@oracle.com> <5464D5DD.7010300@oracle.com> <51F872EF-EFDD-494F-8412-6BABA60CAAC5@oracle.com> <54660847.2060307@oracle.com> Message-ID: <8440D736-5B08-47B6-A80F-C01A39805319@oracle.com> On Nov 14, 2014, at 8:48 AM, Erik Helin wrote: > > new webrevs available at: > - full: > http://cr.openjdk.java.net/~ehelin/8064721/webrev.01/ > - incremental: > http://cr.openjdk.java.net/~ehelin/8064721/webrev.00-01/ New webrev looks good. > See comments inline. > > On 2014-11-13 20:33, Kim Barrett wrote: >>> 57 static const int _max_covered_regions = 2; >> [?] But there is also the >> question of whether this is any different from the maximum number of >> generations, [?] > > Ah ok, now I see. Yes, once the generations cleanup has been pushed, we should use GenCollectedHeap::_max_gens. Do we need a bug for this? >>>> 87 static const int _regions_to_iterate = 3; >>>> [?] >> And yes, I think the comment needs more explanation; > > Agree, I've updated the patch. Thanks. >> And there needs to be a new bug report to clean up the lingering >> permgen stuff in CardTableRS. > > Agree, I've filed JDK-8064876 - "Remove all remnants of PermGen in CardTableRS?. Thanks. I added a followup comment with a pointer to _regions_to_iterate as another place to look. > >>>> src/share/vm/memory/modRefBarrierSet.hpp >>>> 40 ModRefBarrierSet() : BarrierSet() { _kind = BarrierSet::ModRef; } >> >> But the initialization of _kind really should be in the member >> initialization list, and since you are touching the code anyway? > > _kind is actually a member of BarrierSet, not ModRefBarrierSet, so it can't be set in member initialization list of a ModRefBarrierSet constructor. The proper way to clean this up would be to have the BarrierSet constructor take a parameter of type BarrierSet::Name to set _kind. In order to keep this patch focused on only cleaning up max_covered_regions, I reverted the patch to use the implicit call to the BarrierSet constructor in modRefBarrierSet.cpp. Sorry I missed that. Yes, the proper fix is to change the BarrierSet constructor. And get rid of the Uninit tag in the BarrierSet::Name enum. Assuming there is never a genuine need for a default constructed BarrierSet, which seems likely to me. (ctor and dtor should also be protected.) https://bugs.openjdk.java.net/browse/JDK-8064947 From jon.masamitsu at oracle.com Fri Nov 14 21:58:43 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Fri, 14 Nov 2014 13:58:43 -0800 Subject: RFR (XS): -8064865: Remove the debug funciontality RotateCMSCollectionTypes for CMS In-Reply-To: <5465AAA4.9060702@oracle.com> References: <5465A9B7.2060104@oracle.com> <5465AAA4.9060702@oracle.com> Message-ID: <54667B13.3020806@oracle.com> On 11/13/2014 11:09 PM, Bengt Rutisson wrote: > > On 2014-11-14 08:05, Bengt Rutisson wrote: >> >> Hi all, >> >> Can I have a couple of reviews for this small change to remove a >> debug flag from CMS? >> >> http://cr.openjdk.java.net/~brutisso/8064865/webrev.00/ > > I should have pointed out that this patch is based on top of the patch > for JDK-8064702 that is currently out for review. Change looks good. Reviewed. Jon > > Bengt > >> >> https://bugs.openjdk.java.net/browse/JDK-8064865 >> >> Background: >> >> During the review of "8064702: Remove the CMS foreground collector" >> it was discussed how to handle the RotateCMSCollectionTypes feature. >> It was suggested to remove this feature in a separate change. This is >> the change to remove that feature. >> >> From the bug description: >> >> The RotateCMSCollectionTypes flag is used in CMS to rotate between >> the three different kinds of collections of the old generation that >> CMS has; background collection, foreground collection and full >> compacting serial GC. >> >> The functionality is only available in non-product builds. >> >> With JDK-8064702 the foreground collector is being removed. This >> means that there are only two types left. The >> RotateCMSCollectionTypes flag is not being automatically tested and >> it is very rarely used. Rather than maintaining it we should remove >> this functionality. If we need this for debugging in the future it is >> a pretty simple patch to add back. But for the most part it should be >> possible to debug in other ways, by for example using >> ExplicitGCInvokesConcurrent flag and similar. >> >> Thanks, >> Bengt > From kim.barrett at oracle.com Fri Nov 14 22:40:17 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 14 Nov 2014 17:40:17 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <5465D17B.3030209@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> Message-ID: <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> On Nov 14, 2014, at 4:55 AM, Bengt Rutisson wrote: > > On 2014-11-13 23:13, Kim Barrett wrote: >> On Nov 12, 2014, at 11:01 PM, Jon Masamitsu wrote: >>> When any promotion failure occurs the concurrent CMS collections is >>> abandoned and a full collection is done instead. There is some >>> cleanup that is done to get the heap back into a state where the >>> full collection can be used. >> [?] >> The obvious first place to look is the caller(s) of par_promote(), >> e.g. ParNewGeneration::copy_to_survivor_space[_XXX_undo](). Promotion >> failure is handled there by setting an already existing flag and doing >> some additional work. It seems like it might work to just change the >> calls to par_promote() there to be conditional on that existing flag. >> This would make the checks of the proposed new flag outside the lock >> context redundant. If it is acceptable to have a few lingering wasted >> calls occur (which seems likely to me) then this conditionalization of >> calls to par_promote() would entirely eliminate the need for the >> proposed new flag. > > Do you mean something like this? > > diff --git a/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp b/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp > --- a/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp > +++ b/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp > @@ -1196,8 +1196,10 @@ > return real_forwardee(old); > } > > + if (!_promotion_failed) { > new_obj = _next_gen->par_promote(par_scan_state->thread_num(), > old, m, sz); > + } > > if (new_obj == NULL) { > // promotion failed, forward to self Yes, but remember there are two of these places, as there are two variants of copy_to_survivor_space_XXX_undo(). > I tried this with the reproducer that Jungwoo has. It gives the same good results as the originally suggested patch. > > [? snipped timing results ?] > So, I think it is fair to say that it is unnecessary to add the new state variable. Thanks for obtaining experimental support for the theory. > I also agree that the existing flag has the same race behavior as the new one that was proposed. While I agree with Kim that racing on a variable is always scary I think it is pretty safe in this case. We only race on setting it from false to true. And if anybody sees a stale false value they will just take a more expensive path in the code. Nothing will break. I'm more inclined to agree now that I've read a lot more code. A comment in the code would be good though. >> It seems like an even better approach would be to arrange to terminate >> the iteration over surviving objects early once promotion failure has >> occurred, rather than continuing to perform work that will ultimately >> be discarded. That looks to me like a much harder change to make >> though. > > Unfortunately it is not possible to stop iteration over all objects in the young gen. We still need to scan them all to update their references. Even objects we have not yet scanned may have references to objects that we moved before we got the promotion failure. So we can not immediately abandon scanning and go to to a full GC when we hit a promotion failure. First we need to make sure that all objects have their references updated. Yes, that's the kind of thing I was worried about, though I obviously hadn't thought it through very carefully. I think this does leave open the possibility of moving the _promotion_failed flag check earlier in copy_to_survivor_space_XXX(), to also bypass the tenuring query and the possible allocation in to_space, instead just always treating as a promotion failure once the _promotion_failure flag is true. The rationale would be that, once promotion failure has occurred, it might be better to leave all the remaining unprocessed objects in place as promotion failures than (attempt to) copy any of them to to_space. If nothing else, it might reduce the number of to_space pages being dirtied. I'm not sure it would be overall beneficial though, as it looks like treating more objects as promotion failures could result in a lot more mark preservation (which looks to be relatively expensive to do in some cases), and processing the preserved marks appears to be single-threaded at present. So while I think moving the _promotion_failed flag check earlier would work from a correctness standpoint, it's not yet clear to me even what the sign of the resulting performance change might be. From bengt.rutisson at oracle.com Mon Nov 17 09:38:42 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 17 Nov 2014 10:38:42 +0100 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <5465AA61.5040409@oracle.com> References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> <54638F87.30809@oracle.com> <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> <54645C75.2030707@oracle.com> <9F2FBF39-EDEF-4872-9665-8B107E2C4DA7@oracle.com> <80D56232-2E5B-40EB-95F6-D79CBCF3D9AF@oracle.com> <5465AA61.5040409@oracle.com> Message-ID: <5469C222.9050908@oracle.com> Hi all, Ramki reviewed the change and noticed that the removed method decide_foreground_collection_type() method had a side effect that I had missed. It would set the _incremental_collection_failed state that is used for the sizing heuristics. Thanks for looking at this, Ramki! I copied that code in to the place where decide_foreground_collection_type() used to be called. Here's an updated webrev: http://cr.openjdk.java.net/~brutisso/8064702/webrev.03/ Here's the diff compared to the last webrev: http://cr.openjdk.java.net/~brutisso/8064702/webrev.02-03.diff/ Thanks, Bengt On 2014-11-14 08:08, Bengt Rutisson wrote: > > Hi Kim, > > On 2014-11-13 22:28, Kim Barrett wrote: >> On Nov 13, 2014, at 11:11 AM, Kim Barrett >> wrote: >>> On Nov 13, 2014, at 2:23 AM, Bengt Rutisson >>> wrote: >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp >>>>> >>>>> >>>>> 1254 if (RotateCMSCollectionTypes && >>>>> 1255 (_cmsGen->debug_collection_type() != >>>>> 1256 ConcurrentMarkSweepGeneration::Concurrent_collection_type)) { >>>>> 1257 assert(_cmsGen->debug_collection_type() != >>>>> 1258 ConcurrentMarkSweepGeneration::Unknown_collection_type, >>>>> 1259 "Bad cms collection type"); >>>>> 1260 return false; >>>>> 1261 } >>>>> >>>>> => >>>>> >>>>> 1252 if (RotateCMSCollectionTypes) { >>>>> 1253 return _cmsGen->debug_concurrent_cycle(); >>>>> 1254 } >>>>> >>>>> I spent a large amount of time studying this change, because there >>>>> are >>>>> behavioral changes here that looked suspicious. I think I've decided >>>>> it's ok, but RotateCMSCollectionTypes is now badly named, and was >>>>> never well described and had (and still has) unclear semantics. I >>>>> think I've puzzled out more or less what it means, but that required >>>>> reading a lot of code, and I'm still not confident that I know what >>>>> would happen if I used it, or why I might want to. >>>> Right. I agree that it is unclear if this code is needed. I have >>>> never been running with RotateCMSCollectionTypes but I did not want >>>> to remove it as part of this change. >>>> >>>> [?] >>>> >>>> I do agree that the naming is bad. But if we decide to remove this >>>> functionality there is not much need to spend time on the naming >>>> discussion. >>> My inclination would be to remove it. [?] >> Just to be clear, I?m ok with deferring removal to a later task. >> Please file a bug report? > > Great, thanks! > > I filed a bug (JDK-8064865) and sent out a review request. > > Thanks again for looking at this and thanks for the review! > Bengt > > >> > From bengt.rutisson at oracle.com Mon Nov 17 09:42:12 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 17 Nov 2014 10:42:12 +0100 Subject: RFR (XS): -8064865: Remove the debug funciontality RotateCMSCollectionTypes for CMS In-Reply-To: References: <5465A9B7.2060104@oracle.com> Message-ID: <5469C2F4.8060904@oracle.com> Thanks for the reviews Jon and Kim! Bengt On 2014-11-14 21:25, Kim Barrett wrote: > On Nov 14, 2014, at 2:05 AM, Bengt Rutisson wrote: >> Can I have a couple of reviews for this small change to remove a debug flag from CMS? >> >> http://cr.openjdk.java.net/~brutisso/8064865/webrev.00/ >> >> https://bugs.openjdk.java.net/browse/JDK-8064865 > Looks good to me. > From bengt.rutisson at oracle.com Mon Nov 17 10:11:13 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 17 Nov 2014 11:11:13 +0100 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> Message-ID: <5469C9C1.7020306@oracle.com> On 2014-11-14 23:40, Kim Barrett wrote: > On Nov 14, 2014, at 4:55 AM, Bengt Rutisson wrote: >> On 2014-11-13 23:13, Kim Barrett wrote: >>> On Nov 12, 2014, at 11:01 PM, Jon Masamitsu wrote: >>>> When any promotion failure occurs the concurrent CMS collections is >>>> abandoned and a full collection is done instead. There is some >>>> cleanup that is done to get the heap back into a state where the >>>> full collection can be used. >>> [?] >>> The obvious first place to look is the caller(s) of par_promote(), >>> e.g. ParNewGeneration::copy_to_survivor_space[_XXX_undo](). Promotion >>> failure is handled there by setting an already existing flag and doing >>> some additional work. It seems like it might work to just change the >>> calls to par_promote() there to be conditional on that existing flag. >>> This would make the checks of the proposed new flag outside the lock >>> context redundant. If it is acceptable to have a few lingering wasted >>> calls occur (which seems likely to me) then this conditionalization of >>> calls to par_promote() would entirely eliminate the need for the >>> proposed new flag. >> Do you mean something like this? >> >> diff --git a/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp b/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp >> --- a/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp >> +++ b/src/share/vm/gc_implementation/parNew/parNewGeneration.cpp >> @@ -1196,8 +1196,10 @@ >> return real_forwardee(old); >> } >> >> + if (!_promotion_failed) { >> new_obj = _next_gen->par_promote(par_scan_state->thread_num(), >> old, m, sz); >> + } >> >> if (new_obj == NULL) { >> // promotion failed, forward to self > Yes, but remember there are two of these places, as there are two > variants of copy_to_survivor_space_XXX_undo(). Right. I only changed the one used for CMS since that's what we were running the reproducer with. If Jungwoo decides to go with this approach we should of course make it a complete patch. Possibly also adding a flag to control it as Jungwoo did in the original patch. > >> I tried this with the reproducer that Jungwoo has. It gives the same good results as the originally suggested patch. >> >> [? snipped timing results ?] >> So, I think it is fair to say that it is unnecessary to add the new state variable. > Thanks for obtaining experimental support for the theory. The reproducer from Jungwoo is really nice. It is easy to run and produce stable results. > >> I also agree that the existing flag has the same race behavior as the new one that was proposed. While I agree with Kim that racing on a variable is always scary I think it is pretty safe in this case. We only race on setting it from false to true. And if anybody sees a stale false value they will just take a more expensive path in the code. Nothing will break. > I'm more inclined to agree now that I've read a lot more code. A comment > in the code would be good though. Agreed. > >>> It seems like an even better approach would be to arrange to terminate >>> the iteration over surviving objects early once promotion failure has >>> occurred, rather than continuing to perform work that will ultimately >>> be discarded. That looks to me like a much harder change to make >>> though. >> Unfortunately it is not possible to stop iteration over all objects in the young gen. We still need to scan them all to update their references. Even objects we have not yet scanned may have references to objects that we moved before we got the promotion failure. So we can not immediately abandon scanning and go to to a full GC when we hit a promotion failure. First we need to make sure that all objects have their references updated. > Yes, that's the kind of thing I was worried about, though I obviously > hadn't thought it through very carefully. > > I think this does leave open the possibility of moving the > _promotion_failed flag check earlier in copy_to_survivor_space_XXX(), > to also bypass the tenuring query and the possible allocation in > to_space, instead just always treating as a promotion failure once the > _promotion_failure flag is true. > > The rationale would be that, once promotion failure has occurred, it > might be better to leave all the remaining unprocessed objects in > place as promotion failures than (attempt to) copy any of them to > to_space. If nothing else, it might reduce the number of to_space > pages being dirtied. > > I'm not sure it would be overall beneficial though, as it looks like > treating more objects as promotion failures could result in a lot more > mark preservation (which looks to be relatively expensive to do in > some cases), and processing the preserved marks appears to be > single-threaded at present. So while I think moving the > _promotion_failed flag check earlier would work from a correctness > standpoint, it's not yet clear to me even what the sign of the > resulting performance change might be. Yes, there is probably room for more improvement here. I think adding the check in the copy_to_survivor_space_XXX_undo() methods is a clear improvement in many cases. So, doing that as a first step is a good start. We can consider moving it earlier as a future enhancement. Bengt > From thomas.schatzl at oracle.com Mon Nov 17 12:13:51 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 17 Nov 2014 13:13:51 +0100 Subject: RFR: 8064721: The card tables only ever need two covering regions In-Reply-To: <8440D736-5B08-47B6-A80F-C01A39805319@oracle.com> References: <5463ADA1.9070804@oracle.com> <46B47E05-306E-4380-AC2F-A586AEE9BBC4@oracle.com> <5464D5DD.7010300@oracle.com> <51F872EF-EFDD-494F-8412-6BABA60CAAC5@oracle.com> <54660847.2060307@oracle.com> <8440D736-5B08-47B6-A80F-C01A39805319@oracle.com> Message-ID: <1416226431.3234.5.camel@oracle.com> Hi, On Fri, 2014-11-14 at 16:01 -0500, Kim Barrett wrote: > On Nov 14, 2014, at 8:48 AM, Erik Helin wrote: > > > > new webrevs available at: > > - full: > > http://cr.openjdk.java.net/~ehelin/8064721/webrev.01/ > > - incremental: > > http://cr.openjdk.java.net/~ehelin/8064721/webrev.00-01/ > > New webrev looks good. Looks good to me too. > > > See comments inline. > > > > On 2014-11-13 20:33, Kim Barrett wrote: > >>> 57 static const int _max_covered_regions = 2; > >> [?] But there is also the > >> question of whether this is any different from the maximum number of > >> generations, [?] > > > > Ah ok, now I see. Yes, once the generations cleanup has been pushed, we should use GenCollectedHeap::_max_gens. > > Do we need a bug for this? Please file a bug for that that depends on the generations cleanup. [...] > > > >>>> src/share/vm/memory/modRefBarrierSet.hpp > >>>> 40 ModRefBarrierSet() : BarrierSet() { _kind = BarrierSet::ModRef; } > >> > >> But the initialization of _kind really should be in the member > >> initialization list, and since you are touching the code anyway? > > > > _kind is actually a member of BarrierSet, not ModRefBarrierSet, so it can't be set in member initialization list of a ModRefBarrierSet constructor. The proper way to clean this up would be to have the BarrierSet constructor take a parameter of type BarrierSet::Name to set _kind. In order to keep this patch focused on only cleaning up max_covered_regions, I reverted the patch to use the implicit call to the BarrierSet constructor in modRefBarrierSet.cpp. > > Sorry I missed that. Yes, the proper fix is to change the BarrierSet constructor. And get rid of the Uninit tag in the BarrierSet::Name enum. Assuming there is never a genuine need for a default constructed BarrierSet, which seems likely to me. (ctor and dtor should also be protected.) > https://bugs.openjdk.java.net/browse/JDK-8064947 > Thanks, Thomas From erik.helin at oracle.com Mon Nov 17 13:44:33 2014 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Nov 2014 14:44:33 +0100 Subject: RFR: 8064721: The card tables only ever need two covering regions In-Reply-To: <1416226431.3234.5.camel@oracle.com> References: <5463ADA1.9070804@oracle.com> <46B47E05-306E-4380-AC2F-A586AEE9BBC4@oracle.com> <5464D5DD.7010300@oracle.com> <51F872EF-EFDD-494F-8412-6BABA60CAAC5@oracle.com> <54660847.2060307@oracle.com> <8440D736-5B08-47B6-A80F-C01A39805319@oracle.com> <1416226431.3234.5.camel@oracle.com> Message-ID: <20141117134433.GA21743@ehelin-desktop> On 2014-11-17, Thomas Schatzl wrote: > Hi, > > On Fri, 2014-11-14 at 16:01 -0500, Kim Barrett wrote: > > On Nov 14, 2014, at 8:48 AM, Erik Helin wrote: > > > > > > new webrevs available at: > > > - full: > > > http://cr.openjdk.java.net/~ehelin/8064721/webrev.01/ > > > - incremental: > > > http://cr.openjdk.java.net/~ehelin/8064721/webrev.00-01/ > > > > New webrev looks good. > > Looks good to me too. Thanks! > > > > > See comments inline. > > > > > > On 2014-11-13 20:33, Kim Barrett wrote: > > >>> 57 static const int _max_covered_regions = 2; > > >> [?] But there is also the > > >> question of whether this is any different from the maximum number of > > >> generations, [?] > > > > > > Ah ok, now I see. Yes, once the generations cleanup has been pushed, we should use GenCollectedHeap::_max_gens. > > > > Do we need a bug for this? > > Please file a bug for that that depends on the generations cleanup. I filed https://bugs.openjdk.java.net/browse/JDK-8065117. Thanks, Erik > [...] > > > > > >>>> src/share/vm/memory/modRefBarrierSet.hpp > > >>>> 40 ModRefBarrierSet() : BarrierSet() { _kind = BarrierSet::ModRef; } > > >> > > >> But the initialization of _kind really should be in the member > > >> initialization list, and since you are touching the code anyway? > > > > > > _kind is actually a member of BarrierSet, not ModRefBarrierSet, so it can't be set in member initialization list of a ModRefBarrierSet constructor. The proper way to clean this up would be to have the BarrierSet constructor take a parameter of type BarrierSet::Name to set _kind. In order to keep this patch focused on only cleaning up max_covered_regions, I reverted the patch to use the implicit call to the BarrierSet constructor in modRefBarrierSet.cpp. > > > > Sorry I missed that. Yes, the proper fix is to change the BarrierSet constructor. And get rid of the Uninit tag in the BarrierSet::Name enum. Assuming there is never a genuine need for a default constructed BarrierSet, which seems likely to me. (ctor and dtor should also be protected.) > > https://bugs.openjdk.java.net/browse/JDK-8064947 > > > > Thanks, > Thomas > > From kim.barrett at oracle.com Mon Nov 17 15:57:22 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 17 Nov 2014 10:57:22 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <5469C9C1.7020306@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> <5469C9C1.7020306@oracle.com> Message-ID: <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> On Nov 17, 2014, at 5:11 AM, Bengt Rutisson wrote: > >> I think this does leave open the possibility of moving the >> _promotion_failed flag check earlier in copy_to_survivor_space_XXX(), >> [?] >> >> I'm not sure it would be overall beneficial though, [?] > > Yes, there is probably room for more improvement here. I think adding the check in the copy_to_survivor_space_XXX_undo() methods is a clear improvement in many cases. So, doing that as a first step is a good start. We can consider moving it earlier as a future enhancement. Agreed. From erik.helin at oracle.com Mon Nov 17 17:43:22 2014 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 17 Nov 2014 18:43:22 +0100 Subject: RFR: 8059066: CardTableModRefBS might commit the same page twice In-Reply-To: References: <54450C6D.3080603@oracle.com> <986B14CB-1598-47D4-A581-4F82115A8257@oracle.com> <544A4C1D.9000908@oracle.com> Message-ID: <20141117174322.GA23553@ehelin-desktop> On 2014-10-27, Kim Barrett wrote: > On Oct 24, 2014, at 8:54 AM, Erik Helin wrote: > > > > Fortunately for us, we only have have two committed regions (anyone, please correct me if I'm wrong here) and two covering regions, since we only have two generations. Unfortunately, CardTableModRefBS was probably used for the train collector and therefore (unnecessary) tries to support n generations. > > > > In the case of at most two committed regions and no subsets, I believe that my proposed fix will work (I should have mentioned all this is in the original RFR). What do you think? Should we add asserts that checks that the number of committed regions are at most two? > > ... > > I have no idea what that's about yet. But it looks like, for non-G1 > generational collectors the value will actually be 4 rather than 2. I > haven't actually verified that with executing code though. > The code only ever used two covering (and therefore only two committed) regions, see the discussion in the review for JDK-8064721 for more details [0]. The question is now, how many pages can these two committed regions share? By reading CardTableModRefBS::find_covering_region_by_base one can see that when adding a new committed region, the start for the region gets aligned down: 211 jbyte* ct_start = byte_for(base); 212 uintptr_t ct_start_aligned = align_size_down((uintptr_t)ct_start, _page_size); 213 _committed[res].set_start((HeapWord*)ct_start_aligned); Then CardTableModRefBS::resize_covered_region always aligns up the end of a region the nearest page: 271 jbyte* const new_end = byte_after(new_region.last()); 272 HeapWord* new_end_aligned = 273 (HeapWord*) align_size_up((uintptr_t)new_end, _page_size); Because the start is aligned down and the end is aligned up, the two committed regions will always share at least one page (since the start for _committed[1] will be aligned down and the end for _committed[0] will be aligned up). The first committed region can never expand beyond the original, unaligned start of the second region, because a heap generation cannot expand into the next heap generation. In other words, the committed region at index 0 can never expand beyond the page it shares with the committed region at index 1. This means that the committed regions (when fully expanded) always will share exactly one page. The patch in the webrev [1] therefore handles all cases, since it will correctly determine when _committed[0] extends into the page that is shared with _committed[1]. The existing code will then adjust the end of _committed[0] to the start of _committed[1] in order to not commit the same page twice. I uploaded a new patch that adds a test (I did not change the fix). See new webrev at: http://cr.openjdk.java.net/~ehelin/8059066/webrev.01/ The code in CardTableModRefsBS is very generic, it does not utilize the fact that we only have two generations, two committed regions and only one shared page. This makes the code unnecessarily hard to follow. I filed a new issue to remove these, no longer necessary, generalizations [2]. [0]: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014-November/011279.html [1]: http://cr.openjdk.java.net/~ehelin/8059066/webrev.00/ [2]: https://bugs.openjdk.java.net/browse/JDK-8065137 From kim.barrett at oracle.com Mon Nov 17 19:04:17 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 17 Nov 2014 14:04:17 -0500 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: <5469C222.9050908@oracle.com> References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> <54638F87.30809@oracle.com> <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> <54645C75.2030707@oracle.com> <9F2FBF39-EDEF-4872-9665-8B107E2C4DA7@oracle.com> <80D56232-2E5B-40EB-95F6-D79CBCF3D9AF@oracle.com> <5465AA61.5040409@oracle.com> <5469C222.9050908@oracle.com> Message-ID: On Nov 17, 2014, at 4:38 AM, Bengt Rutisson wrote: > > Ramki reviewed the change and noticed that the removed method decide_foreground_collection_type() method had a side effect that I had missed. It would set the _incremental_collection_failed state that is used for the sizing heuristics. Thanks for looking at this, Ramki! Good catch! > Here's an updated webrev: > http://cr.openjdk.java.net/~brutisso/8064702/webrev.03/ > > Here's the diff compared to the last webrev: > http://cr.openjdk.java.net/~brutisso/8064702/webrev.02-03.diff/ I looked it over again. Looks good to me. From bengt.rutisson at oracle.com Mon Nov 17 20:04:07 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 17 Nov 2014 21:04:07 +0100 Subject: RFR (L): 8064702: Remove the CMS foreground collector In-Reply-To: References: <54635A8B.3050402@oracle.com> <54636411.2000006@oracle.com> <54638F87.30809@oracle.com> <9341965D-50BA-4404-91DA-D52FDD13C1EA@oracle.com> <54645C75.2030707@oracle.com> <9F2FBF39-EDEF-4872-9665-8B107E2C4DA7@oracle.com> <80D56232-2E5B-40EB-95F6-D79CBCF3D9AF@oracle.com> <5465AA61.5040409@oracle.com> <5469C222.9050908@oracle.com> Message-ID: <546A54B7.2010508@oracle.com> On 11/17/14 8:04 PM, Kim Barrett wrote: > On Nov 17, 2014, at 4:38 AM, Bengt Rutisson wrote: >> Ramki reviewed the change and noticed that the removed method decide_foreground_collection_type() method had a side effect that I had missed. It would set the _incremental_collection_failed state that is used for the sizing heuristics. Thanks for looking at this, Ramki! > Good catch! > >> Here's an updated webrev: >> http://cr.openjdk.java.net/~brutisso/8064702/webrev.03/ >> >> Here's the diff compared to the last webrev: >> http://cr.openjdk.java.net/~brutisso/8064702/webrev.02-03.diff/ > I looked it over again. Looks good to me. Thanks for looking at this again, Kim! Bengt > > From derek.white at oracle.com Tue Nov 18 16:40:35 2014 From: derek.white at oracle.com (Derek White) Date: Tue, 18 Nov 2014 11:40:35 -0500 Subject: RFR (smallish?) 8061611: Remove deprecated command line flags Message-ID: <546B7683.9000803@oracle.com> Hi Team, First review request. Please let me know if I've forgotten something or have gone completely off the rails. The main point of this bug is to remove deprecated -XX options which are alias for other options. The only complicated part is that one case, /CMSParPromoteBlocksToClaim/ was not a true alias for /OldPLABSize/ but a parallel option with different defaults that were synchronized in ergo processing. This fix removes the /CMSParPromoteBlocksToClaim /variable but preserves using different defaults in the CMS case. Also in this fix I added warning messages to several remaining undocumented command line options aliases. This should ease removal of these options in the future CMSMarkStackSize ==> MarkStackSize (MarkStackSize not documented either, but came in jdk6) G1MarkStackSize ==> MarkStackSize CMSMarkStackSizeMax ==> MarkStackSizeMax (MarkStackSizeMax not documented either) ParallelMarkingThreads => ConcGCThreads (ConcGCThreads came in jdk6) ParallelCMSThreads ==> ConcGCThread Thanks, - Derek *Webrev*: http://cr.openjdk.java.net/~drwhite/8061611/webrev.00/ *Bug*: https://bugs.openjdk.java.net/browse/JDK-8061611 *Testing*: jprt: Saw 1-2 intermittent failures that went away on retesting - hangs and timeouts. refworkload: no differences jtreg: Saw a few unexplained results. Not sure if typical or not: Execution failed: `main' threw exception: java.lang.Exception: jmap -heap exited with error code: 1 * gc/metaspace/CompressedClassSpaceSizeInJmapHeap.java : Checks that jmap -heap contains the flag CompressedClassSpaceSize Program `/export/users/drwhite/test-builds/j2sdk-image.11.17.2014/bin/java' interrupted! (timed out?) * closed/runtime/AppCDS/SharedArchiveConsistency.java : SharedArchiveConsistency Plus a bunch of tests that are labelled "ignored". -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikael.gerdin at oracle.com Tue Nov 18 18:51:54 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 18 Nov 2014 19:51:54 +0100 Subject: Fwd: Re: Review request for 8059492: Wrong spelling in assert: "Not initialied properly?" In-Reply-To: <546B94A9.5020107@oracle.com> References: <546B94A9.5020107@oracle.com> Message-ID: <546B954A.7080906@oracle.com> I just realized that this was sent to jdk9-dev instead of hotspot-gc-dev. Since this is a small change I think it's sufficient to send it to hotspot-gc-dev for the record. /Mikael -------- Forwarded Message -------- Subject: Re: Review request for 8059492: Wrong spelling in assert: "Not initialied properly?" Date: Tue, 18 Nov 2014 19:49:13 +0100 From: Mikael Gerdin Organization: Oracle Corporation To: Alexander Harlap , jdk9-dev at openjdk.java.net Hi Alex, On 2014-11-18 17:29, Alexander Harlap wrote: > Please help to code review the change in JDK9 (trivial) for the > following CR: > 8059492: Wrong spelling in assert: "Not initialied properly?" > https://bugs.openjdk.java.net/browse/JDK-8059492 > > The webrev is located at: > http://cr.openjdk.java.net/~aharlap/8059492/webrev.00/ > The change looks good to me, Reviewed. /Mikael > > Testing completed: JPRT > > Alex From kumarsharma.nitin at gmail.com Tue Nov 18 17:24:23 2014 From: kumarsharma.nitin at gmail.com (nitin sharma) Date: Tue, 18 Nov 2014 12:24:23 -0500 Subject: Major GC occurring before CMS occupancy fraction Message-ID: hi, I am facing the same issue.. We are using JDK 1.6. 0_23 and have set Occupancy Fraction to 80% but i can see CMS triggered at much below ration (around 53%).. is there a solution for same? I have used "-XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly" option... complete JVM argument set ==> /usr/java/jdk1.6.0_23/bin/java -server -Xms16384m -Xmx16384m -XX:PermSize=512m -XX:MaxPermSize=1024m -XX:CompileThreshold=8000 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseParNewGC -XX:MaxNewSize=6144m -XX:NewSize=6144m -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly -DUseSunHttpHandler=true -XX:+HeapDumpOnOutOfMemoryError Regards, Nitin Kumar Sharma. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ecki at zusammenkunft.net Wed Nov 19 01:13:37 2014 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Wed, 19 Nov 2014 02:13:37 +0100 Subject: Major GC occurring before CMS occupancy fraction In-Reply-To: References: Message-ID: <20141119021337.0000667b.ecki@zusammenkunft.net> Hello, you need to provide verbose GC logs so we can say anything on that. But be aware that you might not get much help on ancient Java :) Can you try to set the initial perm size to the maximum size, to make sure resizes in this area do not trigger GC. And are you sure you are not confronted with RMI DGC runs? Gruss Bernd Am Tue, 18 Nov 2014 12:24:23 -0500 schrieb nitin sharma : > hi, > > I am facing the same issue.. We are using JDK 1.6. 0_23 and have set > Occupancy Fraction to 80% but i can see CMS triggered at much below > ration (around 53%).. > > is there a solution for same? > > I have used "-XX:CMSInitiatingOccupancyFraction=80 > -XX:+UseCMSInitiatingOccupancyOnly" option... > > complete JVM argument set ==> > > /usr/java/jdk1.6.0_23/bin/java -server -Xms16384m -Xmx16384m > -XX:PermSize=512m -XX:MaxPermSize=1024m -XX:CompileThreshold=8000 > -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps > -XX:+PrintGCTimeStamps -XX:+UseParNewGC -XX:MaxNewSize=6144m > -XX:NewSize=6144m -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=4 > -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled > -XX:CMSInitiatingOccupancyFraction=80 > -XX:+UseCMSInitiatingOccupancyOnly -DUseSunHttpHandler=true > -XX:+HeapDumpOnOutOfMemoryError > > > > Regards, > Nitin Kumar Sharma. > From david.holmes at oracle.com Wed Nov 19 01:47:45 2014 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Nov 2014 11:47:45 +1000 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <54638FA5.5000505@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <54637AF5.9080600@oracle.com> <54638012.3010202@oracle.com> <546385BB.3080801@oracle.com> <54638FA5.5000505@oracle.com> Message-ID: <546BF6C1.2090607@oracle.com> In case you are still waiting this all looks fine to me. Thanks, David On 13/11/2014 2:49 AM, Evgeniya Stepanova wrote: > Forgotten copyrights were changed > http://cr.openjdk.java.net/~eistepan/8062536/webrev.04/ > > On 12.11.2014 20:07, Evgeniya Stepanova wrote: >> Hi Bengt, >> >> Please see comments inline >> On 12.11.2014 19:43, Bengt Rutisson wrote: >>> >>> On 2014-11-12 16:21, Evgeniya Stepanova wrote: >>>> Hi everyone! >>>> >>>> Since the decision was made to change only tests which fail because >>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>> jdk part of the JDK-8019361 >>>> : >>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ >>> >>> Thanks for updating the webrev! >>> >>> A couple of comments: >>> >>> MemoryTestAllGC.sh >>> >>> The test is run three times, once with no params, once with >>> ParallelGC and once with CMS. So, I think the @requires should just >>> be vm.gc == "null". Similarly to what was done for PendingAllGC.sh. >>> >> The third run (with CMS) is commented. Without this run UseParallelGC >> is valid option >> #runOne -XX:+UseConcMarkSweepGC MemoryTest 3 >> (http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/test/java/lang/management/MemoryMXBean/MemoryTestAllGC.sh.frames.html) >>> >>> TestInputArgument.sh >>> >>> The changes here seem unrelated to @requires. >>> >> This test was changed after conversation with David Holmes (see >> thread below) >>> >>> EnqueuePollRace.java >>> >>> Can you explain why it is safe to remove -XX:+UseSerialGC for this test? >>> >>> >> This test was modified after conversation with Filipp Zhinkin and >> Mandy Chung (https://bugs.openjdk.java.net/browse/JDK-8051723) >>> JpsHelper.java >>> >>> Can you explain why it is safe to remvoe -XX:+UseParallelGC for this >>> test? >>> >> This test was changed after conversation with Katja Kantserova (see >> thread below), GC flag is just an arbitrary chosen test flag >>> >>> When I use Aurora to check what tests that currently are considered >>> known because of JDK-8019361 I get a pretty long list: >>> >>> http://aurora.ru.oracle.com/functional/faces/CRRules.xhtml?cr=JDK-8019361 >>> >>> Are the tests in webrev.03 the only tests that still fail? Have the >>> others been fixed in other ways? >> There would be 2 more changes in reviews in closed part :) >> >> Thanks, >> Evgeniya Stepanova >>> >>> Thanks, >>> Bengt >>> >>> >>> >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 07.11.2014 15:34, Evgeniya Stepanova wrote: >>>>> David, Filipp, Katja >>>>> >>>>> Diff have been updated one more time: >>>>> java/lang/management/RuntimeMXBean/TestInputArgument.sh and >>>>> test/java/lang/ref/EnqueuePollRace.java have been changed >>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >>>>> >>>>> Thanks >>>>> >>>>> On 07.11.2014 9:37, David Holmes wrote: >>>>>> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>>>>>> New webrev: >>>>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >>>>>> >>>>>> In: >>>>>> >>>>>> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >>>>>> >>>>>> the use of the gc options seems incidental - it's just picking two >>>>>> innocuous options to use - similar to the JpsHelper case. You >>>>>> could replace +UseParallelGC with something like >>>>>> +UseFastJNIAccessors (platform independent and normally true). >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>>>>>> Thanks a lot! >>>>>>>> >>>>>>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>>>>>> Hi Katja, >>>>>>>>> >>>>>>>>> Ok, this seems to be a perfect solution. Thank you. I'll change >>>>>>>>> the >>>>>>>>> diff accordingly. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>>>>>> Hi Dima, >>>>>>>>>> >>>>>>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>>>>>> Hi Katja, >>>>>>>>>>> >>>>>>>>>>> You are right, there will be no conflict, because test >>>>>>>>>>> ignores any >>>>>>>>>>> external VM flags. >>>>>>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>>>>>> >>>>>>>>>>> Ignoring external options is bad thing, such "selfish" tests are >>>>>>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>>>>>> >>>>>>>>>> This particular case is to test the defined flags are shown up as >>>>>>>>>> expected. >>>>>>>>>> >>>>>>>>>> Evgeniya, >>>>>>>>>> >>>>>>>>>> would you mind to change JpsHelper.java instead? >>>>>>>>>> >>>>>>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>>>>>> @@ -93,7 +93,7 @@ >>>>>>>>>> /** >>>>>>>>>> * VM arguments to start test application with >>>>>>>>>> */ >>>>>>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>> "-XX:+PrintGCDetails"}; >>>>>>>>>> /** >>>>>>>>>> * VM flag to start test application with >>>>>>>>>> */ >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Katja >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> @requires will allow to modify tests to include external vm >>>>>>>>>>> options without any risk of bumping into conflict and extend >>>>>>>>>>> area >>>>>>>>>>> of test applicability. >>>>>>>>>>> >>>>>>>>>>> But if you still believe, that @requires is not necessary - it's >>>>>>>>>>> not a problem, tests could be kept as is. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>> >>>>>>>>>>>> As David has pointed out these jps tests are not testing gc. >>>>>>>>>>>> The >>>>>>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. There >>>>>>>>>>>> should not be any conflicts either since these tests are >>>>>>>>>>>> running >>>>>>>>>>>> in driver mode: >>>>>>>>>>>> >>>>>>>>>>>> ... >>>>>>>>>>>> * @run driver TestJpsJar >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>>> which means no flags from above are accepted. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Katja >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>> Hi David, >>>>>>>>>>>>> >>>>>>>>>>>>> tag added because tests contain string >>>>>>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>>>>>> >>>>>>>>>>>>> and JpsHelper defines >>>>>>>>>>>>> ... >>>>>>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>>>>> ... >>>>>>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>>>>>> if (testVmArgs == null) { >>>>>>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>>>>>> } >>>>>>>>>>>>> return testVmArgs; >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> Tests itself wouldn't fail if we use another GC, tag added for >>>>>>>>>>>>> cleanup-if we use for example SerialGC we must be sure that >>>>>>>>>>>>> tests >>>>>>>>>>>>> passed with this GC, not with another one. Now you will assume >>>>>>>>>>>>> that concrete test passed with Serial GC, but it run only with >>>>>>>>>>>>> Parallel GC. Plus there is no any sense to run test twice >>>>>>>>>>>>> in TC >>>>>>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part >>>>>>>>>>>>>>> of the >>>>>>>>>>>>>>> JDK-8019361 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>> another GC >>>>>>>>>>>>>>> is set >>>>>>>>>>>>>>> outside >>>>>>>>>>>>>> >>>>>>>>>>>>>> I don't see why you have done this for the >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>>>>>> >>>>>>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>>>>>> skip test >>>>>>>>>>>>>>> if there is a conflict >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just wondering: Does a skipped test get a .jtr file >>>>>>>>>>>>>> showing it >>>>>>>>>>>>>> was skipped; or does it only appear in the higher-level >>>>>>>>>>>>>> jtreg log? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> David >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and >>>>>>>>>>>>>>> without >>>>>>>>>>>>>>> any GC flag). Tests are being excluded as expected. No tests >>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> // >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /Evgeniya Stepanova/ >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ > > -- > /Evgeniya Stepanova/ From evgeniya.stepanova at oracle.com Wed Nov 19 08:45:32 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 19 Nov 2014 12:45:32 +0400 Subject: Fwd: Re: RFR: 8062536: [TESTBUG] Conflicting GC combinations in jdk tests In-Reply-To: <546BF6C1.2090607@oracle.com> References: <545B4D0C.2010005@oracle.com> <545B51A1.7080905@oracle.com> <545B4BF6.4040209@oracle.com> <545B7006.5080104@oracle.com> <545B722C.3020406@oracle.com> <545B793D.5000008@oracle.com> <545B8777.6080403@oracle.com> <545C5AB6.6030103@oracle.com> <545CAE30.9010708@oracle.com> <54637AF5.9080600@oracle.com> <54638012.3010202@oracle.com> <546385BB.3080801@oracle.com> <54638FA5.5000505@oracle.com> <546BF6C1.2090607@oracle.com> Message-ID: <546C58AC.3070401@oracle.com> Hi David! Thank you very much for the review! On 19.11.2014 5:47, David Holmes wrote: > In case you are still waiting this all looks fine to me. > > Thanks, > David > > On 13/11/2014 2:49 AM, Evgeniya Stepanova wrote: >> Forgotten copyrights were changed >> http://cr.openjdk.java.net/~eistepan/8062536/webrev.04/ >> >> On 12.11.2014 20:07, Evgeniya Stepanova wrote: >>> Hi Bengt, >>> >>> Please see comments inline >>> On 12.11.2014 19:43, Bengt Rutisson wrote: >>>> >>>> On 2014-11-12 16:21, Evgeniya Stepanova wrote: >>>>> Hi everyone! >>>>> >>>>> Since the decision was made to change only tests which fail because >>>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>>> jdk part of the JDK-8019361 >>>>> : >>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/ >>>> >>>> Thanks for updating the webrev! >>>> >>>> A couple of comments: >>>> >>>> MemoryTestAllGC.sh >>>> >>>> The test is run three times, once with no params, once with >>>> ParallelGC and once with CMS. So, I think the @requires should just >>>> be vm.gc == "null". Similarly to what was done for PendingAllGC.sh. >>>> >>> The third run (with CMS) is commented. Without this run UseParallelGC >>> is valid option >>> #runOne -XX:+UseConcMarkSweepGC MemoryTest 3 >>> (http://cr.openjdk.java.net/~avstepan/eistepan/8062536/webrev.03/test/java/lang/management/MemoryMXBean/MemoryTestAllGC.sh.frames.html) >>> >>>> >>>> TestInputArgument.sh >>>> >>>> The changes here seem unrelated to @requires. >>>> >>> This test was changed after conversation with David Holmes (see >>> thread below) >>>> >>>> EnqueuePollRace.java >>>> >>>> Can you explain why it is safe to remove -XX:+UseSerialGC for this >>>> test? >>>> >>>> >>> This test was modified after conversation with Filipp Zhinkin and >>> Mandy Chung (https://bugs.openjdk.java.net/browse/JDK-8051723) >>>> JpsHelper.java >>>> >>>> Can you explain why it is safe to remvoe -XX:+UseParallelGC for this >>>> test? >>>> >>> This test was changed after conversation with Katja Kantserova (see >>> thread below), GC flag is just an arbitrary chosen test flag >>>> >>>> When I use Aurora to check what tests that currently are considered >>>> known because of JDK-8019361 I get a pretty long list: >>>> >>>> http://aurora.ru.oracle.com/functional/faces/CRRules.xhtml?cr=JDK-8019361 >>>> >>>> >>>> Are the tests in webrev.03 the only tests that still fail? Have the >>>> others been fixed in other ways? >>> There would be 2 more changes in reviews in closed part :) >>> >>> Thanks, >>> Evgeniya Stepanova >>>> >>>> Thanks, >>>> Bengt >>>> >>>> >>>> >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> On 07.11.2014 15:34, Evgeniya Stepanova wrote: >>>>>> David, Filipp, Katja >>>>>> >>>>>> Diff have been updated one more time: >>>>>> java/lang/management/RuntimeMXBean/TestInputArgument.sh and >>>>>> test/java/lang/ref/EnqueuePollRace.java have been changed >>>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.02/ >>>>>> >>>>>> Thanks >>>>>> >>>>>> On 07.11.2014 9:37, David Holmes wrote: >>>>>>> On 7/11/2014 12:36 AM, Evgeniya Stepanova wrote: >>>>>>>> New webrev: >>>>>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.01/ >>>>>>> >>>>>>> In: >>>>>>> >>>>>>> test/java/lang/management/RuntimeMXBean/TestInputArgument.sh >>>>>>> >>>>>>> the use of the gc options seems incidental - it's just picking two >>>>>>> innocuous options to use - similar to the JpsHelper case. You >>>>>>> could replace +UseParallelGC with something like >>>>>>> +UseFastJNIAccessors (platform independent and normally true). >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> On 06.11.2014 17:35, Yekaterina Kantserova wrote: >>>>>>>>> Thanks a lot! >>>>>>>>> >>>>>>>>> On 11/06/2014 02:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>> Hi Katja, >>>>>>>>>> >>>>>>>>>> Ok, this seems to be a perfect solution. Thank you. I'll change >>>>>>>>>> the >>>>>>>>>> diff accordingly. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> On 06.11.2014 16:56, Yekaterina Kantserova wrote: >>>>>>>>>>> Hi Dima, >>>>>>>>>>> >>>>>>>>>>> On 11/06/2014 11:22 AM, Dmitry Fazunenko wrote: >>>>>>>>>>>> Hi Katja, >>>>>>>>>>>> >>>>>>>>>>>> You are right, there will be no conflict, because test >>>>>>>>>>>> ignores any >>>>>>>>>>>> external VM flags. >>>>>>>>>>>> So, adding @requires seems unnecessary here, but... >>>>>>>>>>>> >>>>>>>>>>>> Ignoring external options is bad thing, such "selfish" >>>>>>>>>>>> tests are >>>>>>>>>>>> not applicable for other areas, like GC, compiler, RT. >>>>>>>>>>> >>>>>>>>>>> This particular case is to test the defined flags are shown >>>>>>>>>>> up as >>>>>>>>>>> expected. >>>>>>>>>>> >>>>>>>>>>> Evgeniya, >>>>>>>>>>> >>>>>>>>>>> would you mind to change JpsHelper.java instead? >>>>>>>>>>> >>>>>>>>>>> +++ b/test/sun/tools/jps/JpsHelper.java >>>>>>>>>>> @@ -93,7 +93,7 @@ >>>>>>>>>>> /** >>>>>>>>>>> * VM arguments to start test application with >>>>>>>>>>> */ >>>>>>>>>>> - public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>>> + public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>> "-XX:+PrintGCDetails"}; >>>>>>>>>>> /** >>>>>>>>>>> * VM flag to start test application with >>>>>>>>>>> */ >>>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Katja >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> @requires will allow to modify tests to include external vm >>>>>>>>>>>> options without any risk of bumping into conflict and extend >>>>>>>>>>>> area >>>>>>>>>>>> of test applicability. >>>>>>>>>>>> >>>>>>>>>>>> But if you still believe, that @requires is not necessary - >>>>>>>>>>>> it's >>>>>>>>>>>> not a problem, tests could be kept as is. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Dima >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 06.11.2014 16:27, Yekaterina Kantserova wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>> >>>>>>>>>>>>> As David has pointed out these jps tests are not testing gc. >>>>>>>>>>>>> The >>>>>>>>>>>>> -XX:+UseParallelGC is just an arbitrary chosen test flag. >>>>>>>>>>>>> There >>>>>>>>>>>>> should not be any conflicts either since these tests are >>>>>>>>>>>>> running >>>>>>>>>>>>> in driver mode: >>>>>>>>>>>>> >>>>>>>>>>>>> ... >>>>>>>>>>>>> * @run driver TestJpsJar >>>>>>>>>>>>> ... >>>>>>>>>>>>> >>>>>>>>>>>>> which means no flags from above are accepted. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Katja >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/06/2014 11:05 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>> >>>>>>>>>>>>>> tag added because tests contain string >>>>>>>>>>>>>> cmd.addAll(JpsHelper.getVmArgs()); >>>>>>>>>>>>>> >>>>>>>>>>>>>> and JpsHelper defines >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> public static final String[] VM_ARGS = {"-Xmx512m", >>>>>>>>>>>>>> "-XX:+UseParallelGC"}; >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> public static List getVmArgs() throws IOException { >>>>>>>>>>>>>> if (testVmArgs == null) { >>>>>>>>>>>>>> testVmArgs = new ArrayList<>(); >>>>>>>>>>>>>> testVmArgs.addAll(Arrays.asList(VM_ARGS)); >>>>>>>>>>>>>> testVmArgs.add("-XX:Flags=" + >>>>>>>>>>>>>> getVmFlagsFile().getAbsolutePath()); >>>>>>>>>>>>>> } >>>>>>>>>>>>>> return testVmArgs; >>>>>>>>>>>>>> } >>>>>>>>>>>>>> >>>>>>>>>>>>>> Tests itself wouldn't fail if we use another GC, tag >>>>>>>>>>>>>> added for >>>>>>>>>>>>>> cleanup-if we use for example SerialGC we must be sure that >>>>>>>>>>>>>> tests >>>>>>>>>>>>>> passed with this GC, not with another one. Now you will >>>>>>>>>>>>>> assume >>>>>>>>>>>>>> that concrete test passed with Serial GC, but it run only >>>>>>>>>>>>>> with >>>>>>>>>>>>>> Parallel GC. Plus there is no any sense to run test twice >>>>>>>>>>>>>> in TC >>>>>>>>>>>>>> (with different GC, since it use only Parallel) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>> On 06.11.2014 6:20, David Holmes wrote: >>>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 6/11/2014 1:33 AM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please review changes for 8062536, the OpenJDK/jdk part >>>>>>>>>>>>>>>> of the >>>>>>>>>>>>>>>> JDK-8019361 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062536 >>>>>>>>>>>>>>>> fix: >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062536/webrev.00/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>>> another GC >>>>>>>>>>>>>>>> is set >>>>>>>>>>>>>>>> outside >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I don't see why you have done this for the >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/sun/tools/jps/TestJps*.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> tests. They don't set any GC flags. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>>> "requires" to >>>>>>>>>>>>>>>> skip test >>>>>>>>>>>>>>>> if there is a conflict >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Just wondering: Does a skipped test get a .jtr file >>>>>>>>>>>>>>> showing it >>>>>>>>>>>>>>> was skipped; or does it only appear in the higher-level >>>>>>>>>>>>>>> jtreg log? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> David >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and >>>>>>>>>>>>>>>> without >>>>>>>>>>>>>>>> any GC flag). Tests are being excluded as expected. No >>>>>>>>>>>>>>>> tests >>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> // >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>> >>> -- >>> /Evgeniya Stepanova/ >> >> -- >> /Evgeniya Stepanova/ -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrey.x.zakharov at oracle.com Wed Nov 19 13:42:16 2014 From: andrey.x.zakharov at oracle.com (Andrey Zakharov) Date: Wed, 19 Nov 2014 17:42:16 +0400 Subject: RFR: 8059661: Need test for SoftReference and OOM In-Reply-To: <1415094963.5218.6.camel@oracle.com> References: <542E871D.4070000@oracle.com> <542E8843.8010301@oracle.com> <1412767715.2690.24.camel@cirrus> <543E8636.4030909@oracle.com> <5440F4B1.2060304@oracle.com> <1415094963.5218.6.camel@oracle.com> Message-ID: <546C9E38.7080101@oracle.com> Thanks, Thomas for the review! Is your review enough or I need another one? Thanks. On 04.11.2014 13:56, Thomas Schatzl wrote: > Hi Andrey, > > On Fri, 2014-10-17 at 14:51 +0400, Andrey Zakharov wrote: >> Hi, Thomas, Jesper, could you please a review this simple test as GC >> experts. >> Thanks. >> > looks okay. > > Thanks, > Thomas > > From thomas.schatzl at oracle.com Wed Nov 19 14:55:34 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 19 Nov 2014 15:55:34 +0100 Subject: RFR: 8059661: Need test for SoftReference and OOM In-Reply-To: <546C9E38.7080101@oracle.com> References: <542E871D.4070000@oracle.com> <542E8843.8010301@oracle.com> <1412767715.2690.24.camel@cirrus> <543E8636.4030909@oracle.com> <5440F4B1.2060304@oracle.com> <1415094963.5218.6.camel@oracle.com> <546C9E38.7080101@oracle.com> Message-ID: <1416408934.13933.3.camel@oracle.com> Hi, On Wed, 2014-11-19 at 17:42 +0400, Andrey Zakharov wrote: > Thanks, Thomas for the review! > Is your review enough or I need another one? one "R"eviewer is enough if nobody else has time. Thomas From staffan.friberg at oracle.com Wed Nov 19 16:23:26 2014 From: staffan.friberg at oracle.com (Staffan Friberg) Date: Wed, 19 Nov 2014 17:23:26 +0100 Subject: RFR: JDK-8055845 - Add trace event for promoted objects In-Reply-To: <545B6F58.2000907@oracle.com> References: <53FB6432.1010501@oracle.com> <53FB6A86.1080908@oracle.com> <53FB7338.9030208@oracle.com> <53FC4A57.3030100@oracle.com> <53FDEB15.2070102@oracle.com> <53FEF5F4.5060509@oracle.com> <53FF9C00.5040306@oracle.com> <54001EFF.3030802@oracle.com> <5400FF6F.5040203@oracle.com> <540A3730.1060603@oracle.com> <5416D64C.9020207@oracle.com> <545B4986.1050700@oracle.com> <545B5520.7010407@oracle.com> <545B70D2.9080008@oracle.com> <545B6F58.2000907@oracle.com> Message-ID: <546CC3FE.7070105@oracle.com> Hi, Erik and Tomas asked me to separate the G1 changes which was pushed separately in JDK-8064473. Erik also had some further comments offline that I have taken care of. Here is the new webrev, cr.openjdk.java.net/~sfriberg/8055845/webrev.06 Thanks, Staffan On 11/06/2014 01:53 PM, Bengt Rutisson wrote: > > > On 2014-11-06 14:00, Staffan Friberg wrote: >> Good catch, fixed. >> >> // Too large; allocate the object individually. >> obj = sp->par_allocate(word_sz); >> if (obj != NULL) { >> gc_tracer()->report_promotion_outside_plab_event(old, word_sz, age, >> false); >> } >> >> Please let me know if anyone wants a full new webrev with this. > > > Looks good. Reviewed. > > Bengt > >> >> Thanks, >> Staffan >> >> >> On 11/06/2014 12:01 PM, Bengt Rutisson wrote: >>> >>> Hi Staffan, >>> >>> On 2014-11-06 11:12, Staffan Friberg wrote: >>>> Hi, >>>> >>>> After further off list discussion it was decided to keep the >>>> gc_tracer in par_promote as is. >>>> >>>> I have uploaded a new webrev, >>>> http://cr.openjdk.java.net/~sfriberg/8055845/webrev.05 >>>> >>>> The main change here is a rewrite of the G1 code which is cleaner >>>> and also reuses the read age. By sending the markOop down through >>>> the call we can now trust the read age and do not need to reread it >>>> when incrementing which improves the YC performance slightly as it >>>> avoids the rather complex bit extraction. >>> >>> >>> Looks good to me. One detail in parNewGeneration.cpp: >>> >>> 274 } else { >>> 275 // Too large; allocate the object individually. >>> 276 gc_tracer()->report_promotion_outside_plab_event(old, word_sz, >>> age, false); >>> 277 obj = sp->par_allocate(word_sz); >>> 278 } >>> >>> Seems like par_allocate() return NULL. Maybe we should check that >>> before reporting the event. Similarly to what you do in the other >>> GCs, for example G1: >>> >>> g1Allocator.cpp >>> >>> 141 obj = _g1h->par_allocate_during_gc(purpose, word_sz, context); >>> 142 if (obj != NULL >>> 143 && >>> _g1h->_gc_tracer_stw->should_report_promotion_outside_plab_event()) { >>> 144 bool tenured = >>> _g1h->heap_region_containing_raw(obj)->is_old(); >>> 145 _g1h->_gc_tracer_stw->report_promotion_outside_plab_event(old, >>> word_sz, >>> 146 age, tenured); >>> 147 } >>> >>> Thanks, >>> Bengt >>> >>> >>> >>>> >>>> Cheers, >>>> Staffan >>>> >>>> On 09/15/2014 02:06 PM, Bengt Rutisson wrote: >>>>> >>>>> Hi Staffan, >>>>> >>>>> psPromotionManager.inline.hpp >>>>> >>>>> I think the PSPromotionManager::copy_to_survivor_space() might >>>>> send multiple events. If the allocation to the young gen fails we >>>>> will fall through to do an old gen allocation. But we send the >>>>> events before we realize that the allocation has failed, i.e. >>>>> new_obj is NULL. >>>>> >>>>> I talked to Erik a bit about how to handle the gc_tracer in >>>>> par_promote. He'll get back to you with some thoughts on that. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> >>>>> On 2014-09-06 00:20, Staffan Friberg wrote: >>>>>> Hi, >>>>>> >>>>>> I have uploaded a new webrev here, >>>>>> cr.openjdk.java.net/~sfriberg/8055845/webrev.03 >>>>>> >>>>>> It contains several changes >>>>>> >>>>>> - Split event into two events (PromoteObjectInNewPLAB, >>>>>> PromoteObjectOutsidePLAB) >>>>>> - Moved events to "vm/gc/detailed/PromoteObject..." >>>>>> - Supporting ParNew+CMS and ParNew+SerialOld tenuring >>>>>> - Not sure if the way I do it with passing the >>>>>> ParNewTracer is the best solution, please let me know if you have >>>>>> an idea how to improve it >>>>>> - Simplified the G1 code to avoid sending age and having a >>>>>> single call site >>>>>> - Fixed so that the generated event has size information in >>>>>> bytes rather than words >>>>>> >>>>>> Thanks for offline comments and suggestions from Dmitry and Thomas. >>>>>> >>>>>> Cheers, >>>>>> Staffan >>>>>> >>>>>> On 08/29/2014 03:32 PM, Staffan Friberg wrote: >>>>>>> Hi Erik, >>>>>>> >>>>>>> On 08/28/2014 11:34 PM, Erik Helin wrote: >>>>>>>> (it seems like we lost hotspot-gc-dev at openjdk.java.net >>>>>>>> somewhere in this thread, I'm adding it back) >>>>>>>> >>>>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>>>> Hi Erik, >>>>>>>>> >>>>>>>>> Thanks for the comments. >>>>>>>>>> - Aren't the events for promotion to the tenured generation >>>>>>>>>> (SerialOld) >>>>>>>>>> and the CMS generation missing? >>>>>>>>> The reason for leaving out these two, Serial completely and CMS >>>>>>>>> promotion, was due to that neither as far as I understand make >>>>>>>>> use of >>>>>>>>> PLABs. >>>>>>>> >>>>>>>> I might be wrong here, but looking at the function >>>>>>>> TenuredGeneration::par_promote (in tenuredGeneration.cpp) it >>>>>>>> looks to me like SerialOld is using PLABs when ParNew is >>>>>>>> promoting objects from young to old. >>>>>>>> >>>>>>>> As for CMS, looking at >>>>>>>> ConcurrentMarkSweepGeneration::par_promote (in >>>>>>>> concurrentMarkSweepGeneration.cpp) it seems like each >>>>>>>> CMSParGCThreadState has a CFLS_LAB (CompactibleFreeListSpace >>>>>>>> Local Allocation Buffer) that is a thread-local allocation >>>>>>>> buffer. See compactibleFreeListSpace.{hpp,cpp} for more details. >>>>>>>> >>>>>>>> Given this, I think we should add events for Serial and CMS as >>>>>>>> well. >>>>>>>> >>>>>>> >>>>>>> Ok I see what you mean with CMS, basically the equivalent to >>>>>>> getting a PLAB would be when we refill the CFLS_LAB in the alloc >>>>>>> function. It still does the allocation to a small memory (~ size >>>>>>> of object) area from the freelist, but at least we did extra >>>>>>> work to refill the local CFLS_LAB. Need to do some analysis to >>>>>>> see how often we refill so the overhead doesn't get too high. >>>>>>> The only issue I run into is how I can in a nice way get access >>>>>>> to the ParNewTracer from ParNewGeneration to call the report >>>>>>> function. Let's sync up next week and see how it can be solved. >>>>>>> >>>>>>> The tenured GC requires something similar to be supported, >>>>>>> however since ParNewGC is deprecated for usage without CMS in >>>>>>> JDK 8 we might want to skip that combination. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>>>>> - Would it make sense to differentiate, either by separate >>>>>>>>>> events or by >>>>>>>>>> a field in a event, between promotions to to-space and to >>>>>>>>>> the old >>>>>>>>>> generation? >>>>>>>>>> - The are two events for TLAB allocations, >>>>>>>>>> java/object_alloc_in_new_TLAB and >>>>>>>>>> java/object_alloc_outside_TLAB. >>>>>>>>>> What do you think about using two events for PLAB >>>>>>>>>> allocations as well: >>>>>>>>>> - java/object_alloc_in_new_PLAB >>>>>>>>>> - java/object_alloc_outside_PLAB >>>>>>>>> I think this is a matter of taste and probably how similar we >>>>>>>>> want the >>>>>>>>> event to be to the existing allocation event. I personally >>>>>>>>> prefer a >>>>>>>>> single event but if the GC team and serviceability team feel >>>>>>>>> it would be >>>>>>>>> better to have two I can certainly rewrite. The reason for me >>>>>>>>> preferring >>>>>>>>> a single event is just ease of analysis, you can easily filter >>>>>>>>> a list of >>>>>>>>> events on a field, it is harder to merge two different events >>>>>>>>> with >>>>>>>>> different fields and work with them in an straight forward >>>>>>>>> fashion. >>>>>>>>> >>>>>>>>> Any general preference for having a single or multiple events? >>>>>>>> >>>>>>>> I would prefer to have two events in this case and try to >>>>>>>> follow the existing allocation events as much as possible (both >>>>>>>> in naming and in style). Keeping the tenured field (I missed it >>>>>>>> the first time I read the patch) is good. >>>>>>>> >>>>>>> Yes, tenured would be independent of having two events, only >>>>>>> PLAB size and directAllocation would be affected when having two >>>>>>> event types. >>>>>>> >>>>>>> *Erik Gahlin*, any preference from your end? >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 2014-08-28 23:15, Staffan Friberg wrote: >>>>>>>>>> - In PSPromotionManager, instead of utilizing the C++ >>>>>>>>>> friendship with >>>>>>>>>> PSScavenge, should we add a getter function for the gc_tracer? >>>>>>>>> Created a getter function. >>>>>>>> >>>>>>>> Thanks! If you make report_promotion_sample const, then the >>>>>>>> getter can return a const ParallelScavengeTracer*, right? >>>>>>>> >>>>>>> Done >>>>>>> >>>>>>> //Staffan >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.masamitsu at oracle.com Wed Nov 19 16:53:11 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 19 Nov 2014 08:53:11 -0800 Subject: Major GC occurring before CMS occupancy fraction In-Reply-To: <20141119021337.0000667b.ecki@zusammenkunft.net> References: <20141119021337.0000667b.ecki@zusammenkunft.net> Message-ID: <546CCAF7.1070602@oracle.com> On 11/18/2014 5:13 PM, Bernd Eckenfels wrote: > Hello, > > you need to provide verbose GC logs so we can say anything on that. But > be aware that you might not get much help on ancient Java :) > > Can you try to set the initial perm size to the maximum size, to make > sure resizes in this area do not trigger GC. And are you sure you are > not confronted with RMI DGC runs? If it looks like the occupancy of the perm gen is the issue, try setting CMSTriggerPermRatio higher (default is 80) or try setting CMSInitiatingPermOccupancyFraction to a high value (similar to the way you choose CMSInitiatingOccupancyFraction. Jon > Gruss > Bernd > > Am Tue, 18 > Nov 2014 12:24:23 -0500 schrieb nitin sharma > : > >> hi, >> >> I am facing the same issue.. We are using JDK 1.6. 0_23 and have set >> Occupancy Fraction to 80% but i can see CMS triggered at much below >> ration (around 53%).. >> >> is there a solution for same? >> >> I have used "-XX:CMSInitiatingOccupancyFraction=80 >> -XX:+UseCMSInitiatingOccupancyOnly" option... >> >> complete JVM argument set ==> >> >> /usr/java/jdk1.6.0_23/bin/java -server -Xms16384m -Xmx16384m >> -XX:PermSize=512m -XX:MaxPermSize=1024m -XX:CompileThreshold=8000 >> -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps >> -XX:+PrintGCTimeStamps -XX:+UseParNewGC -XX:MaxNewSize=6144m >> -XX:NewSize=6144m -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=4 >> -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled >> -XX:CMSInitiatingOccupancyFraction=80 >> -XX:+UseCMSInitiatingOccupancyOnly -DUseSunHttpHandler=true >> -XX:+HeapDumpOnOutOfMemoryError >> >> >> >> Regards, >> Nitin Kumar Sharma. >> From alexander.harlap at oracle.com Wed Nov 19 16:55:56 2014 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Wed, 19 Nov 2014 11:55:56 -0500 Subject: Need a sponsor for push into jdk9/hs-gc In-Reply-To: <546B94A9.5020107@oracle.com> References: <546B94A9.5020107@oracle.com> Message-ID: <546CCB9C.9030701@oracle.com> I need a sponsor to push this (reviewed by Mikael Gerdin) trivial change. My jdk username is aharlap. Change passed JPRT build, test and failed on push because I do not have commiter rights. Thank you, Alex -------- Original Message -------- Subject: Re: Review request for 8059492: Wrong spelling in assert: "Not initialied properly?" Date: Tue, 18 Nov 2014 19:49:13 +0100 From: Mikael Gerdin Organization: Oracle Corporation To: Alexander Harlap , jdk9-dev at openjdk.java.net Hi Alex, On 2014-11-18 17:29, Alexander Harlap wrote: > Please help to code review the change in JDK9 (trivial) for the > following CR: > 8059492: Wrong spelling in assert: "Not initialied properly?" > https://bugs.openjdk.java.net/browse/JDK-8059492 > > The webrev is located at: > http://cr.openjdk.java.net/~aharlap/8059492/webrev.00/ > The change looks good to me, Reviewed. /Mikael > > Testing completed: JPRT > > Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.johansson at oracle.com Wed Nov 19 17:00:34 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 19 Nov 2014 18:00:34 +0100 Subject: RFR: 8065227: Report allocation context stats at end of cleanup Message-ID: <546CCCB2.9080601@oracle.com> Hi, Please review this small fix for: https://bugs.openjdk.java.net/browse/JDK-8065227 Webrev: http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ Summary: The update for the allocation context has previously been done in the remark phase but this is a little to early. The last stw-phase of the marking cycle is the cleanup and if doing the reporting there that simplifies. Thanks, Stefan From jwha at google.com Wed Nov 19 17:13:32 2014 From: jwha at google.com (Jungwoo Ha) Date: Wed, 19 Nov 2014 09:13:32 -0800 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> <5469C9C1.7020306@oracle.com> <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> Message-ID: http://cr.openjdk.java.net/~rasbold/8061259/webrev.06/ PTAL. On Mon, Nov 17, 2014 at 7:57 AM, Kim Barrett wrote: > On Nov 17, 2014, at 5:11 AM, Bengt Rutisson > wrote: > > > >> I think this does leave open the possibility of moving the > >> _promotion_failed flag check earlier in copy_to_survivor_space_XXX(), > >> [?] > >> > >> I'm not sure it would be overall beneficial though, [?] > > > > Yes, there is probably room for more improvement here. I think adding > the check in the copy_to_survivor_space_XXX_undo() methods is a clear > improvement in many cases. So, doing that as a first step is a good start. > We can consider moving it earlier as a future enhancement. > > Agreed. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.wilhelmsson at oracle.com Wed Nov 19 22:29:47 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 19 Nov 2014 23:29:47 +0100 Subject: Need a sponsor for push into jdk9/hs-gc In-Reply-To: <546CCB9C.9030701@oracle.com> References: <546B94A9.5020107@oracle.com> <546CCB9C.9030701@oracle.com> Message-ID: <546D19DB.3020709@oracle.com> Hi Alex, If you haven't found anyone already I can push this. Do an "hg export" of your outgoing change and send me the diff. /Jesper Alexander Harlap skrev 19/11/14 17:55: > I need a sponsor to push this (reviewed by Mikael Gerdin) trivial change. > My jdk username is aharlap. > > Change passed JPRT build, test and failed on push because I do not have > commiter rights. > > Thank you, > > Alex > > > -------- Original Message -------- > Subject: Re: Review request for 8059492: Wrong spelling in assert: "Not > initialied properly?" > Date: Tue, 18 Nov 2014 19:49:13 +0100 > From: Mikael Gerdin > Organization: Oracle Corporation > To: Alexander Harlap , jdk9-dev at openjdk.java.net > > > > Hi Alex, > > On 2014-11-18 17:29, Alexander Harlap wrote: > > Please help to code review the change in JDK9 (trivial) for the > > following CR: > > 8059492: Wrong spelling in assert: "Not initialied properly?" > >https://bugs.openjdk.java.net/browse/JDK-8059492 > > > > The webrev is located at: > >http://cr.openjdk.java.net/~aharlap/8059492/webrev.00/ > > > > The change looks good to me, Reviewed. > > /Mikael > > > > > Testing completed: JPRT > > > > Alex > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.masamitsu at oracle.com Thu Nov 20 03:32:27 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 19 Nov 2014 19:32:27 -0800 Subject: RFR: 8065227: Report allocation context stats at end of cleanup In-Reply-To: <546CCCB2.9080601@oracle.com> References: <546CCCB2.9080601@oracle.com> Message-ID: <546D60CB.6020403@oracle.com> Stefan, Why did you chose to rename update_at_remark() to update_after_mark()? Instead of update_at_cleanup()? Jon On 11/19/2014 9:00 AM, Stefan Johansson wrote: > Hi, > > Please review this small fix for: > https://bugs.openjdk.java.net/browse/JDK-8065227 > > Webrev: > http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ > > Summary: > The update for the allocation context has previously been done in the > remark phase but this is a little to early. The last stw-phase of the > marking cycle is the cleanup and if doing the reporting there that > simplifies. > > Thanks, > Stefan > > From mikael.gerdin at oracle.com Thu Nov 20 08:46:06 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 20 Nov 2014 09:46:06 +0100 Subject: RFR [1/2] 8065358: JDK-8065218 Move CMS-specific fields from Space to CompactibleFreeListSpace Message-ID: <546DAA4E.3080807@oracle.com> Hi all, This is the first patch of two to fix JDK-8065358. In this change I suggest that we move a field used only by CMS to CompactibleFreeListSpace and introduce a virtual getter in the Space base class. I estimate that the performance impact of the virtual call will be small since the getter path already does several virtual calls. Moving the field will allow me to add another field to G1's G1OffsetTableContigSpace in patch [2/2] without introducing a footprint regression. Webrev: http://cr.openjdk.java.net/~mgerdin/8065218/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8065218 Parent bug: https://bugs.openjdk.java.net/browse/JDK-8065358 Testing: JPRT Thanks /Mikael From mikael.gerdin at oracle.com Thu Nov 20 08:46:09 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 20 Nov 2014 09:46:09 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races Message-ID: <546DAA51.80601@oracle.com> Hi all, This is the second patch of two to fix JDK-8065358 In this change I suggest that we introduce a new field _scan_top to G1OffsetTableContigSpace to keep track of the maximum address where a GC worker thread is allowed to scan objects in old regions which have been used as allocation regions during the current evacuation cycle. More background (from the bug): G1 currently uses the _saved_mark_word field in Space to detemine a point in a region above which a current GC cannot scan objects. This is needed to prevent card scanning threads from scanning in memory regions where other gc workers are concurrently allocating objects. This re-use of the _saved_mark_word field causes confusion to readers of the code since it's not used for the same purpose as the other Space classes. The setting and reading of this field, and the per region gc time stamp which accompanies it is also unnecessarily racy. Previously the _saved_mark_word field was set to the value of the _top pointer when a region was selected as a GC allocation region and the time stamp was set to indicate that the saved mark field should be used as a maximum address. This code had some problems with races in JDK-8058209 and could be redesigned in a less racy way. Suggested fix is to introduce a new field in HeapRegions to keep track of the maximum address where card scanning is allowed and set that field at the point of retaining an OldGCAllocRegion instead of when allocations are being started. That way we get rid of the store ordering requirement in the timestamp record path. There is still a race between the per region time stamps and the per region top pointer, where we must ensure that the time stamp store must be visible before any subsequent top stores. This store ordering is enforced by the fact that all stores to top are either ordered with #storestore (the initial allocation) or guarded by a Mutex. To ensure that the reader path sees a consistent view it must be exectued with the proper load ordering, where we must read top before the time stamp in order to ensure that we don't see a top value which has been updated by a concurrent gc worker if we see a time stamp from a previous gc cycle. Webrev: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8065358 Testing: JPRT, Kitchensink (4hr+) Thanks /Mikael From bengt.rutisson at oracle.com Thu Nov 20 08:43:55 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 20 Nov 2014 09:43:55 +0100 Subject: RFR [1/2] 8065358: JDK-8065218 Move CMS-specific fields from Space to CompactibleFreeListSpace In-Reply-To: <546DAA4E.3080807@oracle.com> References: <546DAA4E.3080807@oracle.com> Message-ID: <546DA9CB.6020405@oracle.com> Hi Mikael, Looks good. Bengt On 2014-11-20 09:46, Mikael Gerdin wrote: > Hi all, > > This is the first patch of two to fix JDK-8065358. > > In this change I suggest that we move a field used only by CMS to > CompactibleFreeListSpace and introduce a virtual getter in the Space > base class. > > I estimate that the performance impact of the virtual call will be > small since the getter path already does several virtual calls. > > Moving the field will allow me to add another field to G1's > G1OffsetTableContigSpace in patch [2/2] without introducing a > footprint regression. > > Webrev: http://cr.openjdk.java.net/~mgerdin/8065218/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8065218 > Parent bug: https://bugs.openjdk.java.net/browse/JDK-8065358 > Testing: JPRT > > Thanks > /Mikael From stefan.johansson at oracle.com Thu Nov 20 09:14:50 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 20 Nov 2014 10:14:50 +0100 Subject: RFR: 8065227: Report allocation context stats at end of cleanup In-Reply-To: <546D60CB.6020403@oracle.com> References: <546CCCB2.9080601@oracle.com> <546D60CB.6020403@oracle.com> Message-ID: <546DB10A.5020403@oracle.com> Hi Jon, Thanks for taking a look at this change. On 2014-11-20 04:32, Jon Masamitsu wrote: > Stefan, > > Why did you chose to rename update_at_remark() to update_after_mark()? > Instead of update_at_cleanup()? Changing it to update_at_cleanup() was my first thought, but I think update_after_mark() is more descriptive and a better fit when looking how its implemented. Thanks, Stefan > > Jon > > On 11/19/2014 9:00 AM, Stefan Johansson wrote: >> Hi, >> >> Please review this small fix for: >> https://bugs.openjdk.java.net/browse/JDK-8065227 >> >> Webrev: >> http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ >> >> Summary: >> The update for the allocation context has previously been done in the >> remark phase but this is a little to early. The last stw-phase of the >> marking cycle is the cleanup and if doing the reporting there that >> simplifies. >> >> Thanks, >> Stefan >> >> > From bengt.rutisson at oracle.com Thu Nov 20 10:01:25 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 20 Nov 2014 11:01:25 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races In-Reply-To: <546DAA51.80601@oracle.com> References: <546DAA51.80601@oracle.com> Message-ID: <546DBBF5.4080803@oracle.com> Hi Mikael, This looks good to me. Two minor comments in G1OffsetTableContigSpace::record_timestamp(): The register volatile keywords here look like a remainder from the debugging of the original top() reordering problem. 1027 register HeapWord* volatile st = _scan_top; This check: 1021 if (_gc_time_stamp < curr_gc_time_stamp) { Was already there in the old code but it looks odd to me. We should never have _gc_time_stamp > curr_gc_time_stamp and if they are the same it doesn't hurt to set it. So, I guess this is an unnecessary check. Can we turn it into a guarantee(_gc_time_stamp <= curr_gc_time_stamp) instead? Maybe that should be done as a separate change... Thanks, Bengt On 2014-11-20 09:46, Mikael Gerdin wrote: > Hi all, > > This is the second patch of two to fix JDK-8065358 > > In this change I suggest that we introduce a new field _scan_top to > G1OffsetTableContigSpace to keep track of the maximum address where a > GC worker thread is allowed to scan objects in old regions which have > been used as allocation regions during the current evacuation cycle. > > More background (from the bug): > G1 currently uses the _saved_mark_word field in Space to detemine a > point in a region above which a current GC cannot scan objects. This > is needed to prevent card scanning threads from scanning in memory > regions where other gc workers are concurrently allocating objects. > > This re-use of the _saved_mark_word field causes confusion to readers > of the code since it's not used for the same purpose as the other > Space classes. > The setting and reading of this field, and the per region gc time > stamp which accompanies it is also unnecessarily racy. Previously the > _saved_mark_word field was set to the value of the _top pointer when a > region was selected as a GC allocation region and the time stamp was > set to indicate that the saved mark field should be used as a maximum > address. This code had some problems with races in JDK-8058209 and > could be redesigned in a less racy way. > > Suggested fix is to introduce a new field in HeapRegions to keep track > of the maximum address where card scanning is allowed and set that > field at the point of retaining an OldGCAllocRegion instead of when > allocations are being started. That way we get rid of the store > ordering requirement in the timestamp record path. > > There is still a race between the per region time stamps and the per > region top pointer, where we must ensure that the time stamp store > must be visible before any subsequent top stores. > This store ordering is enforced by the fact that all stores to top are > either ordered with #storestore (the initial allocation) or guarded by > a Mutex. To ensure that the reader path sees a consistent view it must > be exectued with the proper load ordering, where we must read top > before the time stamp in order to ensure that we don't see a top value > which has been updated by a concurrent gc worker if we see a time > stamp from a previous gc cycle. > > Webrev: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8065358 > Testing: JPRT, Kitchensink (4hr+) > > Thanks > /Mikael From mikael.gerdin at oracle.com Thu Nov 20 10:24:20 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 20 Nov 2014 11:24:20 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races In-Reply-To: <546DBBF5.4080803@oracle.com> References: <546DAA51.80601@oracle.com> <546DBBF5.4080803@oracle.com> Message-ID: <546DC154.1030608@oracle.com> Hi Bengt, thanks for taking a look at this and for the discussions leading to me developing this change. On 2014-11-20 11:01, Bengt Rutisson wrote: > > Hi Mikael, > > This looks good to me. > > Two minor comments in G1OffsetTableContigSpace::record_timestamp(): > > The register volatile keywords here look like a remainder from the > debugging of the original top() reordering problem. > > 1027 register HeapWord* volatile st = _scan_top; Yes, I'll remove those. Thanks for catching this. > > > This check: > > 1021 if (_gc_time_stamp < curr_gc_time_stamp) { > > Was already there in the old code but it looks odd to me. We should > never have _gc_time_stamp > curr_gc_time_stamp and if they are the same > it doesn't hurt to set it. So, I guess this is an unnecessary check. Can > we turn it into a guarantee(_gc_time_stamp <= curr_gc_time_stamp) > instead? Maybe that should be done as a separate change... I agree that this looks odd. I'd prefer to defer this to a future change. /Mikael > > Thanks, > Bengt > > > > On 2014-11-20 09:46, Mikael Gerdin wrote: >> Hi all, >> >> This is the second patch of two to fix JDK-8065358 >> >> In this change I suggest that we introduce a new field _scan_top to >> G1OffsetTableContigSpace to keep track of the maximum address where a >> GC worker thread is allowed to scan objects in old regions which have >> been used as allocation regions during the current evacuation cycle. >> >> More background (from the bug): >> G1 currently uses the _saved_mark_word field in Space to detemine a >> point in a region above which a current GC cannot scan objects. This >> is needed to prevent card scanning threads from scanning in memory >> regions where other gc workers are concurrently allocating objects. >> >> This re-use of the _saved_mark_word field causes confusion to readers >> of the code since it's not used for the same purpose as the other >> Space classes. >> The setting and reading of this field, and the per region gc time >> stamp which accompanies it is also unnecessarily racy. Previously the >> _saved_mark_word field was set to the value of the _top pointer when a >> region was selected as a GC allocation region and the time stamp was >> set to indicate that the saved mark field should be used as a maximum >> address. This code had some problems with races in JDK-8058209 and >> could be redesigned in a less racy way. >> >> Suggested fix is to introduce a new field in HeapRegions to keep track >> of the maximum address where card scanning is allowed and set that >> field at the point of retaining an OldGCAllocRegion instead of when >> allocations are being started. That way we get rid of the store >> ordering requirement in the timestamp record path. >> >> There is still a race between the per region time stamps and the per >> region top pointer, where we must ensure that the time stamp store >> must be visible before any subsequent top stores. >> This store ordering is enforced by the fact that all stores to top are >> either ordered with #storestore (the initial allocation) or guarded by >> a Mutex. To ensure that the reader path sees a consistent view it must >> be exectued with the proper load ordering, where we must read top >> before the time stamp in order to ensure that we don't see a top value >> which has been updated by a concurrent gc worker if we see a time >> stamp from a previous gc cycle. >> >> Webrev: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8065358 >> Testing: JPRT, Kitchensink (4hr+) >> >> Thanks >> /Mikael > From thomas.schatzl at oracle.com Thu Nov 20 10:53:09 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 20 Nov 2014 11:53:09 +0100 Subject: RFR [1/2] 8065358: JDK-8065218 Move CMS-specific fields from Space to CompactibleFreeListSpace In-Reply-To: <546DAA4E.3080807@oracle.com> References: <546DAA4E.3080807@oracle.com> Message-ID: <1416480789.4606.6.camel@oracle.com> hi, On Thu, 2014-11-20 at 09:46 +0100, Mikael Gerdin wrote: > Hi all, > > This is the first patch of two to fix JDK-8065358. > > In this change I suggest that we move a field used only by CMS to > CompactibleFreeListSpace and introduce a virtual getter in the Space > base class. > > I estimate that the performance impact of the virtual call will be small > since the getter path already does several virtual calls. > > Moving the field will allow me to add another field to G1's > G1OffsetTableContigSpace in patch [2/2] without introducing a footprint > regression. > > Webrev: http://cr.openjdk.java.net/~mgerdin/8065218/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8065218 > Parent bug: https://bugs.openjdk.java.net/browse/JDK-8065358 > Testing: JPRT Looks good. Thanks, Thomas From mikael.gerdin at oracle.com Thu Nov 20 12:28:17 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 20 Nov 2014 13:28:17 +0100 Subject: RFR: 8065227: Report allocation context stats at end of cleanup In-Reply-To: <546DB10A.5020403@oracle.com> References: <546CCCB2.9080601@oracle.com> <546D60CB.6020403@oracle.com> <546DB10A.5020403@oracle.com> Message-ID: <546DDE61.9000606@oracle.com> Hi Stefan, On 2014-11-20 10:14, Stefan Johansson wrote: > Hi Jon, > > Thanks for taking a look at this change. > > On 2014-11-20 04:32, Jon Masamitsu wrote: >> Stefan, >> >> Why did you chose to rename update_at_remark() to update_after_mark()? >> Instead of update_at_cleanup()? > Changing it to update_at_cleanup() was my first thought, but I think > update_after_mark() is more descriptive and a better fit when looking > how its implemented. Looks good to me, I'm fine with either naming variant. /Mikael > > Thanks, > Stefan >> >> Jon >> >> On 11/19/2014 9:00 AM, Stefan Johansson wrote: >>> Hi, >>> >>> Please review this small fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8065227 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ >>> >>> Summary: >>> The update for the allocation context has previously been done in the >>> remark phase but this is a little to early. The last stw-phase of the >>> marking cycle is the cleanup and if doing the reporting there that >>> simplifies. >>> >>> Thanks, >>> Stefan >>> >>> >> > From stefan.johansson at oracle.com Thu Nov 20 13:16:47 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 20 Nov 2014 14:16:47 +0100 Subject: RFR: 8065227: Report allocation context stats at end of cleanup In-Reply-To: <546DDE61.9000606@oracle.com> References: <546CCCB2.9080601@oracle.com> <546D60CB.6020403@oracle.com> <546DB10A.5020403@oracle.com> <546DDE61.9000606@oracle.com> Message-ID: <546DE9BF.4040104@oracle.com> Thanks for the review Mikael, Stefan On 2014-11-20 13:28, Mikael Gerdin wrote: > Hi Stefan, > > On 2014-11-20 10:14, Stefan Johansson wrote: >> Hi Jon, >> >> Thanks for taking a look at this change. >> >> On 2014-11-20 04:32, Jon Masamitsu wrote: >>> Stefan, >>> >>> Why did you chose to rename update_at_remark() to update_after_mark()? >>> Instead of update_at_cleanup()? >> Changing it to update_at_cleanup() was my first thought, but I think >> update_after_mark() is more descriptive and a better fit when looking >> how its implemented. > > Looks good to me, I'm fine with either naming variant. > > /Mikael > >> >> Thanks, >> Stefan >>> >>> Jon >>> >>> On 11/19/2014 9:00 AM, Stefan Johansson wrote: >>>> Hi, >>>> >>>> Please review this small fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8065227 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ >>>> >>>> Summary: >>>> The update for the allocation context has previously been done in the >>>> remark phase but this is a little to early. The last stw-phase of the >>>> marking cycle is the cleanup and if doing the reporting there that >>>> simplifies. >>>> >>>> Thanks, >>>> Stefan >>>> >>>> >>> >> From mikael.gerdin at oracle.com Thu Nov 20 13:24:48 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 20 Nov 2014 14:24:48 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races In-Reply-To: <546DAA51.80601@oracle.com> References: <546DAA51.80601@oracle.com> Message-ID: <546DEBA0.3040207@oracle.com> Hi, (I just noticed that I mistyped the bug number in the subject, it should be 8065358 twice.) I removed the weird qualifier combination from "st" in record_timestamp(). Thomas also suggested to me that the guarantee in record_retained_region may in fact be incorrect in some corner cases, I removed it since no other part of the code actually depends on this being true. New webrev at: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.1/ Incremental webrev at: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0_to_1/ /Mikael On 2014-11-20 09:46, Mikael Gerdin wrote: > Hi all, > > This is the second patch of two to fix JDK-8065358 > > In this change I suggest that we introduce a new field _scan_top to > G1OffsetTableContigSpace to keep track of the maximum address where a GC > worker thread is allowed to scan objects in old regions which have been > used as allocation regions during the current evacuation cycle. > > More background (from the bug): > G1 currently uses the _saved_mark_word field in Space to detemine a > point in a region above which a current GC cannot scan objects. This is > needed to prevent card scanning threads from scanning in memory regions > where other gc workers are concurrently allocating objects. > > This re-use of the _saved_mark_word field causes confusion to readers of > the code since it's not used for the same purpose as the other Space > classes. > The setting and reading of this field, and the per region gc time stamp > which accompanies it is also unnecessarily racy. Previously the > _saved_mark_word field was set to the value of the _top pointer when a > region was selected as a GC allocation region and the time stamp was set > to indicate that the saved mark field should be used as a maximum > address. This code had some problems with races in JDK-8058209 and could > be redesigned in a less racy way. > > Suggested fix is to introduce a new field in HeapRegions to keep track > of the maximum address where card scanning is allowed and set that field > at the point of retaining an OldGCAllocRegion instead of when > allocations are being started. That way we get rid of the store ordering > requirement in the timestamp record path. > > There is still a race between the per region time stamps and the per > region top pointer, where we must ensure that the time stamp store must > be visible before any subsequent top stores. > This store ordering is enforced by the fact that all stores to top are > either ordered with #storestore (the initial allocation) or guarded by a > Mutex. To ensure that the reader path sees a consistent view it must be > exectued with the proper load ordering, where we must read top before > the time stamp in order to ensure that we don't see a top value which > has been updated by a concurrent gc worker if we see a time stamp from a > previous gc cycle. > > Webrev: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8065358 > Testing: JPRT, Kitchensink (4hr+) > > Thanks > /Mikael From bengt.rutisson at oracle.com Thu Nov 20 14:00:33 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 20 Nov 2014 15:00:33 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races In-Reply-To: <546DEBA0.3040207@oracle.com> References: <546DAA51.80601@oracle.com> <546DEBA0.3040207@oracle.com> Message-ID: <546DF401.7080907@oracle.com> Hi Mikael, On 11/20/14 2:24 PM, Mikael Gerdin wrote: > Hi, > > (I just noticed that I mistyped the bug number in the subject, it > should be 8065358 twice.) > > I removed the weird qualifier combination from "st" in > record_timestamp(). > > Thomas also suggested to me that the guarantee in > record_retained_region may in fact be incorrect in some corner cases, > I removed it since no other part of the code actually depends on this > being true. > > New webrev at: > http://cr.openjdk.java.net/~mgerdin/8065358/webrev.1/ > Incremental webrev at: > http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0_to_1/ New webrev looks good to me. Reviewed. Bengt > > /Mikael > > On 2014-11-20 09:46, Mikael Gerdin wrote: >> Hi all, >> >> This is the second patch of two to fix JDK-8065358 >> >> In this change I suggest that we introduce a new field _scan_top to >> G1OffsetTableContigSpace to keep track of the maximum address where a GC >> worker thread is allowed to scan objects in old regions which have been >> used as allocation regions during the current evacuation cycle. >> >> More background (from the bug): >> G1 currently uses the _saved_mark_word field in Space to detemine a >> point in a region above which a current GC cannot scan objects. This is >> needed to prevent card scanning threads from scanning in memory regions >> where other gc workers are concurrently allocating objects. >> >> This re-use of the _saved_mark_word field causes confusion to readers of >> the code since it's not used for the same purpose as the other Space >> classes. >> The setting and reading of this field, and the per region gc time stamp >> which accompanies it is also unnecessarily racy. Previously the >> _saved_mark_word field was set to the value of the _top pointer when a >> region was selected as a GC allocation region and the time stamp was set >> to indicate that the saved mark field should be used as a maximum >> address. This code had some problems with races in JDK-8058209 and could >> be redesigned in a less racy way. >> >> Suggested fix is to introduce a new field in HeapRegions to keep track >> of the maximum address where card scanning is allowed and set that field >> at the point of retaining an OldGCAllocRegion instead of when >> allocations are being started. That way we get rid of the store ordering >> requirement in the timestamp record path. >> >> There is still a race between the per region time stamps and the per >> region top pointer, where we must ensure that the time stamp store must >> be visible before any subsequent top stores. >> This store ordering is enforced by the fact that all stores to top are >> either ordered with #storestore (the initial allocation) or guarded by a >> Mutex. To ensure that the reader path sees a consistent view it must be >> exectued with the proper load ordering, where we must read top before >> the time stamp in order to ensure that we don't see a top value which >> has been updated by a concurrent gc worker if we see a time stamp from a >> previous gc cycle. >> >> Webrev: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8065358 >> Testing: JPRT, Kitchensink (4hr+) >> >> Thanks >> /Mikael From bengt.rutisson at oracle.com Thu Nov 20 14:02:04 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 20 Nov 2014 15:02:04 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races In-Reply-To: <546DC154.1030608@oracle.com> References: <546DAA51.80601@oracle.com> <546DBBF5.4080803@oracle.com> <546DC154.1030608@oracle.com> Message-ID: <546DF45C.9090707@oracle.com> Inline far below... On 11/20/14 11:24 AM, Mikael Gerdin wrote: > Hi Bengt, > > thanks for taking a look at this and for the discussions leading to me > developing this change. > > On 2014-11-20 11:01, Bengt Rutisson wrote: >> >> Hi Mikael, >> >> This looks good to me. >> >> Two minor comments in G1OffsetTableContigSpace::record_timestamp(): >> >> The register volatile keywords here look like a remainder from the >> debugging of the original top() reordering problem. >> >> 1027 register HeapWord* volatile st = _scan_top; > > Yes, I'll remove those. > Thanks for catching this. > >> >> >> This check: >> >> 1021 if (_gc_time_stamp < curr_gc_time_stamp) { >> >> Was already there in the old code but it looks odd to me. We should >> never have _gc_time_stamp > curr_gc_time_stamp and if they are the same >> it doesn't hurt to set it. So, I guess this is an unnecessary check. Can >> we turn it into a guarantee(_gc_time_stamp <= curr_gc_time_stamp) >> instead? Maybe that should be done as a separate change... > > I agree that this looks odd. I'd prefer to defer this to a future change. Just to close the loop on this. I'm fine with deferring this. Can you file a bug for it? Thanks, Bengt > > /Mikael > >> >> Thanks, >> Bengt >> >> >> >> On 2014-11-20 09:46, Mikael Gerdin wrote: >>> Hi all, >>> >>> This is the second patch of two to fix JDK-8065358 >>> >>> In this change I suggest that we introduce a new field _scan_top to >>> G1OffsetTableContigSpace to keep track of the maximum address where a >>> GC worker thread is allowed to scan objects in old regions which have >>> been used as allocation regions during the current evacuation cycle. >>> >>> More background (from the bug): >>> G1 currently uses the _saved_mark_word field in Space to detemine a >>> point in a region above which a current GC cannot scan objects. This >>> is needed to prevent card scanning threads from scanning in memory >>> regions where other gc workers are concurrently allocating objects. >>> >>> This re-use of the _saved_mark_word field causes confusion to readers >>> of the code since it's not used for the same purpose as the other >>> Space classes. >>> The setting and reading of this field, and the per region gc time >>> stamp which accompanies it is also unnecessarily racy. Previously the >>> _saved_mark_word field was set to the value of the _top pointer when a >>> region was selected as a GC allocation region and the time stamp was >>> set to indicate that the saved mark field should be used as a maximum >>> address. This code had some problems with races in JDK-8058209 and >>> could be redesigned in a less racy way. >>> >>> Suggested fix is to introduce a new field in HeapRegions to keep track >>> of the maximum address where card scanning is allowed and set that >>> field at the point of retaining an OldGCAllocRegion instead of when >>> allocations are being started. That way we get rid of the store >>> ordering requirement in the timestamp record path. >>> >>> There is still a race between the per region time stamps and the per >>> region top pointer, where we must ensure that the time stamp store >>> must be visible before any subsequent top stores. >>> This store ordering is enforced by the fact that all stores to top are >>> either ordered with #storestore (the initial allocation) or guarded by >>> a Mutex. To ensure that the reader path sees a consistent view it must >>> be exectued with the proper load ordering, where we must read top >>> before the time stamp in order to ensure that we don't see a top value >>> which has been updated by a concurrent gc worker if we see a time >>> stamp from a previous gc cycle. >>> >>> Webrev: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065358 >>> Testing: JPRT, Kitchensink (4hr+) >>> >>> Thanks >>> /Mikael >> From Leonid.Mesnik at oracle.com Thu Nov 20 15:13:52 2014 From: Leonid.Mesnik at oracle.com (Leonid Mesnik) Date: Thu, 20 Nov 2014 18:13:52 +0300 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced Message-ID: <546E0530.4060904@oracle.com> Hi Here is the fix which introduce WB method which stars concurrent mark cycle explicitly. It is needed to make CM cycle related tests more stable. Changeset also includes test and very small fix of WB method g1InConcurrentMark . Fix pass JPRT with hotspot testset. JIRA: https://bugs.openjdk.java.net/browse/JDK-8065579 Webrev: http://cr.openjdk.java.net/~sjohanss/lmesnik/8065579/hotspot.00/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.harlap at oracle.com Thu Nov 20 15:16:52 2014 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Thu, 20 Nov 2014 10:16:52 -0500 Subject: Need a sponsor for push into jdk9/hs-gc In-Reply-To: <546D19DB.3020709@oracle.com> References: <546B94A9.5020107@oracle.com> <546CCB9C.9030701@oracle.com> <546D19DB.3020709@oracle.com> Message-ID: <546E05E4.9020801@oracle.com> Thank you Jesper, 8059492.patch attached. Alex On 11/19/2014 5:29 PM, Jesper Wilhelmsson wrote: > Hi Alex, > > If you haven't found anyone already I can push this. Do an "hg export" > of your outgoing change and send me the diff. > /Jesper > > > Alexander Harlap skrev 19/11/14 17:55: >> I need a sponsor to push this (reviewed by Mikael Gerdin) trivial change. >> My jdk username is aharlap. >> >> Change passed JPRT build, test and failed on push because I do not >> have commiter rights. >> >> Thank you, >> >> Alex >> >> >> -------- Original Message -------- >> Subject: Re: Review request for 8059492: Wrong spelling in assert: >> "Not initialied properly?" >> Date: Tue, 18 Nov 2014 19:49:13 +0100 >> From: Mikael Gerdin >> Organization: Oracle Corporation >> To: Alexander Harlap , >> jdk9-dev at openjdk.java.net >> >> >> >> Hi Alex, >> >> On 2014-11-18 17:29, Alexander Harlap wrote: >> > Please help to code review the change in JDK9 (trivial) for the >> > following CR: >> > 8059492: Wrong spelling in assert: "Not initialied properly?" >> >https://bugs.openjdk.java.net/browse/JDK-8059492 >> > >> > The webrev is located at: >> >http://cr.openjdk.java.net/~aharlap/8059492/webrev.00/ >> > >> >> The change looks good to me, Reviewed. >> >> /Mikael >> >> > >> > Testing completed: JPRT >> > >> > Alex >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- # HG changeset patch # User aharlap # Date 1416495802 18000 # Node ID 9ba7b1be98b7333d5f725d044d1aa6058ddbeb47 # Parent 898c20a0184e72610670d0463e5052725f792b10 8059492: Wrong spelling in assert: "Not initialied properly?" Reviewed-by: mgerdin Contributed-by: aharlap diff -r 898c20a0184e -r 9ba7b1be98b7 src/share/vm/memory/metaspace.cpp --- a/src/share/vm/memory/metaspace.cpp Tue Nov 18 12:33:47 2014 +0000 +++ b/src/share/vm/memory/metaspace.cpp Thu Nov 20 10:03:22 2014 -0500 @@ -1412,7 +1412,7 @@ size_t MetaspaceGC::capacity_until_GC() { size_t value = (size_t)OrderAccess::load_ptr_acquire(&_capacity_until_GC); - assert(value >= MetaspaceSize, "Not initialied properly?"); + assert(value >= MetaspaceSize, "Not initialized properly?"); return value; } From sangheon.kim at oracle.com Thu Nov 20 16:56:40 2014 From: sangheon.kim at oracle.com (Sangheon Kim) Date: Thu, 20 Nov 2014 08:56:40 -0800 Subject: RFR [1/2] 8065358: JDK-8065218 Move CMS-specific fields from Space to CompactibleFreeListSpace In-Reply-To: <546DAA4E.3080807@oracle.com> References: <546DAA4E.3080807@oracle.com> Message-ID: <546E1D48.3050507@oracle.com> Hi Mikael, Looks good. One question about a function/variable name. I know the original names are using upper/lower-cases but according to hotspot coding style, only lower-case are recommended for function/variable names. As it seems easy to change, how about changing them? Thanks, Sangheon On 11/20/2014 12:46 AM, Mikael Gerdin wrote: > Hi all, > > This is the first patch of two to fix JDK-8065358. > > In this change I suggest that we move a field used only by CMS to > CompactibleFreeListSpace and introduce a virtual getter in the Space > base class. > > I estimate that the performance impact of the virtual call will be > small since the getter path already does several virtual calls. > > Moving the field will allow me to add another field to G1's > G1OffsetTableContigSpace in patch [2/2] without introducing a > footprint regression. > > Webrev: http://cr.openjdk.java.net/~mgerdin/8065218/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8065218 > Parent bug: https://bugs.openjdk.java.net/browse/JDK-8065358 > Testing: JPRT > > Thanks > /Mikael From jesper.wilhelmsson at oracle.com Thu Nov 20 17:34:02 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 20 Nov 2014 18:34:02 +0100 Subject: RFR: 8065305 - Make it possible to extend the G1CollectorPolicy Message-ID: <546E260A.6060708@oracle.com> Hi, Please review this small change to allow an extended version of the G1CollectorPolicy. This change also includes a small cleanup to remove a couple of extension points in the argument processing. Bug: https://bugs.openjdk.java.net/browse/JDK-8065305 Webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/ Thanks! /Jesper From kim.barrett at oracle.com Thu Nov 20 18:27:14 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 20 Nov 2014 13:27:14 -0500 Subject: RFR: 8059066: CardTableModRefBS might commit the same page twice In-Reply-To: <20141117174322.GA23553@ehelin-desktop> References: <54450C6D.3080603@oracle.com> <986B14CB-1598-47D4-A581-4F82115A8257@oracle.com> <544A4C1D.9000908@oracle.com> <20141117174322.GA23553@ehelin-desktop> Message-ID: On Nov 17, 2014, at 12:43 PM, Erik Helin wrote: > > Because the start is aligned down and the end is aligned up, the two > committed regions will always share at least one page (since the start > for _committed[1] will be aligned down and the end for _committed[0] > will be aligned up). I?m just getting back to this today, and haven?t yet refreshed my understanding of the code. However, the claim that there will be at least one shared page, (with followup reasoning for exactly one shared page), doesn?t seem right to me. It seems to me there could be zero shared pages, since _committed[1] could (perhaps accidentally, if not by design) be page aligned. From kim.barrett at oracle.com Thu Nov 20 18:50:16 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 20 Nov 2014 13:50:16 -0500 Subject: RFR: 8059066: CardTableModRefBS might commit the same page twice In-Reply-To: References: <54450C6D.3080603@oracle.com> <986B14CB-1598-47D4-A581-4F82115A8257@oracle.com> <544A4C1D.9000908@oracle.com> <20141117174322.GA23553@ehelin-desktop> Message-ID: <08259340-A8E8-4EF9-8DA7-55FFCD702336@oracle.com> On Nov 20, 2014, at 1:27 PM, Kim Barrett wrote: > > On Nov 17, 2014, at 12:43 PM, Erik Helin wrote: >> >> Because the start is aligned down and the end is aligned up, the two >> committed regions will always share at least one page (since the start >> for _committed[1] will be aligned down and the end for _committed[0] >> will be aligned up). > > I?m just getting back to this today, and haven?t yet refreshed my understanding > of the code. However, the claim that there will be at least one shared page, > (with followup reasoning for exactly one shared page), doesn?t seem right to me. > It seems to me there could be zero shared pages, since _committed[1] could > (perhaps accidentally, if not by design) be page aligned. Hm, and it seems that committed regions are always page aligned. I?m clearly not understanding something yet. Back to reading code? From jon.masamitsu at oracle.com Thu Nov 20 21:32:36 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 20 Nov 2014 13:32:36 -0800 Subject: RFR: 8065227: Report allocation context stats at end of cleanup In-Reply-To: <546DB10A.5020403@oracle.com> References: <546CCCB2.9080601@oracle.com> <546D60CB.6020403@oracle.com> <546DB10A.5020403@oracle.com> Message-ID: <546E5DF4.1020607@oracle.com> On 11/20/2014 01:14 AM, Stefan Johansson wrote: > Hi Jon, > > Thanks for taking a look at this change. > > On 2014-11-20 04:32, Jon Masamitsu wrote: >> Stefan, >> >> Why did you chose to rename update_at_remark() to update_after_mark()? >> Instead of update_at_cleanup()? > Changing it to update_at_cleanup() was my first thought, but I think > update_after_mark() is more descriptive and a better fit when looking > how its implemented. Ok. Reviewed. Jon > > Thanks, > Stefan >> >> Jon >> >> On 11/19/2014 9:00 AM, Stefan Johansson wrote: >>> Hi, >>> >>> Please review this small fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8065227 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ >>> >>> Summary: >>> The update for the allocation context has previously been done in >>> the remark phase but this is a little to early. The last stw-phase >>> of the marking cycle is the cleanup and if doing the reporting there >>> that simplifies. >>> >>> Thanks, >>> Stefan >>> >>> >> > From kim.barrett at oracle.com Thu Nov 20 22:12:25 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 20 Nov 2014 17:12:25 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> <5469C9C1.7020306@oracle.com> <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> Message-ID: <1A268F67-0A0C-457A-B158-C9C3D9444EC6@oracle.com> On Nov 19, 2014, at 12:13 PM, Jungwoo Ha wrote: > > http://cr.openjdk.java.net/~rasbold/8061259/webrev.06/ > > PTAL. > > On Mon, Nov 17, 2014 at 7:57 AM, Kim Barrett wrote: > On Nov 17, 2014, at 5:11 AM, Bengt Rutisson wrote: > > > >> I think this does leave open the possibility of moving the > >> _promotion_failed flag check earlier in copy_to_survivor_space_XXX(), > >> [?] > >> > >> I'm not sure it would be overall beneficial though, [?] > > > > Yes, there is probably room for more improvement here. I think adding the check in the copy_to_survivor_space_XXX_undo() methods is a clear improvement in many cases. So, doing that as a first step is a good start. We can consider moving it earlier as a future enhancement. > > Agreed. Mostly looks good. A couple of nits: ------------------------------------------------------------------------------ src/share/vm/gc_implementation/parNew/parNewGeneration.cpp 1199 // We can skip par_promote if promotion has been failed already. "... promotion has been failed already." => "... promotion has already failed." There is similar awkward grammar in the comment on the other call, but see below. ------------------------------------------------------------------------------ src/share/vm/gc_implementation/parNew/parNewGeneration.cpp 1199 // We can skip par_promote if promotion has been failed already. 1200 // Checking _promotion_failed can be racy, but the worse case is going 1201 // through the slow path of calling par_promote, which will still correctly 1202 // handle the promotion failure. 1323 // Skip this step if promotion has already been failed. I dislike these two very different comments for the same thing. On the other hand, I dislike duplication of information. And a reference to the earlier comment from the later one can also become stale. Maybe add a (private) helper function that is a wrapper around the call to par_promote, with the flag test and associated comment in the wrapper? [Though this would become obsolete if we later decide we can move the flag test earlier.] Or just duplicate the more informative comment (ick!). Bengt, what do you think? From jwha at google.com Fri Nov 21 02:27:01 2014 From: jwha at google.com (Jungwoo Ha) Date: Thu, 20 Nov 2014 18:27:01 -0800 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <1A268F67-0A0C-457A-B158-C9C3D9444EC6@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> <5469C9C1.7020306@oracle.com> <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> <1A268F67-0A0C-457A-B158-C9C3D9444EC6@oracle.com> Message-ID: If we are to put a wrapper around, why not just go with the original change? I don't see adding a single field on a nearly singleton class is a bad thing. The changes are well wrapped inside par_promote. It is the fast path of the promotion, so I think it is more readable. We don't need a long explanation to the code reader why we are allowing the data-race. On Thu, Nov 20, 2014 at 2:12 PM, Kim Barrett wrote: > On Nov 19, 2014, at 12:13 PM, Jungwoo Ha wrote: > > > > http://cr.openjdk.java.net/~rasbold/8061259/webrev.06/ > > > > PTAL. > > > > On Mon, Nov 17, 2014 at 7:57 AM, Kim Barrett > wrote: > > On Nov 17, 2014, at 5:11 AM, Bengt Rutisson > wrote: > > > > > >> I think this does leave open the possibility of moving the > > >> _promotion_failed flag check earlier in copy_to_survivor_space_XXX(), > > >> [?] > > >> > > >> I'm not sure it would be overall beneficial though, [?] > > > > > > Yes, there is probably room for more improvement here. I think adding > the check in the copy_to_survivor_space_XXX_undo() methods is a clear > improvement in many cases. So, doing that as a first step is a good start. > We can consider moving it earlier as a future enhancement. > > > > Agreed. > > Mostly looks good. A couple of nits: > > > ------------------------------------------------------------------------------ > src/share/vm/gc_implementation/parNew/parNewGeneration.cpp > 1199 // We can skip par_promote if promotion has been failed already. > > "... promotion has been failed already." > => > "... promotion has already failed." > > There is similar awkward grammar in the comment on the other call, but > see below. > > > ------------------------------------------------------------------------------ > src/share/vm/gc_implementation/parNew/parNewGeneration.cpp > 1199 // We can skip par_promote if promotion has been failed already. > 1200 // Checking _promotion_failed can be racy, but the worse case is > going > 1201 // through the slow path of calling par_promote, which will still > correctly > 1202 // handle the promotion failure. > > 1323 // Skip this step if promotion has already been failed. > > I dislike these two very different comments for the same thing. On > the other hand, I dislike duplication of information. And a reference > to the earlier comment from the later one can also become stale. > > Maybe add a (private) helper function that is a wrapper around the > call to par_promote, with the flag test and associated comment in the > wrapper? [Though this would become obsolete if we later decide we can > move the flag test earlier.] Or just duplicate the more informative > comment (ick!). > > Bengt, what do you think? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sangheon.kim at oracle.com Fri Nov 21 05:35:01 2014 From: sangheon.kim at oracle.com (Sangheon Kim) Date: Thu, 20 Nov 2014 21:35:01 -0800 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. Message-ID: <546ECF05.8000504@oracle.com> Hi all, Please review this change to reset PromotionFailedInfo before use: https://bugs.openjdk.java.net/browse/JDK-8055239 Webrev: http://cr.openjdk.java.net/~sangheki/8055239/webrev.00/ Summary: This issue happens only with ParallelRefProcEnabled. And this assert happens as the current thread (when second promotion failure happened) is different from saved thread (when first promotion failure happened). From ParNewGeneration::collect(), there are two mt processing, for live object processing (by ParNewGenTask) and reference processing (by ParNewRefEnqueueTaskProxy). When first promotion failure happened from ParNewGenTask, thread pointer will be stored by PromotionFailedInfo::register_copy_failure(). And at second failure, assert happens as thread pointer is not guaranteed to be same. As the thread counts increase, its frequency become higher. Fix: Added ParScanThreadStateSet::trace_promotion_failed() to trace and reset promotion failure information instance. Test: - jprt Thanks, Sangheon From stefan.johansson at oracle.com Fri Nov 21 08:52:07 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 21 Nov 2014 09:52:07 +0100 Subject: RFR: 8065305 - Make it possible to extend the G1CollectorPolicy In-Reply-To: <546E260A.6060708@oracle.com> References: <546E260A.6060708@oracle.com> Message-ID: <546EFD37.1080704@oracle.com> Hi Jesper, Changes look good, Stefan On 2014-11-20 18:34, Jesper Wilhelmsson wrote: > Hi, > > Please review this small change to allow an extended version of the > G1CollectorPolicy. > This change also includes a small cleanup to remove a couple of > extension points in the argument processing. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8065305 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/ > > Thanks! > /Jesper > From thomas.schatzl at oracle.com Fri Nov 21 09:00:30 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 21 Nov 2014 10:00:30 +0100 Subject: RFR: 8065305 - Make it possible to extend the G1CollectorPolicy In-Reply-To: <546E260A.6060708@oracle.com> References: <546E260A.6060708@oracle.com> Message-ID: <1416560430.3340.2.camel@oracle.com> Hi, On Thu, 2014-11-20 at 18:34 +0100, Jesper Wilhelmsson wrote: > Hi, > > Please review this small change to allow an extended version of the > G1CollectorPolicy. > This change also includes a small cleanup to remove a couple of extension points > in the argument processing. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8065305 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/ Looks okay. Do not really like the "Ext" name, but I do not have a better name right now. Thanks, Thomas From kirk at kodewerk.com Fri Nov 21 11:45:04 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Fri, 21 Nov 2014 12:45:04 +0100 Subject: RFR: 8065305 - Make it possible to extend the G1CollectorPolicy In-Reply-To: <1416560430.3340.2.camel@oracle.com> References: <546E260A.6060708@oracle.com> <1416560430.3340.2.camel@oracle.com> Message-ID: <1D535851-6E7C-4C9F-B47E-2750FBCD8FC0@kodewerk.com> On Nov 21, 2014, at 10:00 AM, Thomas Schatzl wrote: > Hi, > > On Thu, 2014-11-20 at 18:34 +0100, Jesper Wilhelmsson wrote: >> Hi, >> >> Please review this small change to allow an extended version of the >> G1CollectorPolicy. >> This change also includes a small cleanup to remove a couple of extension points >> in the argument processing. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8065305 >> Webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/ > > Looks okay. Do not really like the "Ext" name, but I do not have a > better name right now. What is the extension suppose to do? Regards, Kirk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jesper.wilhelmsson at oracle.com Fri Nov 21 11:56:38 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 21 Nov 2014 12:56:38 +0100 Subject: RFR: 8065305 - Make it possible to extend the G1CollectorPolicy In-Reply-To: <1D535851-6E7C-4C9F-B47E-2750FBCD8FC0@kodewerk.com> References: <546E260A.6060708@oracle.com> <1416560430.3340.2.camel@oracle.com> <1D535851-6E7C-4C9F-B47E-2750FBCD8FC0@kodewerk.com> Message-ID: <546F2876.3030902@oracle.com> Kirk Pepperdine skrev 21/11/14 12:45: > On Nov 21, 2014, at 10:00 AM, Thomas Schatzl wrote: > >> Hi, >> >> On Thu, 2014-11-20 at 18:34 +0100, Jesper Wilhelmsson wrote: >>> Hi, >>> >>> Please review this small change to allow an extended version of the >>> G1CollectorPolicy. >>> This change also includes a small cleanup to remove a couple of extension points >>> in the argument processing. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065305 >>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/ >> Looks okay. Do not really like the "Ext" name, but I do not have a >> better name right now. > What is the extension suppose to do? It will allow people who would like to experiment/implement their own G1 heuristics to do so without having to patch and merge every time theres an update to the G1 code. By adding overloading methods in G1CollectorPolicyExt conflicts with changes in the "normal code" is avoided. I know that some companies use these extension points to build their own proprietary features on top of OpenJDK for instance. /Jesper > Regards, > Kirk > From kirk at kodewerk.com Fri Nov 21 12:48:20 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Fri, 21 Nov 2014 13:48:20 +0100 Subject: RFR: 8065305 - Make it possible to extend the G1CollectorPolicy In-Reply-To: <546F2876.3030902@oracle.com> References: <546E260A.6060708@oracle.com> <1416560430.3340.2.camel@oracle.com> <1D535851-6E7C-4C9F-B47E-2750FBCD8FC0@kodewerk.com> <546F2876.3030902@oracle.com> Message-ID: >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065305 >>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/ >>> Looks okay. Do not really like the "Ext" name, but I do not have a >>> better name right now. >> What is the extension suppose to do? > > It will allow people who would like to experiment/implement their own G1 heuristics to do so without having to patch and merge every time theres an update to the G1 code. By adding overloading methods in G1CollectorPolicyExt conflicts with changes in the "normal code" is avoided. Interesting? > > I know that some companies use these extension points to build their own proprietary features on top of OpenJDK for instance. So this is a G1CollectorPolicyExtensionPoint??? Regards, Kirk -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jesper.wilhelmsson at oracle.com Fri Nov 21 14:13:49 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 21 Nov 2014 15:13:49 +0100 Subject: RFR: 8065305 - Make it possible to extend the G1CollectorPolicy In-Reply-To: References: <546E260A.6060708@oracle.com> <1416560430.3340.2.camel@oracle.com> <1D535851-6E7C-4C9F-B47E-2750FBCD8FC0@kodewerk.com> <546F2876.3030902@oracle.com> Message-ID: <546F489D.2090206@oracle.com> Kirk Pepperdine skrev 21/11/14 13:48: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065305 >>>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/ >>>> Looks okay. Do not really like the "Ext" name, but I do not have a >>>> better name right now. >>> What is the extension suppose to do? >> It will allow people who would like to experiment/implement their own G1 heuristics to do so without having to patch and merge every time theres an update to the G1 code. By adding overloading methods in G1CollectorPolicyExt conflicts with changes in the "normal code" is avoided. > Interesting? > >> I know that some companies use these extension points to build their own proprietary features on top of OpenJDK for instance. > So this is a G1CollectorPolicyExtensionPoint??? Yes. /Jesper > > Regards, > Kirk > From kim.barrett at oracle.com Fri Nov 21 20:44:06 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Nov 2014 15:44:06 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> <5469C9C1.7020306@oracle.com> <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> <1A268F67-0A0C-457A-B158-C9C3D9444EC6@oracle.com> Message-ID: On Nov 20, 2014, at 9:27 PM, Jungwoo Ha wrote: > If we are to put a wrapper around, why not just go with the original change? > I don't see adding a single field on a nearly singleton class is a bad thing. > The changes are well wrapped inside par_promote. There are multiple implementations of par_promote, for different old-space collectors. (par_promote is a virtual function.) At least some of the others may have similar issues (for example, at a quick look, ParallelOld appears to have a similar locking structure), and it seems like all could benefit from this short-circuiting. [One of the copy_to_survivor_space_XXX functions (the callers of par_promote) is used exclusively when CMS is the old-space collector, the other is used for other old-space collectors.] > It is the fast path of the promotion, so I think it is more readable. > We don't need a long explanation to the code reader why we are allowing the data-race. Any intentional data race needs to have an explanation. From kim.barrett at oracle.com Sat Nov 22 20:45:42 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 22 Nov 2014 15:45:42 -0500 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: <546ECF05.8000504@oracle.com> References: <546ECF05.8000504@oracle.com> Message-ID: On Nov 21, 2014, at 12:35 AM, Sangheon Kim wrote: > > Please review this change to reset PromotionFailedInfo before use: > https://bugs.openjdk.java.net/browse/JDK-8055239 > > Webrev: > http://cr.openjdk.java.net/~sangheki/8055239/webrev.00/ > > Summary: > This issue happens only with ParallelRefProcEnabled. > And this assert happens as the current thread (when second promotion failure happened) is different from saved thread (when first promotion failure happened). > From ParNewGeneration::collect(), there are two mt processing, for live object processing (by ParNewGenTask) and reference processing (by ParNewRefEnqueueTaskProxy). When first promotion failure happened from ParNewGenTask, thread pointer will be stored by PromotionFailedInfo::register_copy_failure(). And at second failure, assert happens as thread pointer is not guaranteed to be same. > As the thread counts increase, its frequency become higher. > > Fix: > Added ParScanThreadStateSet::trace_promotion_failed() to trace and reset promotion failure information instance. > > Test: > - jprt > > Thanks, > Sangheon This proposed change doesn't address what I think is the root cause of the problem, which appears to me to be the semantically questionable use of a single ParScanThreadStateSet for what are conceptually two distinct sets of threads - the original generation scanning threads and the reference processing threads. (The same underlying OS threads are being used for both, but they are being run with completely different "tasks", and the ThreadState info is really per-task) Besides the unexpected reassociation of PromotionFailedInfo objects (from owning ParScanThreadState objects) with new OS threads, it looks like there are probably other things that are "messed up" by this reuse, e.g. timing information and TASKQUEUE_STATS. Creating a second state set for use by reference processing would require changing the signature of handle_promotion_failed to deal with two sets, but that function is private and only called in one place, so that shouldn't be hard. The real cost would be in the runtime construction of a second state set. I'm not sure whether that would be important or not. It is also only needed when the ParallelRefProcEnabled option is true, since that's the case where the re-use occurs, which adds a bit of complexity. One option might be to capture the information to be reported in a new object and reinitialize the ThreadState objects, capture additional information after the reuse (if needed), and pass both sets of captured information to handle_promotion_failed. An entirely different approach to addressing just the proximate failure would be to eliminate the _thread member from PromotionFailureInfo and deal with the consequences of that change. I can't see any utility in reporting the OS thread associated with an internal worker task when reporting promotion failure statistics. The relevant context is long gone, after all. And really, the ThreadState objects exist to avoid race conditions amongst the multiple threads. [I think what *should* be reported is a combined summary rather than per-thread promotion failure statistics.] Removing the _thread member has the added benefit of being a code removal and simplficiation change. The downside is that it affects a messaging API; I'm not sure what would be involved in dealing with that. The needed changes to the (open) jdk look straightforward: remove thread - src/share/vm/gc_implementation/shared/copyFailedInfo.hpp Remove _thread member and associated code from PromotionFailedInfo. - hotspot/src/share/vm/gc_implementation/share/gcTraceSend.cpp Remove thread field from constructed message. - hotspot/src/share/vm/trace/trace.xml Remove thread field from PromotionFailed event description. This is the API change that needs to be appropriately handled. From kim.barrett at oracle.com Mon Nov 24 03:08:17 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 23 Nov 2014 22:08:17 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> <5469C9C1.7020306@oracle.com> <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> <1A268F67-0A0C-457A-B158-C9C3D9444EC6@oracle.com> Message-ID: <737C76A3-6C09-4F12-9B34-FC52A89C82B4@oracle.com> On Nov 21, 2014, at 3:44 PM, Kim Barrett wrote: > > On Nov 20, 2014, at 9:27 PM, Jungwoo Ha wrote: >> If we are to put a wrapper around, why not just go with the original change? >> I don't see adding a single field on a nearly singleton class is a bad thing. >> The changes are well wrapped inside par_promote. > > There are multiple implementations of par_promote, for different > old-space collectors. (par_promote is a virtual function.) At least > some of the others may have similar issues (for example, at a quick > look, ParallelOld appears to have a similar locking structure), and it > seems like all could benefit from this short-circuiting. > > [One of the copy_to_survivor_space_XXX functions (the callers of > par_promote) is used exclusively when CMS is the old-space collector, > the other is used for other old-space collectors.] Except I?ve now been reminded that ParNew doesn?t get used with ParallelOld. In fact, several combinations of old/new collectors were deprecated in jdk8 and are slated to be removed in jdk9 (some already have), and it looks like ParNew will only remain in conjunction with CMS, making the split of ParNew?s copy_to_survivor_space into two variants no longer needed, and one of them redundant, making this wrapper vs comment duplication vs whatever issue moot. From sangheon.kim at oracle.com Mon Nov 24 05:47:03 2014 From: sangheon.kim at oracle.com (Sangheon Kim) Date: Sun, 23 Nov 2014 21:47:03 -0800 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: References: <546ECF05.8000504@oracle.com> Message-ID: <5472C657.2090501@oracle.com> Hi Kim, Thank you for reviewing this! And this detailed answer. On 11/22/2014 12:45 PM, Kim Barrett wrote: > On Nov 21, 2014, at 12:35 AM, Sangheon Kim wrote: >> Please review this change to reset PromotionFailedInfo before use: >> https://bugs.openjdk.java.net/browse/JDK-8055239 >> >> Webrev: >> http://cr.openjdk.java.net/~sangheki/8055239/webrev.00/ >> >> Summary: >> This issue happens only with ParallelRefProcEnabled. >> And this assert happens as the current thread (when second promotion failure happened) is different from saved thread (when first promotion failure happened). >> From ParNewGeneration::collect(), there are two mt processing, for live object processing (by ParNewGenTask) and reference processing (by ParNewRefEnqueueTaskProxy). When first promotion failure happened from ParNewGenTask, thread pointer will be stored by PromotionFailedInfo::register_copy_failure(). And at second failure, assert happens as thread pointer is not guaranteed to be same. >> As the thread counts increase, its frequency become higher. >> >> Fix: >> Added ParScanThreadStateSet::trace_promotion_failed() to trace and reset promotion failure information instance. >> >> Test: >> - jprt >> >> Thanks, >> Sangheon > This proposed change doesn't address what I think is the root cause of > the problem, which appears to me to be the semantically questionable > use of a single ParScanThreadStateSet for what are conceptually two > distinct sets of threads - the original generation scanning threads > and the reference processing threads. (The same underlying OS threads > are being used for both, but they are being run with completely > different "tasks", and the ThreadState info is really per-task) I think my proposal is same way as your second opinion. > > Besides the unexpected reassociation of PromotionFailedInfo objects > (from owning ParScanThreadState objects) with new OS threads, it looks > like there are probably other things that are "messed up" by this > reuse, e.g. timing information and TASKQUEUE_STATS. Thanks for catching up about TASKQUEUE_STATS. Reset is really needed. > > Creating a second state set for use by reference processing would > require changing the signature of handle_promotion_failed to deal with > two sets, but that function is private and only called in one place, > so that shouldn't be hard. I didn't think about creating second ThreadState as original implementation is already considered about reusing it. ParScanThreadStateSet::reset() is called before second use. > > The real cost would be in the runtime construction of a second state > set. I'm not sure whether that would be important or not. It is also > only needed when the ParallelRefProcEnabled option is true, since > that's the case where the re-use occurs, which adds a bit of > complexity. Agree if second ThreadState is needed. > > One option might be to capture the information to be reported in a new > object and reinitialize the ThreadState objects, capture additional > information after the reuse (if needed), and pass both sets of > captured information to handle_promotion_failed. Yes, this a little bit same as mine. The difference is that I wanted to report promotion failure before reuse not together. And I think ThreadState is already reset just above line of my change. (ParScanThreadStateSet::reset()) So I just added to reset/trace PromotionFailedInfo. Can you suggest some 'additional information'? It would be nice if add more information. > > An entirely different approach to addressing just the proximate > failure would be to eliminate the _thread member from > PromotionFailureInfo and deal with the consequences of that change. I > can't see any utility in reporting the OS thread associated with an > internal worker task when reporting promotion failure statistics. The > relevant context is long gone, after all. And really, the ThreadState > objects exist to avoid race conditions amongst the multiple > threads. [I think what *should* be reported is a combined summary > rather than per-thread promotion failure statistics.] > > Removing the _thread member has the added benefit of being a code > removal and simplficiation change. The downside is that it affects a > messaging API; I'm not sure what would be involved in dealing with > that. The needed changes to the (open) jdk look straightforward: > > remove thread > - src/share/vm/gc_implementation/shared/copyFailedInfo.hpp > Remove _thread member and associated code from PromotionFailedInfo. > > - hotspot/src/share/vm/gc_implementation/share/gcTraceSend.cpp > Remove thread field from constructed message. > > - hotspot/src/share/vm/trace/trace.xml > Remove thread field from PromotionFailed event description. > This is the API change that needs to be appropriately handled. Oh. I didn't think about this approach since I do believe there are some reason of saving thread id and deliver it by YoungGCTracer::report_promotion_failed. This issue is occurred because of thread id but it doesn't mean that this thread id information is not needed. Thanks, Sangheon From jesper.wilhelmsson at oracle.com Mon Nov 24 07:32:35 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 24 Nov 2014 08:32:35 +0100 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: <5472C657.2090501@oracle.com> References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> Message-ID: <5472DF13.4020800@oracle.com> Hi, I think the thread info can be useful in some cases, for instance to see if the same thread gets multiple promotion failures or if it is separate threads. If it is decided to remove this field please also consider how to treat it in G1 where it was supposed to be added. https://bugs.openjdk.java.net/browse/JDK-8012217 Also, consult Bengt before removing the field. I remember that he had opinions around the _threads field when I implemented this. /Jesper Sangheon Kim skrev 24/11/14 06:47: > Hi Kim, > > Thank you for reviewing this! > And this detailed answer. > > On 11/22/2014 12:45 PM, Kim Barrett wrote: >> On Nov 21, 2014, at 12:35 AM, Sangheon Kim wrote: >>> Please review this change to reset PromotionFailedInfo before use: >>> https://bugs.openjdk.java.net/browse/JDK-8055239 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sangheki/8055239/webrev.00/ >>> >>> Summary: >>> This issue happens only with ParallelRefProcEnabled. >>> And this assert happens as the current thread (when second promotion failure >>> happened) is different from saved thread (when first promotion failure >>> happened). >>> From ParNewGeneration::collect(), there are two mt processing, for live >>> object processing (by ParNewGenTask) and reference processing (by >>> ParNewRefEnqueueTaskProxy). When first promotion failure happened from >>> ParNewGenTask, thread pointer will be stored by >>> PromotionFailedInfo::register_copy_failure(). And at second failure, assert >>> happens as thread pointer is not guaranteed to be same. >>> As the thread counts increase, its frequency become higher. >>> >>> Fix: >>> Added ParScanThreadStateSet::trace_promotion_failed() to trace and reset >>> promotion failure information instance. >>> >>> Test: >>> - jprt >>> >>> Thanks, >>> Sangheon >> This proposed change doesn't address what I think is the root cause of >> the problem, which appears to me to be the semantically questionable >> use of a single ParScanThreadStateSet for what are conceptually two >> distinct sets of threads - the original generation scanning threads >> and the reference processing threads. (The same underlying OS threads >> are being used for both, but they are being run with completely >> different "tasks", and the ThreadState info is really per-task) > I think my proposal is same way as your second opinion. >> >> Besides the unexpected reassociation of PromotionFailedInfo objects >> (from owning ParScanThreadState objects) with new OS threads, it looks >> like there are probably other things that are "messed up" by this >> reuse, e.g. timing information and TASKQUEUE_STATS. > Thanks for catching up about TASKQUEUE_STATS. > Reset is really needed. >> >> Creating a second state set for use by reference processing would >> require changing the signature of handle_promotion_failed to deal with >> two sets, but that function is private and only called in one place, >> so that shouldn't be hard. > I didn't think about creating second ThreadState as original implementation is > already considered about reusing it. > ParScanThreadStateSet::reset() is called before second use. >> >> The real cost would be in the runtime construction of a second state >> set. I'm not sure whether that would be important or not. It is also >> only needed when the ParallelRefProcEnabled option is true, since >> that's the case where the re-use occurs, which adds a bit of >> complexity. > Agree if second ThreadState is needed. >> >> One option might be to capture the information to be reported in a new >> object and reinitialize the ThreadState objects, capture additional >> information after the reuse (if needed), and pass both sets of >> captured information to handle_promotion_failed. > Yes, this a little bit same as mine. > The difference is that I wanted to report promotion failure before reuse not > together. > And I think ThreadState is already reset just above line of my change. > (ParScanThreadStateSet::reset()) > So I just added to reset/trace PromotionFailedInfo. > > Can you suggest some 'additional information'? It would be nice if add more > information. > >> >> An entirely different approach to addressing just the proximate >> failure would be to eliminate the _thread member from >> PromotionFailureInfo and deal with the consequences of that change. I >> can't see any utility in reporting the OS thread associated with an >> internal worker task when reporting promotion failure statistics. The >> relevant context is long gone, after all. And really, the ThreadState >> objects exist to avoid race conditions amongst the multiple >> threads. [I think what *should* be reported is a combined summary >> rather than per-thread promotion failure statistics.] >> >> Removing the _thread member has the added benefit of being a code >> removal and simplficiation change. The downside is that it affects a >> messaging API; I'm not sure what would be involved in dealing with >> that. The needed changes to the (open) jdk look straightforward: >> >> remove thread >> - src/share/vm/gc_implementation/shared/copyFailedInfo.hpp >> Remove _thread member and associated code from PromotionFailedInfo. >> >> - hotspot/src/share/vm/gc_implementation/share/gcTraceSend.cpp >> Remove thread field from constructed message. >> >> - hotspot/src/share/vm/trace/trace.xml >> Remove thread field from PromotionFailed event description. >> This is the API change that needs to be appropriately handled. > Oh. I didn't think about this approach since I do believe there are some > reason of saving thread id and deliver it by > YoungGCTracer::report_promotion_failed. > This issue is occurred because of thread id but it doesn't mean that this > thread id information is not needed. > > Thanks, > Sangheon > From bengt.rutisson at oracle.com Mon Nov 24 09:10:23 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 24 Nov 2014 10:10:23 +0100 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: <5472DF13.4020800@oracle.com> References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> Message-ID: <5472F5FF.4040102@oracle.com> On 2014-11-24 08:32, Jesper Wilhelmsson wrote: > Hi, > > I think the thread info can be useful in some cases, for instance to > see if the same thread gets multiple promotion failures or if it is > separate threads. If it is decided to remove this field please also > consider how to treat it in G1 where it was supposed to be added. > https://bugs.openjdk.java.net/browse/JDK-8012217 > > Also, consult Bengt before removing the field. I remember that he had > opinions around the _threads field when I implemented this. Here's what I remember from that discussion. The promotion failed events are not sent at the time or by the thread where the promotion failure happens. Instead the information about the promotion failure is collected into a PromotionFailedInfo object for each thread (stored in ParScanThreadState). The main GC thread then iterates over all thread states and sends the actual events at the end of a GC. See ParScanThreadStateSet::trace_promotion_failed(). This means that the thread that sends the events is the main GC thread, which is not the same as the thread that experienced the promotion failure (which is one or several of the GC worker threads). So, removing the _thread instance variable can not be done without affecting how JFR works. I think it may be useful information to have the thread information available, but one alternative would be to just not report a thread with the promotion failure. A separate approach would be to send the promotion failed events from the GC worker threads rather than postponing the event to the end of the GC. That is also a slight change to the current JFR implementation but may be considered an improvement. However, my recollection is that we opted for the postponed solution because there were some issues with trying to send the events at the time when they occur. It may just have been that it was difficult to propagate the GC tracer to the right places. Hths, Bengt > /Jesper > > > Sangheon Kim skrev 24/11/14 06:47: >> Hi Kim, >> >> Thank you for reviewing this! >> And this detailed answer. >> >> On 11/22/2014 12:45 PM, Kim Barrett wrote: >>> On Nov 21, 2014, at 12:35 AM, Sangheon Kim >>> wrote: >>>> Please review this change to reset PromotionFailedInfo before use: >>>> https://bugs.openjdk.java.net/browse/JDK-8055239 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~sangheki/8055239/webrev.00/ >>>> >>>> Summary: >>>> This issue happens only with ParallelRefProcEnabled. >>>> And this assert happens as the current thread (when second >>>> promotion failure happened) is different from saved thread (when >>>> first promotion failure happened). >>>> From ParNewGeneration::collect(), there are two mt processing, for >>>> live object processing (by ParNewGenTask) and reference processing >>>> (by ParNewRefEnqueueTaskProxy). When first promotion failure >>>> happened from ParNewGenTask, thread pointer will be stored by >>>> PromotionFailedInfo::register_copy_failure(). And at second >>>> failure, assert happens as thread pointer is not guaranteed to be >>>> same. >>>> As the thread counts increase, its frequency become higher. >>>> >>>> Fix: >>>> Added ParScanThreadStateSet::trace_promotion_failed() to trace and >>>> reset promotion failure information instance. >>>> >>>> Test: >>>> - jprt >>>> >>>> Thanks, >>>> Sangheon >>> This proposed change doesn't address what I think is the root cause of >>> the problem, which appears to me to be the semantically questionable >>> use of a single ParScanThreadStateSet for what are conceptually two >>> distinct sets of threads - the original generation scanning threads >>> and the reference processing threads. (The same underlying OS threads >>> are being used for both, but they are being run with completely >>> different "tasks", and the ThreadState info is really per-task) >> I think my proposal is same way as your second opinion. >>> >>> Besides the unexpected reassociation of PromotionFailedInfo objects >>> (from owning ParScanThreadState objects) with new OS threads, it looks >>> like there are probably other things that are "messed up" by this >>> reuse, e.g. timing information and TASKQUEUE_STATS. >> Thanks for catching up about TASKQUEUE_STATS. >> Reset is really needed. >>> >>> Creating a second state set for use by reference processing would >>> require changing the signature of handle_promotion_failed to deal with >>> two sets, but that function is private and only called in one place, >>> so that shouldn't be hard. >> I didn't think about creating second ThreadState as original >> implementation is already considered about reusing it. >> ParScanThreadStateSet::reset() is called before second use. >>> >>> The real cost would be in the runtime construction of a second state >>> set. I'm not sure whether that would be important or not. It is also >>> only needed when the ParallelRefProcEnabled option is true, since >>> that's the case where the re-use occurs, which adds a bit of >>> complexity. >> Agree if second ThreadState is needed. >>> >>> One option might be to capture the information to be reported in a new >>> object and reinitialize the ThreadState objects, capture additional >>> information after the reuse (if needed), and pass both sets of >>> captured information to handle_promotion_failed. >> Yes, this a little bit same as mine. >> The difference is that I wanted to report promotion failure before >> reuse not together. >> And I think ThreadState is already reset just above line of my >> change. (ParScanThreadStateSet::reset()) >> So I just added to reset/trace PromotionFailedInfo. >> >> Can you suggest some 'additional information'? It would be nice if >> add more information. >> >>> >>> An entirely different approach to addressing just the proximate >>> failure would be to eliminate the _thread member from >>> PromotionFailureInfo and deal with the consequences of that change. I >>> can't see any utility in reporting the OS thread associated with an >>> internal worker task when reporting promotion failure statistics. The >>> relevant context is long gone, after all. And really, the ThreadState >>> objects exist to avoid race conditions amongst the multiple >>> threads. [I think what *should* be reported is a combined summary >>> rather than per-thread promotion failure statistics.] >>> >>> Removing the _thread member has the added benefit of being a code >>> removal and simplficiation change. The downside is that it affects a >>> messaging API; I'm not sure what would be involved in dealing with >>> that. The needed changes to the (open) jdk look straightforward: >>> >>> remove thread >>> - src/share/vm/gc_implementation/shared/copyFailedInfo.hpp >>> Remove _thread member and associated code from PromotionFailedInfo. >>> >>> - hotspot/src/share/vm/gc_implementation/share/gcTraceSend.cpp >>> Remove thread field from constructed message. >>> >>> - hotspot/src/share/vm/trace/trace.xml >>> Remove thread field from PromotionFailed event description. >>> This is the API change that needs to be appropriately handled. >> Oh. I didn't think about this approach since I do believe there are >> some reason of saving thread id and deliver it by >> YoungGCTracer::report_promotion_failed. >> This issue is occurred because of thread id but it doesn't mean that >> this thread id information is not needed. >> >> Thanks, >> Sangheon >> > From filipp.zhinkin at oracle.com Mon Nov 24 10:22:31 2014 From: filipp.zhinkin at oracle.com (Filipp Zhinkin) Date: Mon, 24 Nov 2014 14:22:31 +0400 Subject: [9] RFR (L): 8037968: Add tests on alignment of objects copied to survivor space In-Reply-To: <54510FAB.6050206@oracle.com> References: <5433B362.9090403@oracle.com> <544E4F78.9050401@oracle.com> <545110D8.3050302@oracle.com> <54510FAB.6050206@oracle.com> Message-ID: <547306E7.7020102@oracle.com> Hi all, I'm still looking for a second review. Everyone is welcome. ;) Regards, Filipp. On 10/29/2014 08:02 PM, Filipp Zhinkin wrote: > Hi Jon, > > thank you for looking at it! > > On 10/29/2014 08:07 PM, Jon Masamitsu wrote: >> Filipp, >> >> Sorry for the delay. These tests look good. >> >> Can you point me at the source for >> >> 59 CommandLineOptionTest.verifyJVMStartup( >> >> >> I just want to see what the messages are where there is >> a failure. > Sure: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/f7cb546710c8/test/testlibrary/com/oracle/java/testlibrary/cli/CommandLineOptionTest.java#l71 > > > Thanks, > Filipp. >> >> Thanks. >> >> Jon >> >> On 10/27/2014 6:58 AM, Filipp Zhinkin wrote: >>> friendly reminder >>> >>> On 10/07/2014 01:33 PM, Filipp Zhinkin wrote: >>>> Hi all, >>>> >>>> please review tests on survivor alignment feature (JDK-8031323 [1]). >>>> >>>> Webrev: http://cr.openjdk.java.net/~fzhinkin/8037968/webrev.00/ >>>> Bug id: https://bugs.openjdk.java.net/browse/JDK-8037968 >>>> Testing: manual, automated >>>> Description: >>>> >>>> There are 6 new tests: >>>> one verifies SurvivorAlignmentInBytes option processing >>>> and other 5 tests verify that that option only affect alignment >>>> of objects copied to survivor space and that their alignment is >>>> equal to SurvivorAlignmentInBytes value. >>>> >>>> Since there are no guarantee that _all_ objects copied to >>>> survivor space will be alignedthese tests are verifying that >>>> objects occupy some expected amount of memory depending on alignment. >>>> >>>> All tests use following algorithm: >>>> - decide how many objects should be allocated >>>> - allocate objects >>>> - force minor or full GCs if needed >>>> - verify usage of particular heap space (eden, survivor or tenured). >>>> >>>> There are several differences between Hotspot garbage collectors that affect >>>> memory usage measurements: >>>> 1) precision of such measurements; >>>> 2) min amount of heap space that will be occupied by a single object. >>>> >>>> The first one is about the G1 GC, where usage of eden and survivor regions >>>> is reported as (number_of_used_regions * G1HeapRegionSize) [2]. >>>> >>>> Tests verify that actual memory usage is close to expected usage, >>>> i.e. difference between actual and expected usage hasto be lower >>>> or equal to 5% of expected usage. >>>> However, for G1 GC max allowed difference has to be aligned up to >>>> G1HeapRegionSize. >>>> >>>> The second one is about the CMS GC, where an objecthas to occupy at least >>>> MinChunkSize bytes [3][4]. >>>> >>>> And, of course, amount of memory occupied by an object depends on the >>>> alignment >>>> used in particular heap space: SurvivorAlignmentInBytes for survivor space and >>>> ObjectsAlignmentInBytes for other spaces. >>>> >>>> To take all such aspects into account tests use AlignmentHelper class, >>>> whose instances are created for each heap space. >>>> >>>> That class is aimed to answer following questions: >>>> - how much space will be occupied by an object whose size is M; >>>> - how many objects of size M should be allocated to occupy N bytes >>>> of memory in particular heap space; >>>> - how much space will be occupied by K objects whose size is M; >>>> - how close the actual memory usage has to be to the expected memory usage >>>> given the precision with which we can measure the memory usage. >>>> >>>> In order to avoid false failures caused by memory allocated by different >>>> threads >>>> (for example, some agent was attached to tested JVM and allocated objects >>>> during >>>> the test run) tests check that only the main test thread was allocating >>>> objects. >>>> >>>> Thanks, >>>> Filipp. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8031323 >>>> [2] >>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp#l124 >>>> [3] >>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp#l1427 >>>> [4] >>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.hpp#l477 >>>> >>> >> > From bengt.rutisson at oracle.com Mon Nov 24 11:55:12 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 24 Nov 2014 12:55:12 +0100 Subject: RFR (smallish?) 8061611: Remove deprecated command line flags In-Reply-To: <546B7683.9000803@oracle.com> References: <546B7683.9000803@oracle.com> Message-ID: <54731CA0.4030302@oracle.com> Hi Derek, Sorry for taking so long to review this. On 2014-11-18 17:40, Derek White wrote: > Hi Team, > > First review request. Please let me know if I've forgotten something > or have gone completely off the rails. > > The main point of this bug is to remove deprecated -XX options which > are alias for other options. > > The only complicated part is that one case, > /CMSParPromoteBlocksToClaim/ was not a true alias for /OldPLABSize/ > but a parallel option with different defaults that were synchronized > in ergo processing. This fix removes the /CMSParPromoteBlocksToClaim > /variable but preserves using different defaults in the CMS case. > > Also in this fix I added warning messages to several remaining > undocumented command line options aliases. This should ease removal of > these options in the future > CMSMarkStackSize ==> MarkStackSize (MarkStackSize not documented either, but came in jdk6) > G1MarkStackSize ==> MarkStackSize > CMSMarkStackSizeMax ==> MarkStackSizeMax (MarkStackSizeMax not documented either) > ParallelMarkingThreads => ConcGCThreads (ConcGCThreads came in jdk6) > ParallelCMSThreads ==> ConcGCThread > > Thanks, > - Derek > > *Webrev*: http://cr.openjdk.java.net/~drwhite/8061611/webrev.00/ > > *Bug*: https://bugs.openjdk.java.net/browse/JDK-8061611 Looks good to me. One minor nit regarding the naming of the new constants that you introduce: 682 static const int defaultDynamicOldPLABSize = 16; 683 static const int defaultStaticOldPLABSize = 50; HotSpot does not have a common guideline for how to name constants. There are at least three widely used naming standards: - All caps: DEFAULT_DYNAMIC_OLD_PLAB_SIZE - Like an instance variable: _default_dynamic_old_plab_size - Like a flag: DefaultDynamicOldPLABSize I think it is hard to say which one is "standard" but I would prefer if we don't introduce yet another naming standard for constants. Do you mind changing to one of these? Personally I prefer the last one (the "Like a flag" version). Bengt > > *Testing*: > jprt: > Saw 1-2 intermittent failures that went away on retesting - hangs > and timeouts. > > refworkload: > no differences > > jtreg: Saw a few unexplained results. Not sure if typical or not: > > > Execution failed: `main' threw exception: > java.lang.Exception: jmap -heap exited with error code: 1 > > * gc/metaspace/CompressedClassSpaceSizeInJmapHeap.java > : > Checks that jmap -heap contains the flag CompressedClassSpaceSize > > > Program > `/export/users/drwhite/test-builds/j2sdk-image.11.17.2014/bin/java' > interrupted! (timed out?) > > * closed/runtime/AppCDS/SharedArchiveConsistency.java > : > SharedArchiveConsistency > > Plus a bunch of tests that are labelled "ignored". > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.johansson at oracle.com Mon Nov 24 12:23:31 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 24 Nov 2014 13:23:31 +0100 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced In-Reply-To: <546E0530.4060904@oracle.com> References: <546E0530.4060904@oracle.com> Message-ID: <54732343.6000307@oracle.com> Hi Leonid, I think the changes for making it possible to trigger a marking cycle through the WB API looks good. There are however some problems with the test. --- 24 assertTrue(wb.g1StartConcMarkCycle()); 25 wb.fullGC(); 26 assertTrue(wb.g1StartConcMarkCycle()); Here the test assumes that it should be able to trigger a new marking cycle since the Full-GC have aborted the previous one, but it is possible that the concurrent mark thread hasn't yet finished the cycle even if it has been aborted. That is, during cycle will return true for some time after the concurrent mark has been aborted. --- 28 while (wb.g1InConcurrentMark()) { 29 assertFalse(wb.g1StartConcMarkCycle()); 30 } 31 32 assertTrue(wb.g1StartConcMarkCycle()); 33 34 while (wb.g1InConcurrentMark()) { 35 assertFalse(wb.g1StartConcMarkCycle()); 36 } Here the problem is that we can't guarantee that the concurrent cycle won't finish just before the call to wb.g1StartConcMarkCycle(), and therefor the assert might fail. --- I'm not sure how you want to improve the test to avoid these problems, the only really robust way I think is to only use the g1InConcurrentMark() call to wait until marking is finished and then assert that we can trigger a new cycle. Thanks, Stefan On 2014-11-20 16:13, Leonid Mesnik wrote: > Hi > > Here is the fix which introduce WB method which stars concurrent mark > cycle explicitly. It is needed to make CM cycle related tests more stable. > > Changeset also includes test and very small fix of WB method > g1InConcurrentMark . > Fix pass JPRT with hotspot testset. > > JIRA: https://bugs.openjdk.java.net/browse/JDK-8065579 > Webrev: http://cr.openjdk.java.net/~sjohanss/lmesnik/8065579/hotspot.00/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Mon Nov 24 13:13:46 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Mon, 24 Nov 2014 17:13:46 +0400 Subject: RFR: 8065749: [TESTBUG]: gc/arguments/TestG1HeapRegionSize.java fails at nightly In-Reply-To: <54732905.3070804@oracle.com> References: <54732905.3070804@oracle.com> Message-ID: <54732F0A.3020806@oracle.com> Hi, Could you please review changes for 8065749 ? Test TestG1HeapRegionSize.java is G1-specific and could not work without -XX:+UseG1GC passed directly. Option added to the run. bug: https://bugs.openjdk.java.net/browse/JDK-8065749 fix: http://cr.openjdk.java.net/~eistepan/8065749/webrev.00/ Thanks, Evgeniya Stepanova -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Mon Nov 24 12:29:31 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Mon, 24 Nov 2014 16:29:31 +0400 Subject: [9] RFR (L): 8037968: Add tests on alignment of objects copied to survivor space In-Reply-To: <547306E7.7020102@oracle.com> References: <5433B362.9090403@oracle.com> <544E4F78.9050401@oracle.com> <545110D8.3050302@oracle.com> <54510FAB.6050206@oracle.com> <547306E7.7020102@oracle.com> Message-ID: <547324AB.1050608@oracle.com> Hi Filipp, The new version looks good to me. Thanks Dima On 24.11.2014 14:22, Filipp Zhinkin wrote: > Hi all, > > I'm still looking for a second review. > Everyone is welcome. ;) > > Regards, > Filipp. > > On 10/29/2014 08:02 PM, Filipp Zhinkin wrote: >> Hi Jon, >> >> thank you for looking at it! >> >> On 10/29/2014 08:07 PM, Jon Masamitsu wrote: >>> Filipp, >>> >>> Sorry for the delay. These tests look good. >>> >>> Can you point me at the source for >>> >>> 59 CommandLineOptionTest.verifyJVMStartup( >>> >>> >>> I just want to see what the messages are where there is >>> a failure. >> Sure: >> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/f7cb546710c8/test/testlibrary/com/oracle/java/testlibrary/cli/CommandLineOptionTest.java#l71 >> >> >> Thanks, >> Filipp. >>> >>> Thanks. >>> >>> Jon >>> >>> On 10/27/2014 6:58 AM, Filipp Zhinkin wrote: >>>> friendly reminder >>>> >>>> On 10/07/2014 01:33 PM, Filipp Zhinkin wrote: >>>>> Hi all, >>>>> >>>>> please review tests on survivor alignment feature (JDK-8031323 [1]). >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~fzhinkin/8037968/webrev.00/ >>>>> Bug id: https://bugs.openjdk.java.net/browse/JDK-8037968 >>>>> Testing: manual, automated >>>>> Description: >>>>> >>>>> There are 6 new tests: >>>>> one verifies SurvivorAlignmentInBytes option processing >>>>> and other 5 tests verify that that option only affect alignment >>>>> of objects copied to survivor space and that their alignment is >>>>> equal to SurvivorAlignmentInBytes value. >>>>> >>>>> Since there are no guarantee that _all_ objects copied to >>>>> survivor space will be alignedthese tests are verifying that >>>>> objects occupy some expected amount of memory depending on alignment. >>>>> >>>>> All tests use following algorithm: >>>>> - decide how many objects should be allocated >>>>> - allocate objects >>>>> - force minor or full GCs if needed >>>>> - verify usage of particular heap space (eden, survivor or tenured). >>>>> >>>>> There are several differences between Hotspot garbage collectors >>>>> that affect >>>>> memory usage measurements: >>>>> 1) precision of such measurements; >>>>> 2) min amount of heap space that will be occupied by a single object. >>>>> >>>>> The first one is about the G1 GC, where usage of eden and survivor >>>>> regions >>>>> is reported as (number_of_used_regions * G1HeapRegionSize) [2]. >>>>> >>>>> Tests verify that actual memory usage is close to expected usage, >>>>> i.e. difference between actual and expected usage hasto be lower >>>>> or equal to 5% of expected usage. >>>>> However, for G1 GC max allowed difference has to be aligned up to >>>>> G1HeapRegionSize. >>>>> >>>>> The second one is about the CMS GC, where an objecthas to occupy >>>>> at least >>>>> MinChunkSize bytes [3][4]. >>>>> >>>>> And, of course, amount of memory occupied by an object depends on >>>>> the alignment >>>>> used in particular heap space: SurvivorAlignmentInBytes for >>>>> survivor space and >>>>> ObjectsAlignmentInBytes for other spaces. >>>>> >>>>> To take all such aspects into account tests use AlignmentHelper >>>>> class, >>>>> whose instances are created for each heap space. >>>>> >>>>> That class is aimed to answer following questions: >>>>> - how much space will be occupied by an object whose size is M; >>>>> - how many objects of size M should be allocated to occupy N bytes >>>>> of memory in particular heap space; >>>>> - how much space will be occupied by K objects whose size is M; >>>>> - how close the actual memory usage has to be to the expected >>>>> memory usage >>>>> given the precision with which we can measure the memory usage. >>>>> >>>>> In order to avoid false failures caused by memory allocated by >>>>> different threads >>>>> (for example, some agent was attached to tested JVM and allocated >>>>> objects during >>>>> the test run) tests check that only the main test thread was >>>>> allocating objects. >>>>> >>>>> Thanks, >>>>> Filipp. >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8031323 >>>>> [2] >>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp#l124 >>>>> [3] >>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp#l1427 >>>>> [4] >>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.hpp#l477 >>>>> >>>> >>> >> > From jesper.wilhelmsson at oracle.com Mon Nov 24 13:49:41 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 24 Nov 2014 14:49:41 +0100 Subject: RFR: 8065749: [TESTBUG]: gc/arguments/TestG1HeapRegionSize.java fails at nightly In-Reply-To: <54732F0A.3020806@oracle.com> References: <54732905.3070804@oracle.com> <54732F0A.3020806@oracle.com> Message-ID: <54733775.7040109@oracle.com> Looks good! /Jesper Evgeniya Stepanova skrev 24/11/14 14:13: > Hi, > > Could you please review changes for 8065749 ? > Test TestG1HeapRegionSize.java is G1-specific and could not work without > -XX:+UseG1GC passed directly. > Option added to the run. > bug: https://bugs.openjdk.java.net/browse/JDK-8065749 > fix: http://cr.openjdk.java.net/~eistepan/8065749/webrev.00/ > > Thanks, > Evgeniya Stepanova > > -- > /Evgeniya Stepanova/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Mon Nov 24 14:33:12 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 24 Nov 2014 15:33:12 +0100 Subject: RFR: 8065749: [TESTBUG]: gc/arguments/TestG1HeapRegionSize.java fails at nightly In-Reply-To: <54732F0A.3020806@oracle.com> References: <54732905.3070804@oracle.com> <54732F0A.3020806@oracle.com> Message-ID: <547341A8.7060108@oracle.com> Hi Evgeniya, On 2014-11-24 14:13, Evgeniya Stepanova wrote: > Hi, > > Could you please review changes for 8065749 ? > Test TestG1HeapRegionSize.java is G1-specific and could not work > without -XX:+UseG1GC passed directly. > Option added to the run. > bug: https://bugs.openjdk.java.net/browse/JDK-8065749 > fix: http://cr.openjdk.java.net/~eistepan/8065749/webrev.00/ I think your suggested fix will work, but I have a question regarding the original change. Before the original change the test explicitly checked that it was run with G1 and just skipped otherwise: http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/comparison/6464714dd742/test/gc/arguments/TestG1HeapRegionSize.java Was this test really failing at all? I thought we were only going to add @requires to tests that were failing due to conflicting GC combinations. If it was working before I think a different approach to fix the current problem would be to revert back to the old code without @requires. Thanks, Bengt > > Thanks, > Evgeniya Stepanova > > -- > /Evgeniya Stepanova/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Mon Nov 24 14:03:23 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Mon, 24 Nov 2014 18:03:23 +0400 Subject: RFR: 8065749: [TESTBUG]: gc/arguments/TestG1HeapRegionSize.java fails at nightly In-Reply-To: <547341A8.7060108@oracle.com> References: <54732905.3070804@oracle.com> <54732F0A.3020806@oracle.com> <547341A8.7060108@oracle.com> Message-ID: <54733AAB.5020005@oracle.com> Hi Bengt, On 24.11.2014 18:33, Bengt Rutisson wrote: > > Hi Evgeniya, > > On 2014-11-24 14:13, Evgeniya Stepanova wrote: >> Hi, >> >> Could you please review changes for 8065749 ? >> Test TestG1HeapRegionSize.java is G1-specific and could not work >> without -XX:+UseG1GC passed directly. >> Option added to the run. >> bug: https://bugs.openjdk.java.net/browse/JDK-8065749 >> fix: http://cr.openjdk.java.net/~eistepan/8065749/webrev.00/ > > I think your suggested fix will work, but I have a question regarding > the original change. > > Before the original change the test explicitly checked that it was run > with G1 and just skipped otherwise: > > http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/comparison/6464714dd742/test/gc/arguments/TestG1HeapRegionSize.java > > Was this test really failing at all? I thought we were only going to > add @requires to tests that were failing due to conflicting GC > combinations. If it was working before I think a different approach to > fix the current problem would be to revert back to the old code > without @requires. You are right, it was our mistake. This test wasn't reverted when we decided to update only failing tests. Yes, a possible solution would be just to roll back the previous change, but I don't think it makes sense just to roll back. What I suggest: 1) keep this test G1 specific (@requires g1 and +UseG1 explicitly) - what is proposed by this change 2) make tests GC independent - remove @requires and check in tests that RegionSize == in case of G1, and RegionSize == 0 in case of another GC. I don't think that original variant (silently pass if not g1) is the right solution. What do you think? -- Dima > > Thanks, > Bengt > >> >> Thanks, >> Evgeniya Stepanova >> >> -- >> /Evgeniya Stepanova/ >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Nov 24 15:13:31 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 24 Nov 2014 10:13:31 -0500 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: <737C76A3-6C09-4F12-9B34-FC52A89C82B4@oracle.com> References: <544E0C5F.6030505@oracle.com> <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> <5469C9C1.7020306@oracle.com> <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> <1A268F67-0A0C-457A-B158-C9C3D9444EC6@oracle.com> <737C76A3-6C09-4F12-9B34-FC52A89C! 82B4@oracle.com> Message-ID: On Nov 23, 2014, at 10:08 PM, Kim Barrett wrote: > > On Nov 21, 2014, at 3:44 PM, Kim Barrett wrote: >> >> On Nov 20, 2014, at 9:27 PM, Jungwoo Ha wrote: >>> If we are to put a wrapper around, why not just go with the original change? >>> I don't see adding a single field on a nearly singleton class is a bad thing. >>> The changes are well wrapped inside par_promote. >> >> There are multiple implementations of par_promote, for different >> old-space collectors. (par_promote is a virtual function.) At least >> some of the others may have similar issues (for example, at a quick >> look, ParallelOld appears to have a similar locking structure), and it >> seems like all could benefit from this short-circuiting. >> >> [One of the copy_to_survivor_space_XXX functions (the callers of >> par_promote) is used exclusively when CMS is the old-space collector, >> the other is used for other old-space collectors.] > > Except I?ve now been reminded that ParNew doesn?t get used with ParallelOld. > In fact, several combinations of old/new collectors were deprecated in jdk8 and > are slated to be removed in jdk9 (some already have), and it looks like ParNew > will only remain in conjunction with CMS, making the split of ParNew?s > copy_to_survivor_space into two variants no longer needed, and one of them > redundant, making this wrapper vs comment duplication vs whatever issue moot. To be clear, what I'm suggesting is that, in light of the deprecation and expected removal of some combinations with ParNew, only copy_to_survivor_space_avoiding_promotion_undo() really needs the proposed change, and copy_to_survivor_space_with_undo() could be left unmodified. From derek.white at oracle.com Mon Nov 24 16:10:16 2014 From: derek.white at oracle.com (Derek White) Date: Mon, 24 Nov 2014 11:10:16 -0500 Subject: RFR (smallish?) 8061611: Remove deprecated command line flags In-Reply-To: <54731CA0.4030302@oracle.com> References: <546B7683.9000803@oracle.com> <54731CA0.4030302@oracle.com> Message-ID: <54735868.5090003@oracle.com> Thanks Bengt, I'll make the spelling change you suggested. One other question I had is does this require a CCC decision to actually change the command line arguments? Some of the arguments have been discussed in public, so are not truly "internal". Thanks! - Derek On 11/24/14 6:55 AM, Bengt Rutisson wrote: > > Hi Derek, > > Sorry for taking so long to review this. > > On 2014-11-18 17:40, Derek White wrote: >> Hi Team, >> >> First review request. Please let me know if I've forgotten something >> or have gone completely off the rails. >> >> The main point of this bug is to remove deprecated -XX options which >> are alias for other options. >> >> The only complicated part is that one case, >> /CMSParPromoteBlocksToClaim/ was not a true alias for /OldPLABSize/ >> but a parallel option with different defaults that were synchronized >> in ergo processing. This fix removes the /CMSParPromoteBlocksToClaim >> /variable but preserves using different defaults in the CMS case. >> >> Also in this fix I added warning messages to several remaining >> undocumented command line options aliases. This should ease removal >> of these options in the future >> CMSMarkStackSize ==> MarkStackSize (MarkStackSize not documented either, but came in jdk6) >> G1MarkStackSize ==> MarkStackSize >> CMSMarkStackSizeMax ==> MarkStackSizeMax (MarkStackSizeMax not documented either) >> ParallelMarkingThreads => ConcGCThreads (ConcGCThreads came in jdk6) >> ParallelCMSThreads ==> ConcGCThread >> >> Thanks, >> - Derek >> >> *Webrev*: http://cr.openjdk.java.net/~drwhite/8061611/webrev.00/ >> >> *Bug*: https://bugs.openjdk.java.net/browse/JDK-8061611 > > Looks good to me. One minor nit regarding the naming of the new > constants that you introduce: > > 682 static const int defaultDynamicOldPLABSize = 16; > 683 static const int defaultStaticOldPLABSize = 50; > > HotSpot does not have a common guideline for how to name constants. > There are at least three widely used naming standards: > > - All caps: DEFAULT_DYNAMIC_OLD_PLAB_SIZE > - Like an instance variable: _default_dynamic_old_plab_size > - Like a flag: DefaultDynamicOldPLABSize > > I think it is hard to say which one is "standard" but I would prefer > if we don't introduce yet another naming standard for constants. Do > you mind changing to one of these? Personally I prefer the last one > (the "Like a flag" version). > > Bengt > >> >> *Testing*: >> jprt: >> Saw 1-2 intermittent failures that went away on retesting - hangs >> and timeouts. >> >> refworkload: >> no differences >> >> jtreg: Saw a few unexplained results. Not sure if typical or not: >> >> >> Execution failed: `main' threw exception: >> java.lang.Exception: jmap -heap exited with error code: 1 >> >> * gc/metaspace/CompressedClassSpaceSizeInJmapHeap.java >> : >> Checks that jmap -heap contains the flag >> CompressedClassSpaceSize >> >> >> Program >> `/export/users/drwhite/test-builds/j2sdk-image.11.17.2014/bin/java' >> interrupted! (timed out?) >> >> * closed/runtime/AppCDS/SharedArchiveConsistency.java >> : >> SharedArchiveConsistency >> >> Plus a bunch of tests that are labelled "ignored". >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.helin at oracle.com Mon Nov 24 16:09:27 2014 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 24 Nov 2014 17:09:27 +0100 Subject: RFR: 8059066: CardTableModRefBS might commit the same page twice In-Reply-To: <08259340-A8E8-4EF9-8DA7-55FFCD702336@oracle.com> References: <54450C6D.3080603@oracle.com> <986B14CB-1598-47D4-A581-4F82115A8257@oracle.com> <544A4C1D.9000908@oracle.com> <20141117174322.GA23553@ehelin-desktop> <08259340-A8E8-4EF9-8DA7-55FFCD702336@oracle.com> Message-ID: <20141124160926.GD23108@ehelin-desktop> On 2014-11-20, Kim Barrett wrote: > On Nov 20, 2014, at 1:27 PM, Kim Barrett wrote: > > > > On Nov 17, 2014, at 12:43 PM, Erik Helin wrote: > >> > >> Because the start is aligned down and the end is aligned up, the two > >> committed regions will always share at least one page (since the start > >> for _committed[1] will be aligned down and the end for _committed[0] > >> will be aligned up). > > > > I?m just getting back to this today, and haven?t yet refreshed my understanding > > of the code. However, the claim that there will be at least one shared page, > > (with followup reasoning for exactly one shared page), doesn?t seem right to me. > > It seems to me there could be zero shared pages, since _committed[1] could > > (perhaps accidentally, if not by design) be page aligned. > > Hm, and it seems that committed regions are always page aligned. I?m clearly not > understanding something yet. Back to reading code? Me and Kim discussed this offline and we both now agree that the existing patch does work, but the check if the first _committed region expands into the second _committed region can be simpler. The existing check in the patch is: if (new_end_aligned > _committed[ri].start() && new_end_aligned <= _committed[ri].end()) { The check if new_end_aligned <= _committed[ri].end() is not necessary because this must always be the case. This is because _committed[0] can't expand further than the original, unaligned start of _committed[1] and _committed[1] will always commit at least one page in the byte array for the card table (starting at _committed[ri].start()). Therefore, it is enough to check: if (new_end_aligned > _committed[ri].start()) { I updated the patch to use this simple check and also added an assert to show that new_end_aligned <= _committed[ri].end() always is true. Please see new webrevs at: - full: http://cr.openjdk.java.net/~ehelin/8059066/webrev.01/index.html - inc: http://cr.openjdk.java.net/~ehelin/8059066/webrev.00-01/index.html Thanks, Erik From jesper.wilhelmsson at oracle.com Mon Nov 24 16:37:13 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 24 Nov 2014 17:37:13 +0100 Subject: RFR (smallish?) 8061611: Remove deprecated command line flags In-Reply-To: <54735868.5090003@oracle.com> References: <546B7683.9000803@oracle.com> <54731CA0.4030302@oracle.com> <54735868.5090003@oracle.com> Message-ID: <54735EB9.6080006@oracle.com> Hi Derek, Yes, any change to a publicly available API requires a CCC. Let me know if you need any help with the CCC process. /Jesper Derek White skrev 24/11/14 17:10: > Thanks Bengt, > > I'll make the spelling change you suggested. > > One other question I had is does this require a CCC decision to actually > change the command line arguments? Some of the arguments have been discussed > in public, so are not truly "internal". > > Thanks! > > - Derek > > On 11/24/14 6:55 AM, Bengt Rutisson wrote: >> >> Hi Derek, >> >> Sorry for taking so long to review this. >> >> On 2014-11-18 17:40, Derek White wrote: >>> Hi Team, >>> >>> First review request. Please let me know if I've forgotten something or have >>> gone completely off the rails. >>> >>> The main point of this bug is to remove deprecated -XX options which are >>> alias for other options. >>> >>> The only complicated part is that one case, /CMSParPromoteBlocksToClaim/ was >>> not a true alias for /OldPLABSize/ but a parallel option with different >>> defaults that were synchronized in ergo processing. This fix removes the >>> /CMSParPromoteBlocksToClaim /variable but preserves using different defaults >>> in the CMS case. >>> >>> Also in this fix I added warning messages to several remaining undocumented >>> command line options aliases. This should ease removal of these options in >>> the future >>> CMSMarkStackSize ==> MarkStackSize (MarkStackSize not documented either, but came in jdk6) >>> G1MarkStackSize ==> MarkStackSize >>> CMSMarkStackSizeMax ==> MarkStackSizeMax (MarkStackSizeMax not documented either) >>> ParallelMarkingThreads => ConcGCThreads (ConcGCThreads came in jdk6) >>> ParallelCMSThreads ==> ConcGCThread >>> >>> Thanks, >>> - Derek >>> >>> *Webrev*: http://cr.openjdk.java.net/~drwhite/8061611/webrev.00/ >>> >>> *Bug*: https://bugs.openjdk.java.net/browse/JDK-8061611 >> >> Looks good to me. One minor nit regarding the naming of the new constants >> that you introduce: >> >> 682 static const int defaultDynamicOldPLABSize = 16; >> 683 static const int defaultStaticOldPLABSize = 50; >> >> HotSpot does not have a common guideline for how to name constants. There are >> at least three widely used naming standards: >> >> - All caps: DEFAULT_DYNAMIC_OLD_PLAB_SIZE >> - Like an instance variable: _default_dynamic_old_plab_size >> - Like a flag: DefaultDynamicOldPLABSize >> >> I think it is hard to say which one is "standard" but I would prefer if we >> don't introduce yet another naming standard for constants. Do you mind >> changing to one of these? Personally I prefer the last one (the "Like a flag" >> version). >> >> Bengt >> >>> >>> *Testing*: >>> jprt: >>> Saw 1-2 intermittent failures that went away on retesting - hangs and >>> timeouts. >>> >>> refworkload: >>> no differences >>> >>> jtreg: Saw a few unexplained results. Not sure if typical or not: >>> >>> >>> Execution failed: `main' threw exception: java.lang.Exception: >>> jmap -heap exited with error code: 1 >>> >>> * gc/metaspace/CompressedClassSpaceSizeInJmapHeap.java >>> : >>> Checks that jmap -heap contains the flag CompressedClassSpaceSize >>> >>> >>> Program >>> `/export/users/drwhite/test-builds/j2sdk-image.11.17.2014/bin/java' >>> interrupted! (timed out?) >>> >>> * closed/runtime/AppCDS/SharedArchiveConsistency.java >>> : >>> SharedArchiveConsistency >>> >>> Plus a bunch of tests that are labelled "ignored". >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Nov 24 18:06:06 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 24 Nov 2014 13:06:06 -0500 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: <5472F5FF.4040102@oracle.com> References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> Message-ID: On Nov 24, 2014, at 4:10 AM, Bengt Rutisson wrote: > The promotion failed events are not sent at the time or by the thread > where the promotion failure happens. Instead the information about the > promotion failure is collected into a PromotionFailedInfo object for > each thread (stored in ParScanThreadState). The main GC thread then > iterates over all thread states and sends the actual events at the end > of a GC. See ParScanThreadStateSet::trace_promotion_failed(). Yes. My working hypothesis is that (1) Reporting is being deferred because it is deemed too expensive to report the individual occurrences as they happen. [Promotion failures are presumably supposed to be relatively rare, but once they happen at all in a given collection, it seems like there's a pretty good chance of (possibly many) more. That would be especially true with the fix being discussed for JDK-8061259.] (2) The PromotionFailedInfo objects are per-thread to avoid update races without requiring atomic counters or locks. > This means that the thread that sends the events is the main GC > thread, which is not the same as the thread that experienced the > promotion failure (which is one or several of the GC worker threads). > > So, removing the _thread instance variable can not be done without > affecting how JFR works. I think it may be useful information to have > the thread information available, but one alternative would be to just > not report a thread with the promotion failure. Recall that it's not "the" promotion failure that is being reported. It is a per-thread summary of promotion failures. There may be many promotion failures being summarized. I'm having a very hard time coming up with a way to make use of the association of such summary promotion (or evacuation) failure information with OS-level threads. The relevant context is long gone, and the actual information being captured is pretty limited. I'd like to see a real use-case. All that said, a more complete and cleaned up fix for the ParScanThreadState[Set] reuse problem should also suffice, e.g. either don't reuse or capture / report data and reinitialize before reuse. From jon.masamitsu at oracle.com Mon Nov 24 18:21:18 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 24 Nov 2014 10:21:18 -0800 Subject: RFR (smallish?) 8061611: Remove deprecated command line flags In-Reply-To: <54731CA0.4030302@oracle.com> References: <546B7683.9000803@oracle.com> <54731CA0.4030302@oracle.com> Message-ID: <5473771E.8000202@oracle.com> On 11/24/2014 03:55 AM, Bengt Rutisson wrote: > > Hi Derek, > > Sorry for taking so long to review this. > > On 2014-11-18 17:40, Derek White wrote: >> Hi Team, >> >> First review request. Please let me know if I've forgotten something >> or have gone completely off the rails. >> >> The main point of this bug is to remove deprecated -XX options which >> are alias for other options. >> >> The only complicated part is that one case, >> /CMSParPromoteBlocksToClaim/ was not a true alias for /OldPLABSize/ >> but a parallel option with different defaults that were synchronized >> in ergo processing. This fix removes the /CMSParPromoteBlocksToClaim >> /variable but preserves using different defaults in the CMS case. >> >> Also in this fix I added warning messages to several remaining >> undocumented command line options aliases. This should ease removal >> of these options in the future >> CMSMarkStackSize ==> MarkStackSize (MarkStackSize not documented either, but came in jdk6) >> G1MarkStackSize ==> MarkStackSize >> CMSMarkStackSizeMax ==> MarkStackSizeMax (MarkStackSizeMax not documented either) >> ParallelMarkingThreads => ConcGCThreads (ConcGCThreads came in jdk6) >> ParallelCMSThreads ==> ConcGCThread >> >> Thanks, >> - Derek >> >> *Webrev*: http://cr.openjdk.java.net/~drwhite/8061611/webrev.00/ >> >> *Bug*: https://bugs.openjdk.java.net/browse/JDK-8061611 > > Looks good to me. One minor nit regarding the naming of the new > constants that you introduce: > > 682 static const int defaultDynamicOldPLABSize = 16; > 683 static const int defaultStaticOldPLABSize = 50; > > HotSpot does not have a common guideline for how to name constants. > There are at least three widely used naming standards: > > - All caps: DEFAULT_DYNAMIC_OLD_PLAB_SIZE > - Like an instance variable: _default_dynamic_old_plab_size > - Like a flag: DefaultDynamicOldPLABSize > > I think it is hard to say which one is "standard" but I would prefer > if we don't introduce yet another naming standard for constants. Do > you mind changing to one of these? Personally I prefer the last one > (the "Like a flag" version). Sorry for this Derek, but I prefer the "_default_dynamic_old_plab_size" because of the leading "_". It is least likely to confuse me (as does like a flag or like an All caps macro). But as Bengt says, there is no standard. Jon > > Bengt > >> >> *Testing*: >> jprt: >> Saw 1-2 intermittent failures that went away on retesting - hangs >> and timeouts. >> >> refworkload: >> no differences >> >> jtreg: Saw a few unexplained results. Not sure if typical or not: >> >> >> Execution failed: `main' threw exception: >> java.lang.Exception: jmap -heap exited with error code: 1 >> >> * gc/metaspace/CompressedClassSpaceSizeInJmapHeap.java >> : >> Checks that jmap -heap contains the flag >> CompressedClassSpaceSize >> >> >> Program >> `/export/users/drwhite/test-builds/j2sdk-image.11.17.2014/bin/java' >> interrupted! (timed out?) >> >> * closed/runtime/AppCDS/SharedArchiveConsistency.java >> : >> SharedArchiveConsistency >> >> Plus a bunch of tests that are labelled "ignored". >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kim.barrett at oracle.com Mon Nov 24 18:44:12 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 24 Nov 2014 13:44:12 -0500 Subject: RFR: 8059066: CardTableModRefBS might commit the same page twice In-Reply-To: <20141124160926.GD23108@ehelin-desktop> References: <54450C6D.3080603@oracle.com> <986B14CB-1598-47D4-A581-4F82115A8257@oracle.com> <544A4C1D.9000908@oracle.com> <20141117174322.GA23553@ehelin-desktop> <08259340-A8E8-4EF9-8DA7-55FFCD702336@oracle.com> <20141124160926.GD23108@ehelin-desktop> Message-ID: On Nov 24, 2014, at 11:09 AM, Erik Helin wrote: > also added an assert to > how that new_end_aligned <= _committed[ri].end() always is true. Based on your earlier analysis about there being at most one overlapping page, I think a tighter assertion could be used, e.g. something like new_end_aligned <= (HeapWord*)align_ptr_up(_committed[ri].start() + 1, _page_size) That would definitely deserve a comment though, and I'm not sure it adds enough to be worthwhile. I looked a bit further down in the file, and initially worried about the uncommit case messing things up, but I think it shouldn't be possible for a committed region to have zero size because the associated covered region never has zero size. > Please > see new webrevs at: > - full: http://cr.openjdk.java.net/~ehelin/8059066/webrev.01/index.html > - inc: http://cr.openjdk.java.net/~ehelin/8059066/webrev.00-01/index.html Looks good to me. From jon.masamitsu at oracle.com Mon Nov 24 18:52:08 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 24 Nov 2014 10:52:08 -0800 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> Message-ID: <54737E58.2040904@oracle.com> On 11/24/2014 10:06 AM, Kim Barrett wrote: > [...] > All that said, a more complete and cleaned up fix for the > ParScanThreadState[Set] reuse problem should also suffice, e.g. either > don't reuse or capture / report data and reinitialize before reuse. I'd suggest that this to be a separate (from fixing this failure) effort. OK if this more extensive clean up is deferred. I can create a new CR for it. Priority? Jon From kim.barrett at oracle.com Mon Nov 24 19:37:11 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 24 Nov 2014 14:37:11 -0500 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: <54737E58.2040904@oracle.com> References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> <54737E58.2040904@oracle.com> Message-ID: On Nov 24, 2014, at 1:52 PM, Jon Masamitsu wrote: > > > On 11/24/2014 10:06 AM, Kim Barrett wrote: >> [...] > >> All that said, a more complete and cleaned up fix for the ParScanThreadState[Set] reuse problem should also suffice, e.g. either don't reuse or capture / report data and reinitialize before reuse. > > I'd suggest that this to be a separate (from fixing this failure) effort. > OK if this more extensive clean up is deferred. I can create a new CR for it. > Priority? Is there an urgency to fixing the failure that makes it worth putting in code that we think we?ll be taking back out later? (Perhaps even sooner than later?) I?m not suggesting there isn?t, just wondering if there is. I also think it might not be all that difficult, but of course I haven?t actually done the work to prove that! From kim.barrett at oracle.com Mon Nov 24 20:02:30 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 24 Nov 2014 15:02:30 -0500 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> Message-ID: <6BE5C1A9-D7DE-49FC-A6C1-6465EA2DC5C2@oracle.com> On Nov 24, 2014, at 1:06 PM, Kim Barrett wrote: > > I'm having a very hard time coming up with a way to make use of the > association of such summary promotion (or evacuation) failure > information with OS-level threads. The relevant context is long gone, > and the actual information being captured is pretty limited. I'd like > to see a real use-case. Here?s another idea. Do we actually need the OS-level thread id, or would the worker gang thread id be sufficient? The latter is captured in the ParScanThreadState object which the PromotionFailedInfo is associated with. We could pass it along to the info object at construction time, replacing the OS-level _thread with the worker gang thread id. This is still technically a message API change, but I think not a visible one; it looks to me as if the relevant primitive type in tracetypes.xml (?OSTHREAD?) is effectively a synonym for ?U4?. The worker gang id is an int, but only small positive values are used. I think this change would be sufficient to address the proximate failure, though not addressing the larger issue of inappropriate / improper reuse of the thread state [set]. From jesper.wilhelmsson at oracle.com Mon Nov 24 22:44:38 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 24 Nov 2014 23:44:38 +0100 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: <6BE5C1A9-D7DE-49FC-A6C1-6465EA2DC5C2@oracle.com> References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> <6BE5C1A9-D7DE-49FC-A6C1-6465EA2DC5C2@oracle.com> Message-ID: <5473B4D6.8020703@oracle.com> Kim, The major pain point throughout the JFR project was to figure out what information to include in the events and if anyone actually needed some info. Some events was gives from the start, we knew from JRockit experience that the information was useful, but in many cases information has been included because it was at hand. The promotion failure events was very much a case of the second kind. We knew it was useful to get information about promotion failures, but exactly how to deliver that information and what details to pass along was not clear. The thread info was initially added because it was there. Nobody asked for it. Later it turned out to be hard to provide the same information in the evacuation failed events in G1, and at that point, since we thought it would be nice it the promotion failed events and the evacuation failed events looked the same, one of the options discussed was to remove the thread info from the event. The discussions lead to the RFE I linked to earlier that asks for thread info in the evacuation failed events. But I repeat, nobody asked for this information, it's there because it was easy to pass it along with the event. I seriously doubt that anyone has stared to use this information out there, but to know for sure we need to ask sustaining and PM. /Jesper Kim Barrett skrev 24/11/14 21:02: > On Nov 24, 2014, at 1:06 PM, Kim Barrett wrote: >> I'm having a very hard time coming up with a way to make use of the >> association of such summary promotion (or evacuation) failure >> information with OS-level threads. The relevant context is long gone, >> and the actual information being captured is pretty limited. I'd like >> to see a real use-case. > Here?s another idea. Do we actually need the OS-level thread id, or would the > worker gang thread id be sufficient? The latter is captured in the ParScanThreadState > object which the PromotionFailedInfo is associated with. We could pass it along > to the info object at construction time, replacing the OS-level _thread with the worker > gang thread id. > > This is still technically a message API change, but I think not a visible one; it looks to me > as if the relevant primitive type in tracetypes.xml (?OSTHREAD?) is effectively a synonym > for ?U4?. The worker gang id is an int, but only small positive values are used. > > I think this change would be sufficient to address the proximate failure, though not > addressing the larger issue of inappropriate / improper reuse of the thread state [set]. > From filipp.zhinkin at oracle.com Tue Nov 25 07:26:42 2014 From: filipp.zhinkin at oracle.com (Filipp Zhinkin) Date: Tue, 25 Nov 2014 11:26:42 +0400 Subject: [9] RFR (L): 8037968: Add tests on alignment of objects copied to survivor space In-Reply-To: <547324AB.1050608@oracle.com> References: <5433B362.9090403@oracle.com> <544E4F78.9050401@oracle.com> <545110D8.3050302@oracle.com> <54510FAB.6050206@oracle.com> <547306E7.7020102@oracle.com> <547324AB.1050608@oracle.com> Message-ID: <54742F32.8000207@oracle.com> Dima, thank you for review! Jon, would you mind sponsoring this change? Regards, Filipp. On 11/24/2014 04:29 PM, Dmitry Fazunenko wrote: > Hi Filipp, > > The new version looks good to me. > > Thanks > Dima > > On 24.11.2014 14:22, Filipp Zhinkin wrote: >> Hi all, >> >> I'm still looking for a second review. >> Everyone is welcome. ;) >> >> Regards, >> Filipp. >> >> On 10/29/2014 08:02 PM, Filipp Zhinkin wrote: >>> Hi Jon, >>> >>> thank you for looking at it! >>> >>> On 10/29/2014 08:07 PM, Jon Masamitsu wrote: >>>> Filipp, >>>> >>>> Sorry for the delay. These tests look good. >>>> >>>> Can you point me at the source for >>>> >>>> 59 CommandLineOptionTest.verifyJVMStartup( >>>> >>>> >>>> I just want to see what the messages are where there is >>>> a failure. >>> Sure: >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/f7cb546710c8/test/testlibrary/com/oracle/java/testlibrary/cli/CommandLineOptionTest.java#l71 >>> >>> >>> Thanks, >>> Filipp. >>>> >>>> Thanks. >>>> >>>> Jon >>>> >>>> On 10/27/2014 6:58 AM, Filipp Zhinkin wrote: >>>>> friendly reminder >>>>> >>>>> On 10/07/2014 01:33 PM, Filipp Zhinkin wrote: >>>>>> Hi all, >>>>>> >>>>>> please review tests on survivor alignment feature (JDK-8031323 [1]). >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~fzhinkin/8037968/webrev.00/ >>>>>> Bug id: https://bugs.openjdk.java.net/browse/JDK-8037968 >>>>>> Testing: manual, automated >>>>>> Description: >>>>>> >>>>>> There are 6 new tests: >>>>>> one verifies SurvivorAlignmentInBytes option processing >>>>>> and other 5 tests verify that that option only affect alignment >>>>>> of objects copied to survivor space and that their alignment is >>>>>> equal to SurvivorAlignmentInBytes value. >>>>>> >>>>>> Since there are no guarantee that _all_ objects copied to >>>>>> survivor space will be alignedthese tests are verifying that >>>>>> objects occupy some expected amount of memory depending on alignment. >>>>>> >>>>>> All tests use following algorithm: >>>>>> - decide how many objects should be allocated >>>>>> - allocate objects >>>>>> - force minor or full GCs if needed >>>>>> - verify usage of particular heap space (eden, survivor or tenured). >>>>>> >>>>>> There are several differences between Hotspot garbage collectors that affect >>>>>> memory usage measurements: >>>>>> 1) precision of such measurements; >>>>>> 2) min amount of heap space that will be occupied by a single object. >>>>>> >>>>>> The first one is about the G1 GC, where usage of eden and survivor regions >>>>>> is reported as (number_of_used_regions * G1HeapRegionSize) [2]. >>>>>> >>>>>> Tests verify that actual memory usage is close to expected usage, >>>>>> i.e. difference between actual and expected usage hasto be lower >>>>>> or equal to 5% of expected usage. >>>>>> However, for G1 GC max allowed difference has to be aligned up to >>>>>> G1HeapRegionSize. >>>>>> >>>>>> The second one is about the CMS GC, where an objecthas to occupy at least >>>>>> MinChunkSize bytes [3][4]. >>>>>> >>>>>> And, of course, amount of memory occupied by an object depends on the >>>>>> alignment >>>>>> used in particular heap space: SurvivorAlignmentInBytes for survivor >>>>>> space and >>>>>> ObjectsAlignmentInBytes for other spaces. >>>>>> >>>>>> To take all such aspects into account tests use AlignmentHelper class, >>>>>> whose instances are created for each heap space. >>>>>> >>>>>> That class is aimed to answer following questions: >>>>>> - how much space will be occupied by an object whose size is M; >>>>>> - how many objects of size M should be allocated to occupy N bytes >>>>>> of memory in particular heap space; >>>>>> - how much space will be occupied by K objects whose size is M; >>>>>> - how close the actual memory usage has to be to the expected memory usage >>>>>> given the precision with which we can measure the memory usage. >>>>>> >>>>>> In order to avoid false failures caused by memory allocated by different >>>>>> threads >>>>>> (for example, some agent was attached to tested JVM and allocated objects >>>>>> during >>>>>> the test run) tests check that only the main test thread was allocating >>>>>> objects. >>>>>> >>>>>> Thanks, >>>>>> Filipp. >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8031323 >>>>>> [2] >>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp#l124 >>>>>> [3] >>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp#l1427 >>>>>> [4] >>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/5c722dffbc0f/src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.hpp#l477 >>>>>> >>>>> >>>> >>> >> > From bengt.rutisson at oracle.com Tue Nov 25 09:12:24 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 25 Nov 2014 10:12:24 +0100 Subject: RFR: 8065749: [TESTBUG]: gc/arguments/TestG1HeapRegionSize.java fails at nightly In-Reply-To: <54733AAB.5020005@oracle.com> References: <54732905.3070804@oracle.com> <54732F0A.3020806@oracle.com> <547341A8.7060108@oracle.com> <54733AAB.5020005@oracle.com> Message-ID: <547447F8.5040307@oracle.com> Hi Dima, On 2014-11-24 15:03, Dmitry Fazunenko wrote: > Hi Bengt, > > > On 24.11.2014 18:33, Bengt Rutisson wrote: >> >> Hi Evgeniya, >> >> On 2014-11-24 14:13, Evgeniya Stepanova wrote: >>> Hi, >>> >>> Could you please review changes for 8065749 ? >>> Test TestG1HeapRegionSize.java is G1-specific and could not work >>> without -XX:+UseG1GC passed directly. >>> Option added to the run. >>> bug: https://bugs.openjdk.java.net/browse/JDK-8065749 >>> fix: http://cr.openjdk.java.net/~eistepan/8065749/webrev.00/ >> >> I think your suggested fix will work, but I have a question regarding >> the original change. >> >> Before the original change the test explicitly checked that it was >> run with G1 and just skipped otherwise: >> >> http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/comparison/6464714dd742/test/gc/arguments/TestG1HeapRegionSize.java >> >> Was this test really failing at all? I thought we were only going to >> add @requires to tests that were failing due to conflicting GC >> combinations. If it was working before I think a different approach >> to fix the current problem would be to revert back to the old code >> without @requires. > > You are right, it was our mistake. This test wasn't reverted when we > decided to update only failing tests. > Yes, a possible solution would be just to roll back the previous > change, but I don't think it makes sense just to roll back. > > What I suggest: > 1) keep this test G1 specific (@requires g1 and +UseG1 explicitly) - > what is proposed by this change > 2) make tests GC independent - remove @requires and check in tests > that RegionSize == in case of G1, and RegionSize == 0 in > case of another GC. > > I don't think that original variant (silently pass if not g1) is the > right solution. I don't think 2) makes sense. G1HeapRegionSize is not set to 0 for other collectors and there is really no reason to do that either. So, testing the value of that with other GCs is just a waste of time. Considering that, I don't really see why 1) is better than reverting to the original code. After all, we are not sure how we want @requires to work and in my mind it is better to use it as little as possible until we have figured out how it should work. Thanks, Bengt > > What do you think? > > -- Dima > > > > > >> >> Thanks, >> Bengt >> >>> >>> Thanks, >>> Evgeniya Stepanova >>> >>> -- >>> /Evgeniya Stepanova/ >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.helin at oracle.com Tue Nov 25 09:17:54 2014 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 25 Nov 2014 10:17:54 +0100 Subject: RFR: 8059066: CardTableModRefBS might commit the same page twice In-Reply-To: References: <54450C6D.3080603@oracle.com> <986B14CB-1598-47D4-A581-4F82115A8257@oracle.com> <544A4C1D.9000908@oracle.com> <20141117174322.GA23553@ehelin-desktop> <08259340-A8E8-4EF9-8DA7-55FFCD702336@oracle.com> <20141124160926.GD23108@ehelin-desktop> Message-ID: <20141125091753.GE23108@ehelin-desktop> On 2014-11-24, Kim Barrett wrote: > On Nov 24, 2014, at 11:09 AM, Erik Helin wrote: > > also added an assert to > > how that new_end_aligned <= _committed[ri].end() always is true. > > Based on your earlier analysis about there being at most one > overlapping page, I think a tighter assertion could be used, e.g. > something like > > new_end_aligned <= (HeapWord*)align_ptr_up(_committed[ri].start() + 1, _page_size) > > That would definitely deserve a comment though, and I'm not sure it > adds enough to be worthwhile. > > I looked a bit further down in the file, and initially worried about > the uncommit case messing things up, but I think it shouldn't be > possible for a committed region to have zero size because the > associated covered region never has zero size. > > > Please > > see new webrevs at: > > - full: http://cr.openjdk.java.net/~ehelin/8059066/webrev.01/index.html > > - inc: http://cr.openjdk.java.net/~ehelin/8059066/webrev.00-01/index.html > > Looks good to me. Many thanks for taking your time to review this tricky change! Erik From evgeniya.stepanova at oracle.com Tue Nov 25 11:11:59 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Tue, 25 Nov 2014 15:11:59 +0400 Subject: RFR: 8065749: [TESTBUG]: gc/arguments/TestG1HeapRegionSize.java fails at nightly In-Reply-To: <547447F8.5040307@oracle.com> References: <54732905.3070804@oracle.com> <54732F0A.3020806@oracle.com> <547341A8.7060108@oracle.com> <54733AAB.5020005@oracle.com> <547447F8.5040307@oracle.com> Message-ID: <547463FF.8010702@oracle.com> Hi Bengt, Ok, file is reverted. http://cr.openjdk.java.net/~eistepan/8065749/webrev.01 But please note that test will run only if UseG1GC set. It will be skipped otherwise (with no GC option set for example). Thanks, Evgeniya Stepanova On 25.11.2014 13:12, Bengt Rutisson wrote: > > Hi Dima, > > On 2014-11-24 15:03, Dmitry Fazunenko wrote: >> Hi Bengt, >> >> >> On 24.11.2014 18:33, Bengt Rutisson wrote: >>> >>> Hi Evgeniya, >>> >>> On 2014-11-24 14:13, Evgeniya Stepanova wrote: >>>> Hi, >>>> >>>> Could you please review changes for 8065749 ? >>>> Test TestG1HeapRegionSize.java is G1-specific and could not work >>>> without -XX:+UseG1GC passed directly. >>>> Option added to the run. >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8065749 >>>> fix: http://cr.openjdk.java.net/~eistepan/8065749/webrev.00/ >>> >>> I think your suggested fix will work, but I have a question >>> regarding the original change. >>> >>> Before the original change the test explicitly checked that it was >>> run with G1 and just skipped otherwise: >>> >>> http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/comparison/6464714dd742/test/gc/arguments/TestG1HeapRegionSize.java >>> >>> Was this test really failing at all? I thought we were only going to >>> add @requires to tests that were failing due to conflicting GC >>> combinations. If it was working before I think a different approach >>> to fix the current problem would be to revert back to the old code >>> without @requires. >> >> You are right, it was our mistake. This test wasn't reverted when we >> decided to update only failing tests. >> Yes, a possible solution would be just to roll back the previous >> change, but I don't think it makes sense just to roll back. >> >> What I suggest: >> 1) keep this test G1 specific (@requires g1 and +UseG1 explicitly) - >> what is proposed by this change >> 2) make tests GC independent - remove @requires and check in tests >> that RegionSize == in case of G1, and RegionSize == 0 in >> case of another GC. >> >> I don't think that original variant (silently pass if not g1) is the >> right solution. > > I don't think 2) makes sense. G1HeapRegionSize is not set to 0 for > other collectors and there is really no reason to do that either. So, > testing the value of that with other GCs is just a waste of time. > > Considering that, I don't really see why 1) is better than reverting > to the original code. After all, we are not sure how we want @requires > to work and in my mind it is better to use it as little as possible > until we have figured out how it should work. > > Thanks, > Bengt > >> >> What do you think? >> >> -- Dima >> >> >> >> >> >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>>> >>>> >>> >> > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Tue Nov 25 13:01:59 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 25 Nov 2014 14:01:59 +0100 Subject: RFR: 8065749: [TESTBUG]: gc/arguments/TestG1HeapRegionSize.java fails at nightly In-Reply-To: <547463FF.8010702@oracle.com> References: <54732905.3070804@oracle.com> <54732F0A.3020806@oracle.com> <547341A8.7060108@oracle.com> <54733AAB.5020005@oracle.com> <547447F8.5040307@oracle.com> <547463FF.8010702@oracle.com> Message-ID: <54747DC7.7090107@oracle.com> On 2014-11-25 12:11, Evgeniya Stepanova wrote: > Hi Bengt, > Ok, file is reverted. > > http://cr.openjdk.java.net/~eistepan/8065749/webrev.01 Thanks, looks good. > > But please note that test will run only if UseG1GC set. It will be > skipped otherwise (with no GC option set for example). Agreed. Bengt > > Thanks, > Evgeniya Stepanova > > > On 25.11.2014 13:12, Bengt Rutisson wrote: >> >> Hi Dima, >> >> On 2014-11-24 15:03, Dmitry Fazunenko wrote: >>> Hi Bengt, >>> >>> >>> On 24.11.2014 18:33, Bengt Rutisson wrote: >>>> >>>> Hi Evgeniya, >>>> >>>> On 2014-11-24 14:13, Evgeniya Stepanova wrote: >>>>> Hi, >>>>> >>>>> Could you please review changes for 8065749 ? >>>>> Test TestG1HeapRegionSize.java is G1-specific and could not work >>>>> without -XX:+UseG1GC passed directly. >>>>> Option added to the run. >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8065749 >>>>> fix: http://cr.openjdk.java.net/~eistepan/8065749/webrev.00/ >>>> >>>> I think your suggested fix will work, but I have a question >>>> regarding the original change. >>>> >>>> Before the original change the test explicitly checked that it was >>>> run with G1 and just skipped otherwise: >>>> >>>> http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/comparison/6464714dd742/test/gc/arguments/TestG1HeapRegionSize.java >>>> >>>> Was this test really failing at all? I thought we were only going >>>> to add @requires to tests that were failing due to conflicting GC >>>> combinations. If it was working before I think a different approach >>>> to fix the current problem would be to revert back to the old code >>>> without @requires. >>> >>> You are right, it was our mistake. This test wasn't reverted when we >>> decided to update only failing tests. >>> Yes, a possible solution would be just to roll back the previous >>> change, but I don't think it makes sense just to roll back. >>> >>> What I suggest: >>> 1) keep this test G1 specific (@requires g1 and +UseG1 explicitly) - >>> what is proposed by this change >>> 2) make tests GC independent - remove @requires and check in tests >>> that RegionSize == in case of G1, and RegionSize == 0 in >>> case of another GC. >>> >>> I don't think that original variant (silently pass if not g1) is the >>> right solution. >> >> I don't think 2) makes sense. G1HeapRegionSize is not set to 0 for >> other collectors and there is really no reason to do that either. So, >> testing the value of that with other GCs is just a waste of time. >> >> Considering that, I don't really see why 1) is better than reverting >> to the original code. After all, we are not sure how we want >> @requires to work and in my mind it is better to use it as little as >> possible until we have figured out how it should work. >> >> Thanks, >> Bengt >> >>> >>> What do you think? >>> >>> -- Dima >>> >>> >>> >>> >>> >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>>> >>>>> >>>> >>> >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.wilhelmsson at oracle.com Tue Nov 25 14:39:03 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 25 Nov 2014 15:39:03 +0100 Subject: RFR: 8065305 - Make it possible to extend the G1CollectorPolicy In-Reply-To: <546E260A.6060708@oracle.com> References: <546E260A.6060708@oracle.com> Message-ID: <54749487.3040502@oracle.com> Hi, This change is now being forward ported to JDK 9. The patch did not apply clean due to unrelated changes below this change in arguments_ext.hpp. So please have a look at it. Webrev for the JDK 9 change: http://cr.openjdk.java.net/~jwilhelm/8065305/webrev_9.00/ Old 8u40 webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/webrev.00/ Thanks, /Jesper Jesper Wilhelmsson skrev 20/11/14 18:34: > Hi, > > Please review this small change to allow an extended version of the > G1CollectorPolicy. > This change also includes a small cleanup to remove a couple of extension > points in the argument processing. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8065305 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8065305/ > > Thanks! > /Jesper > From evgeniya.stepanova at oracle.com Tue Nov 25 15:00:47 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Tue, 25 Nov 2014 19:00:47 +0400 Subject: RFR: 8065749: [TESTBUG]: gc/arguments/TestG1HeapRegionSize.java fails at nightly In-Reply-To: <54747DC7.7090107@oracle.com> References: <54732905.3070804@oracle.com> <54732F0A.3020806@oracle.com> <547341A8.7060108@oracle.com> <54733AAB.5020005@oracle.com> <547447F8.5040307@oracle.com> <547463FF.8010702@oracle.com> <54747DC7.7090107@oracle.com> Message-ID: <5474999F.2080408@oracle.com> Bengt, Jesper Thank you for the review! Evgeniya On 25.11.2014 17:01, Bengt Rutisson wrote: > > On 2014-11-25 12:11, Evgeniya Stepanova wrote: >> Hi Bengt, >> Ok, file is reverted. >> >> http://cr.openjdk.java.net/~eistepan/8065749/webrev.01 > > Thanks, looks good. > >> >> But please note that test will run only if UseG1GC set. It will be >> skipped otherwise (with no GC option set for example). > > Agreed. > > Bengt > >> >> Thanks, >> Evgeniya Stepanova >> >> >> On 25.11.2014 13:12, Bengt Rutisson wrote: >>> >>> Hi Dima, >>> >>> On 2014-11-24 15:03, Dmitry Fazunenko wrote: >>>> Hi Bengt, >>>> >>>> >>>> On 24.11.2014 18:33, Bengt Rutisson wrote: >>>>> >>>>> Hi Evgeniya, >>>>> >>>>> On 2014-11-24 14:13, Evgeniya Stepanova wrote: >>>>>> Hi, >>>>>> >>>>>> Could you please review changes for 8065749 ? >>>>>> Test TestG1HeapRegionSize.java is G1-specific and could not work >>>>>> without -XX:+UseG1GC passed directly. >>>>>> Option added to the run. >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8065749 >>>>>> fix: http://cr.openjdk.java.net/~eistepan/8065749/webrev.00/ >>>>> >>>>> I think your suggested fix will work, but I have a question >>>>> regarding the original change. >>>>> >>>>> Before the original change the test explicitly checked that it was >>>>> run with G1 and just skipped otherwise: >>>>> >>>>> http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/comparison/6464714dd742/test/gc/arguments/TestG1HeapRegionSize.java >>>>> >>>>> Was this test really failing at all? I thought we were only going >>>>> to add @requires to tests that were failing due to conflicting GC >>>>> combinations. If it was working before I think a different >>>>> approach to fix the current problem would be to revert back to the >>>>> old code without @requires. >>>> >>>> You are right, it was our mistake. This test wasn't reverted when >>>> we decided to update only failing tests. >>>> Yes, a possible solution would be just to roll back the previous >>>> change, but I don't think it makes sense just to roll back. >>>> >>>> What I suggest: >>>> 1) keep this test G1 specific (@requires g1 and +UseG1 explicitly) >>>> - what is proposed by this change >>>> 2) make tests GC independent - remove @requires and check in tests >>>> that RegionSize == in case of G1, and RegionSize == 0 in >>>> case of another GC. >>>> >>>> I don't think that original variant (silently pass if not g1) is >>>> the right solution. >>> >>> I don't think 2) makes sense. G1HeapRegionSize is not set to 0 for >>> other collectors and there is really no reason to do that either. >>> So, testing the value of that with other GCs is just a waste of time. >>> >>> Considering that, I don't really see why 1) is better than reverting >>> to the original code. After all, we are not sure how we want >>> @requires to work and in my mind it is better to use it as little as >>> possible until we have figured out how it should work. >>> >>> Thanks, >>> Bengt >>> >>>> >>>> What do you think? >>>> >>>> -- Dima >>>> >>>> >>>> >>>> >>>> >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>>>> >>>>>> >>>>> >>>> >>> >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.masamitsu at oracle.com Tue Nov 25 18:19:13 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 25 Nov 2014 10:19:13 -0800 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> <54737E58.2040904@oracle.com> Message-ID: <5474C821.1070109@oracle.com> On 11/24/2014 11:37 AM, Kim Barrett wrote: > On Nov 24, 2014, at 1:52 PM, Jon Masamitsu wrote: >> >> On 11/24/2014 10:06 AM, Kim Barrett wrote: >>> [...] >>> All that said, a more complete and cleaned up fix for the ParScanThreadState[Set] reuse problem should also suffice, e.g. either don't reuse or capture / report data and reinitialize before reuse. >> I'd suggest that this to be a separate (from fixing this failure) effort. >> OK if this more extensive clean up is deferred. I can create a new CR for it. >> Priority? > Is there an urgency to fixing the failure that makes it worth putting in code that > we think we?ll be taking back out later? (Perhaps even sooner than later?) I?m > not suggesting there isn?t, just wondering if there is. I also think it might not > be all that difficult, but of course I haven?t actually done the work to prove that! I think the fix is simple so not much of a burden to fix the bug. The only urgency currently is getting the bug back log down and avoiding this failure is future testing. It was ranked P3 so higher that many bugs. This fix identifies the cause and is the fix that could be backported if needed. I don't think a larger change that deals with the ParScanThreadState reuse issue should be backported. There will be unknown risks in that. I would suggest starting the ParScanThreadState reuse improvement now while we still remember what it is so I'm not suggesting a delay. Jon > > > From jon.masamitsu at oracle.com Tue Nov 25 21:53:53 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 25 Nov 2014 13:53:53 -0800 Subject: RFR: 8059066: CardTableModRefBS might commit the same page twice In-Reply-To: <20141124160926.GD23108@ehelin-desktop> References: <54450C6D.3080603@oracle.com> <986B14CB-1598-47D4-A581-4F82115A8257@oracle.com> <544A4C1D.9000908@oracle.com> <20141117174322.GA23553@ehelin-desktop> <08259340-A8E8-4EF9-8DA7-55FFCD702336@oracle.com> <20141124160926.GD23108@ehelin-desktop> Message-ID: <5474FA71.20403@oracle.com> On 11/24/2014 08:09 AM, Erik Helin wrote: > On 2014-11-20, Kim Barrett wrote: >> On Nov 20, 2014, at 1:27 PM, Kim Barrett wrote: >>> On Nov 17, 2014, at 12:43 PM, Erik Helin wrote: >>>> Because the start is aligned down and the end is aligned up, the two >>>> committed regions will always share at least one page (since the start >>>> for _committed[1] will be aligned down and the end for _committed[0] >>>> will be aligned up). >>> I?m just getting back to this today, and haven?t yet refreshed my understanding >>> of the code. However, the claim that there will be at least one shared page, >>> (with followup reasoning for exactly one shared page), doesn?t seem right to me. >>> It seems to me there could be zero shared pages, since _committed[1] could >>> (perhaps accidentally, if not by design) be page aligned. >> Hm, and it seems that committed regions are always page aligned. I?m clearly not >> understanding something yet. Back to reading code? > Me and Kim discussed this offline and we both now agree that the existing > patch does work, but the check if the first _committed region expands into > the second _committed region can be simpler. The existing check in the > patch is: > > if (new_end_aligned > _committed[ri].start() && > new_end_aligned <= _committed[ri].end()) { > > The check if new_end_aligned <= _committed[ri].end() is not necessary > because this must always be the case. This is because _committed[0] can't > expand further than the original, unaligned start of _committed[1] and > _committed[1] will always commit at least one page in the byte array for > the card table (starting at _committed[ri].start()). Therefore, it is > enough to check: > > if (new_end_aligned > _committed[ri].start()) { > > I updated the patch to use this simple check and also added an assert to > show that new_end_aligned <= _committed[ri].end() always is true. Please > see new webrevs at: > - full: http://cr.openjdk.java.net/~ehelin/8059066/webrev.01/index.html > - inc: http://cr.openjdk.java.net/~ehelin/8059066/webrev.00-01/index.html Changes look good. Reviewed. Jon > > Thanks, > Erik From mikael.gerdin at oracle.com Wed Nov 26 09:50:35 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 26 Nov 2014 10:50:35 +0100 Subject: RFR [1/2] 8065358: JDK-8065218 Move CMS-specific fields from Space to CompactibleFreeListSpace In-Reply-To: <546E1D48.3050507@oracle.com> References: <546DAA4E.3080807@oracle.com> <546E1D48.3050507@oracle.com> Message-ID: <5475A26B.90501@oracle.com> Hi Sangheon, On 2014-11-20 17:56, Sangheon Kim wrote: > Hi Mikael, > > Looks good. > > One question about a function/variable name. > I know the original names are using upper/lower-cases but according to > hotspot coding style, only lower-case are recommended for > function/variable names. > As it seems easy to change, how about changing them? Although I agree with you in principle, most of compactibleFreeListSpace uses camelCase rather than underscore-separated lower-case so I'd rather keep it the way it is in order to not make it even more inconsistent with itself. /Mikael > > Thanks, > Sangheon > > > On 11/20/2014 12:46 AM, Mikael Gerdin wrote: >> Hi all, >> >> This is the first patch of two to fix JDK-8065358. >> >> In this change I suggest that we move a field used only by CMS to >> CompactibleFreeListSpace and introduce a virtual getter in the Space >> base class. >> >> I estimate that the performance impact of the virtual call will be >> small since the getter path already does several virtual calls. >> >> Moving the field will allow me to add another field to G1's >> G1OffsetTableContigSpace in patch [2/2] without introducing a >> footprint regression. >> >> Webrev: http://cr.openjdk.java.net/~mgerdin/8065218/webrev.0/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8065218 >> Parent bug: https://bugs.openjdk.java.net/browse/JDK-8065358 >> Testing: JPRT >> >> Thanks >> /Mikael > From mikael.gerdin at oracle.com Wed Nov 26 10:04:35 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 26 Nov 2014 11:04:35 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races In-Reply-To: <546DAA51.80601@oracle.com> References: <546DAA51.80601@oracle.com> Message-ID: <5475A5B3.5000801@oracle.com> Hi, any more takers? I need an additional review. /Mikael On 2014-11-20 09:46, Mikael Gerdin wrote: > Hi all, > > This is the second patch of two to fix JDK-8065358 > > In this change I suggest that we introduce a new field _scan_top to > G1OffsetTableContigSpace to keep track of the maximum address where a GC > worker thread is allowed to scan objects in old regions which have been > used as allocation regions during the current evacuation cycle. > > More background (from the bug): > G1 currently uses the _saved_mark_word field in Space to detemine a > point in a region above which a current GC cannot scan objects. This is > needed to prevent card scanning threads from scanning in memory regions > where other gc workers are concurrently allocating objects. > > This re-use of the _saved_mark_word field causes confusion to readers of > the code since it's not used for the same purpose as the other Space > classes. > The setting and reading of this field, and the per region gc time stamp > which accompanies it is also unnecessarily racy. Previously the > _saved_mark_word field was set to the value of the _top pointer when a > region was selected as a GC allocation region and the time stamp was set > to indicate that the saved mark field should be used as a maximum > address. This code had some problems with races in JDK-8058209 and could > be redesigned in a less racy way. > > Suggested fix is to introduce a new field in HeapRegions to keep track > of the maximum address where card scanning is allowed and set that field > at the point of retaining an OldGCAllocRegion instead of when > allocations are being started. That way we get rid of the store ordering > requirement in the timestamp record path. > > There is still a race between the per region time stamps and the per > region top pointer, where we must ensure that the time stamp store must > be visible before any subsequent top stores. > This store ordering is enforced by the fact that all stores to top are > either ordered with #storestore (the initial allocation) or guarded by a > Mutex. To ensure that the reader path sees a consistent view it must be > exectued with the proper load ordering, where we must read top before > the time stamp in order to ensure that we don't see a top value which > has been updated by a concurrent gc worker if we see a time stamp from a > previous gc cycle. > > Webrev: http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8065358 > Testing: JPRT, Kitchensink (4hr+) > > Thanks > /Mikael From stefan.johansson at oracle.com Wed Nov 26 10:07:39 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 26 Nov 2014 11:07:39 +0100 Subject: RFR: 8065227: Report allocation context stats at end of cleanup Message-ID: <5475A66B.3030007@oracle.com> Hi, Please review this forward port to fix this issue in jdk9: https://bugs.openjdk.java.net/browse/JDK-8065227 Webrev: jdk9: http://cr.openjdk.java.net/~sjohanss/8065227/jdk9-hotspot.00/ jdk8u40: http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ Old review thread: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014-November/011348.html Summary: The changeset didn't apply clean due to surrounding changes in concurrentMark.cpp, but the idea for the fix remains the same. Move the allocation_context_stats-update to the clean up phase instead of remark. Testing: * Built through JPRT Thanks, Stefan From thomas.schatzl at oracle.com Wed Nov 26 10:45:47 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 26 Nov 2014 11:45:47 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races In-Reply-To: <546DEBA0.3040207@oracle.com> References: <546DAA51.80601@oracle.com> <546DEBA0.3040207@oracle.com> Message-ID: <1416998747.3098.6.camel@oracle.com> Hi, On Thu, 2014-11-20 at 14:24 +0100, Mikael Gerdin wrote: > Hi, > > (I just noticed that I mistyped the bug number in the subject, it should > be 8065358 twice.) > > I removed the weird qualifier combination from "st" in record_timestamp(). > > Thomas also suggested to me that the guarantee in record_retained_region > may in fact be incorrect in some corner cases, I removed it since no > other part of the code actually depends on this being true. > > New webrev at: > http://cr.openjdk.java.net/~mgerdin/8065358/webrev.1/ > Incremental webrev at: > http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0_to_1/ > Looks good. Thanks. Thomas From thomas.schatzl at oracle.com Wed Nov 26 10:47:27 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 26 Nov 2014 11:47:27 +0100 Subject: RFR: 8065227: Report allocation context stats at end of cleanup In-Reply-To: <5475A66B.3030007@oracle.com> References: <5475A66B.3030007@oracle.com> Message-ID: <1416998847.3098.7.camel@oracle.com> Hi Stefan, On Wed, 2014-11-26 at 11:07 +0100, Stefan Johansson wrote: > Hi, > > Please review this forward port to fix this issue in jdk9: > https://bugs.openjdk.java.net/browse/JDK-8065227 > > Webrev: > jdk9: http://cr.openjdk.java.net/~sjohanss/8065227/jdk9-hotspot.00/ > jdk8u40: http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ > > Old review thread: > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014-November/011348.html > > Summary: > The changeset didn't apply clean due to surrounding changes in > concurrentMark.cpp, but the idea for the fix remains the same. Move the > allocation_context_stats-update to the clean up phase instead of remark. Looks good. Thomas From thomas.schatzl at oracle.com Wed Nov 26 10:49:42 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 26 Nov 2014 11:49:42 +0100 Subject: RFR: 8059066: CardTableModRefBS might commit the same page twice In-Reply-To: <20141125091753.GE23108@ehelin-desktop> References: <54450C6D.3080603@oracle.com> <986B14CB-1598-47D4-A581-4F82115A8257@oracle.com> <544A4C1D.9000908@oracle.com> <20141117174322.GA23553@ehelin-desktop> <08259340-A8E8-4EF9-8DA7-55FFCD702336@oracle.com> <20141124160926.GD23108@ehelin-desktop> <20141125091753.GE23108@ehelin-desktop> Message-ID: <1416998982.3098.8.camel@oracle.com> Hi, On Tue, 2014-11-25 at 10:17 +0100, Erik Helin wrote: > On 2014-11-24, Kim Barrett wrote: > > On Nov 24, 2014, at 11:09 AM, Erik Helin wrote: > > > also added an assert to > > > how that new_end_aligned <= _committed[ri].end() always is true. > > > > Based on your earlier analysis about there being at most one > > overlapping page, I think a tighter assertion could be used, e.g. > > something like > > > > new_end_aligned <= (HeapWord*)align_ptr_up(_committed[ri].start() + 1, _page_size) > > > > That would definitely deserve a comment though, and I'm not sure it > > adds enough to be worthwhile. > > > > I looked a bit further down in the file, and initially worried about > > the uncommit case messing things up, but I think it shouldn't be > > possible for a committed region to have zero size because the > > associated covered region never has zero size. > > > > > Please > > > see new webrevs at: > > > - full: http://cr.openjdk.java.net/~ehelin/8059066/webrev.01/index.html > > > - inc: http://cr.openjdk.java.net/~ehelin/8059066/webrev.00-01/index.html > > > > Looks good to me. > Looks good. Thomas From evgeniya.stepanova at oracle.com Wed Nov 26 12:01:01 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 26 Nov 2014 16:01:01 +0400 Subject: RFR: 8065865: gc/TestSoftReferencesBehaviorOnOOME.java: Error. Can't find source file: TestSoftReference.java Message-ID: <5475C0FD.3010307@oracle.com> Hi, Could you please review very small fix for https://bugs.openjdk.java.net/browse/JDK-8065865. Problem: while test renaming one string was not updated so the test fails to build. Solution: String has been updated bug: https://bugs.openjdk.java.net/browse/JDK-8065865 webrev: http://cr.openjdk.java.net/~eistepan/8065865/webrev.00/ Thanks, Evgeniya Stepanova -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Wed Nov 26 12:09:39 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 26 Nov 2014 13:09:39 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS Message-ID: <5475C303.5060402@oracle.com> Hi everyone, Can I have a couple of reviews for this change to remove the support for ParNew+SerialOld and DefNew+CMS? This work is part of JEP 214 (http://openjdk.java.net/jeps/214). The main change is in the hotspot repo: http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ I also removed the ParNew runs from the jprt configuration: http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ JBS bug: https://bugs.openjdk.java.net/browse/JDK-8065972 Some notes about the change: - UseParNewGC is now a redundant flag. It can only be used with CMS, so it will always have the same value as UseConcMarkSweepGC. Since UseParNewGC was not one of the flags that were deprecated in JDK 8, we decided to leave the flag in for JDK 9 but print a warning message that it is deprecated. We plan on removing UseParNewGC in JDK 10. Here's how the VM will behave with different values of UseParNewGC after my change: $ java -XX:+UseParNewGC -version It is not possible to combine the ParNew young collector with the Serial old collector. Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. $ java -XX:-UseParNewGC -version Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is deprecated and will likely be removed in a future release java version "1.9.0-internal-debug" Java(TM) SE Runtime Environment (build 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is deprecated and will likely be removed in a future release java version "1.9.0-internal-debug" Java(TM) SE Runtime Environment (build 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) Java HotSpot(TM) 64-Bit Server VM (build 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version It is not possible to combine the DefNew young collector with the CMS collector. Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() methods will exit if they fail. No need to check for null after those calls. This simplified the two implementations significantly. - At a few places I've changed from "DefNewGeneration* dng = (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method does the asserts that were often placed before the original casting code. - In concurrentMarkSweepGeneration.cpp I left the casting to DefNewGeneration even though we could now use the more specific ParNewGeneration. I prefer to do such an update later (if at all) since that might come with its own complications regarding virtual calls etc. - The two methods must_be_youngest() and must_be_oldest() were unused. Removing them was not strictly needed for this change, but seemed like a nice thing to do anyway and I don't think it clutters the webrev much. - UseParNewGC is now always true when UseConcMarkSweepGC is true. And it is always false when UseConcMarkSweepGC is false. If statements that used to test against both flags have been simplified to only test against UseConcMarkSweepGC. This means that UseParNewGC is now only used in arguments.cpp - Similarly JTreg tests that used UseParNewGC have been changed to use UseConcMarkSweepGC instead. This should make it easy to remove in JDK 10. - The change to make/jprt.properties only affects the internal testing system we have. It will no longer test the ParNew+SerialOld combination (which is no longer supported). Note that DefNew+CMS was not tested at all by JPRT. Thanks, Bengt From staffan.larsen at oracle.com Wed Nov 26 12:10:27 2014 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Wed, 26 Nov 2014 13:10:27 +0100 Subject: RFR: 8065865: gc/TestSoftReferencesBehaviorOnOOME.java: Error. Can't find source file: TestSoftReference.java In-Reply-To: <5475C0FD.3010307@oracle.com> References: <5475C0FD.3010307@oracle.com> Message-ID: <7C921AB7-9C0F-4C7F-94FB-C51A5A4EBF5A@oracle.com> Looks good! Please push this with JPRT to jdk9/hs-gc and remember to use ?-testset hotspot? when pushing. Thanks, /Staffan > On 26 nov 2014, at 13:01, Evgeniya Stepanova wrote: > > Hi, > > Could you please review very small fix for https://bugs.openjdk.java.net/browse/JDK-8065865 . > > Problem: while test renaming one string was not updated so the test fails to build. > Solution: String has been updated > > bug: https://bugs.openjdk.java.net/browse/JDK-8065865 > webrev: http://cr.openjdk.java.net/~eistepan/8065865/webrev.00/ > > Thanks, > Evgeniya Stepanova > -- > Evgeniya Stepanova -------------- next part -------------- An HTML attachment was scrubbed... URL: From kirk at kodewerk.com Wed Nov 26 12:20:08 2014 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 26 Nov 2014 12:20:08 +0000 Subject: another GC log funny Message-ID: Hi all, I?ve seen this a few times in the past but just recently it seems to be showing up more frequently?. 2014-11-24T17:18:02.364-0100: 0,602: [GC2014-11-24T17:18:02.364-0100: 0,602: [ParNew Desired survivor size 2621440 bytes, new threshold 1 (max 1) - age 1: 5062272 bytes, 5062272 total - age 2: 16 bytes, 5062288 total This is a 1.7.0_51. I?ve also got a log file with new threshold 15 (max 64). This, I guess, is a less serious problem where the user hasn?t been warned that you might ask for 64 but you?re getting 15. Kind regards, Kirk Pepperdine -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mikael.gerdin at oracle.com Wed Nov 26 12:26:49 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 26 Nov 2014 13:26:49 +0100 Subject: RFR [2/2] 8065258: JDK-8065358 Refactor G1s usage of save_marks and reduce related races In-Reply-To: <1416998747.3098.6.camel@oracle.com> References: <546DAA51.80601@oracle.com> <546DEBA0.3040207@oracle.com> <1416998747.3098.6.camel@oracle.com> Message-ID: <5475C709.6030802@oracle.com> Thanks Bengt and Thomas for the reviews, I plan to push this together with the CMS member variable change now. /Mikael On 2014-11-26 11:45, Thomas Schatzl wrote: > Hi, > > On Thu, 2014-11-20 at 14:24 +0100, Mikael Gerdin wrote: >> Hi, >> >> (I just noticed that I mistyped the bug number in the subject, it should >> be 8065358 twice.) >> >> I removed the weird qualifier combination from "st" in record_timestamp(). >> >> Thomas also suggested to me that the guarantee in record_retained_region >> may in fact be incorrect in some corner cases, I removed it since no >> other part of the code actually depends on this being true. >> >> New webrev at: >> http://cr.openjdk.java.net/~mgerdin/8065358/webrev.1/ >> Incremental webrev at: >> http://cr.openjdk.java.net/~mgerdin/8065358/webrev.0_to_1/ >> > > Looks good. Thanks. > > Thomas > > From mikael.gerdin at oracle.com Wed Nov 26 13:07:01 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 26 Nov 2014 14:07:01 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <5475C303.5060402@oracle.com> References: <5475C303.5060402@oracle.com> Message-ID: <5475D075.4040503@oracle.com> Hi Bengt, On 2014-11-26 13:09, Bengt Rutisson wrote: > > Hi everyone, > > Can I have a couple of reviews for this change to remove the support for > ParNew+SerialOld and DefNew+CMS? > > This work is part of JEP 214 (http://openjdk.java.net/jeps/214). > > The main change is in the hotspot repo: > http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ Not a complete review yet, but: * Should we make CMSCollector::_young_gen a ParNewGeneration* now that the type is statically known? ** In that case _young_gen->as_DefNewGeneration() calls can be removed as well. * I think you removed the only usage of ParGCAllocBufferWithBOT (don't forget to remove it from the mapfiles as well if you do remove it) * You can also remove the whole machinery around set_avoid_promotion_undo in ParNew, since CMS does not allow promotion undo. This should rid us of some code duplication since ParNewGeneration::copy_to_survivor_space_with_undo can go away and ParNewGeneration::copy_to_survivor_space__avoiding_promotion_undo can be folded into ParNewGeneration::copy_to_survivor_space. * In fact, OneContigSpaceCardGeneration can be merged with TenuredGeneration, but that may be more appropriate in a separate change. /Mikael > > I also removed the ParNew runs from the jprt configuration: > http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ > > JBS bug: > https://bugs.openjdk.java.net/browse/JDK-8065972 > > Some notes about the change: > > - UseParNewGC is now a redundant flag. It can only be used with CMS, so > it will always have the same value as UseConcMarkSweepGC. Since > UseParNewGC was not one of the flags that were deprecated in JDK 8, we > decided to leave the flag in for JDK 9 but print a warning message that > it is deprecated. We plan on removing UseParNewGC in JDK 10. > > Here's how the VM will behave with different values of UseParNewGC after > my change: > > $ java -XX:+UseParNewGC -version > It is not possible to combine the ParNew young collector with the Serial > old collector. > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > > $ java -XX:-UseParNewGC -version > Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is > deprecated and will likely be removed in a future release > java version "1.9.0-internal-debug" > Java(TM) SE Runtime Environment (build > 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) > Java HotSpot(TM) 64-Bit Server VM (build > 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) > > $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version > Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is > deprecated and will likely be removed in a future release > java version "1.9.0-internal-debug" > Java(TM) SE Runtime Environment (build > 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) > Java HotSpot(TM) 64-Bit Server VM (build > 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) > > $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version > It is not possible to combine the DefNew young collector with the CMS > collector. > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > > - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() > methods will exit if they fail. No need to check for null after those > calls. This simplified the two implementations significantly. > > - At a few places I've changed from "DefNewGeneration* dng = > (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = > _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method > does the asserts that were often placed before the original casting code. > > - In concurrentMarkSweepGeneration.cpp I left the casting to > DefNewGeneration even though we could now use the more specific > ParNewGeneration. I prefer to do such an update later (if at all) since > that might come with its own complications regarding virtual calls etc. > > - The two methods must_be_youngest() and must_be_oldest() were unused. > Removing them was not strictly needed for this change, but seemed like a > nice thing to do anyway and I don't think it clutters the webrev much. > > - UseParNewGC is now always true when UseConcMarkSweepGC is true. And it > is always false when UseConcMarkSweepGC is false. If statements that > used to test against both flags have been simplified to only test > against UseConcMarkSweepGC. This means that UseParNewGC is now only used > in arguments.cpp > > - Similarly JTreg tests that used UseParNewGC have been changed to use > UseConcMarkSweepGC instead. This should make it easy to remove in JDK 10. > > - The change to make/jprt.properties only affects the internal testing > system we have. It will no longer test the ParNew+SerialOld combination > (which is no longer supported). Note that DefNew+CMS was not tested at > all by JPRT. > > Thanks, > Bengt From bengt.rutisson at oracle.com Wed Nov 26 13:24:42 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 26 Nov 2014 14:24:42 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <5475D075.4040503@oracle.com> References: <5475C303.5060402@oracle.com> <5475D075.4040503@oracle.com> Message-ID: <5475D49A.9060803@oracle.com> Hi Mikael, Thanks for looking at this! On 2014-11-26 14:07, Mikael Gerdin wrote: > Hi Bengt, > > On 2014-11-26 13:09, Bengt Rutisson wrote: >> >> Hi everyone, >> >> Can I have a couple of reviews for this change to remove the support for >> ParNew+SerialOld and DefNew+CMS? >> >> This work is part of JEP 214 (http://openjdk.java.net/jeps/214). >> >> The main change is in the hotspot repo: >> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ > > Not a complete review yet, but: > > * Should we make CMSCollector::_young_gen a ParNewGeneration* now that > the type is statically known? > ** In that case _young_gen->as_DefNewGeneration() calls can be removed > as well. Good point. I prefer to do this a separate change. I filed this bug to track that work: Change CMSCollector::_young_gen to be a ParNewGeneration* https://bugs.openjdk.java.net/browse/JDK-8065992 > > * I think you removed the only usage of ParGCAllocBufferWithBOT (don't > forget to remove it from the mapfiles as well if you do remove it) Good catch! Will remove it. > > * You can also remove the whole machinery around > set_avoid_promotion_undo in ParNew, since CMS does not allow promotion > undo. This should rid us of some code duplication since > ParNewGeneration::copy_to_survivor_space_with_undo can go away and > ParNewGeneration::copy_to_survivor_space__avoiding_promotion_undo can > be folded into ParNewGeneration::copy_to_survivor_space. Great. I'll remove that too. > > * In fact, OneContigSpaceCardGeneration can be merged with > TenuredGeneration, but that may be more appropriate in a separate change. Good point. Filed a bug for that too: Merge OneContigSpaceCardGeneration with TenuredGeneration https://bugs.openjdk.java.net/browse/JDK-8065993 Will send out an updated webrev soon. Thanks, Bengt > > /Mikael > >> >> I also removed the ParNew runs from the jprt configuration: >> http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ >> >> JBS bug: >> https://bugs.openjdk.java.net/browse/JDK-8065972 >> >> Some notes about the change: >> >> - UseParNewGC is now a redundant flag. It can only be used with CMS, so >> it will always have the same value as UseConcMarkSweepGC. Since >> UseParNewGC was not one of the flags that were deprecated in JDK 8, we >> decided to leave the flag in for JDK 9 but print a warning message that >> it is deprecated. We plan on removing UseParNewGC in JDK 10. >> >> Here's how the VM will behave with different values of UseParNewGC after >> my change: >> >> $ java -XX:+UseParNewGC -version >> It is not possible to combine the ParNew young collector with the Serial >> old collector. >> Error: Could not create the Java Virtual Machine. >> Error: A fatal exception has occurred. Program will exit. >> >> $ java -XX:-UseParNewGC -version >> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >> deprecated and will likely be removed in a future release >> java version "1.9.0-internal-debug" >> Java(TM) SE Runtime Environment (build >> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >> Java HotSpot(TM) 64-Bit Server VM (build >> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >> >> $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version >> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >> deprecated and will likely be removed in a future release >> java version "1.9.0-internal-debug" >> Java(TM) SE Runtime Environment (build >> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >> Java HotSpot(TM) 64-Bit Server VM (build >> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >> >> $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version >> It is not possible to combine the DefNew young collector with the CMS >> collector. >> Error: Could not create the Java Virtual Machine. >> Error: A fatal exception has occurred. Program will exit. >> >> - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() >> methods will exit if they fail. No need to check for null after those >> calls. This simplified the two implementations significantly. >> >> - At a few places I've changed from "DefNewGeneration* dng = >> (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = >> _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method >> does the asserts that were often placed before the original casting >> code. >> >> - In concurrentMarkSweepGeneration.cpp I left the casting to >> DefNewGeneration even though we could now use the more specific >> ParNewGeneration. I prefer to do such an update later (if at all) since >> that might come with its own complications regarding virtual calls etc. >> >> - The two methods must_be_youngest() and must_be_oldest() were unused. >> Removing them was not strictly needed for this change, but seemed like a >> nice thing to do anyway and I don't think it clutters the webrev much. >> >> - UseParNewGC is now always true when UseConcMarkSweepGC is true. And it >> is always false when UseConcMarkSweepGC is false. If statements that >> used to test against both flags have been simplified to only test >> against UseConcMarkSweepGC. This means that UseParNewGC is now only used >> in arguments.cpp >> >> - Similarly JTreg tests that used UseParNewGC have been changed to use >> UseConcMarkSweepGC instead. This should make it easy to remove in JDK >> 10. >> >> - The change to make/jprt.properties only affects the internal testing >> system we have. It will no longer test the ParNew+SerialOld combination >> (which is no longer supported). Note that DefNew+CMS was not tested at >> all by JPRT. >> >> Thanks, >> Bengt From michail.chernov at oracle.com Wed Nov 26 13:36:10 2014 From: michail.chernov at oracle.com (Michail Chernov) Date: Wed, 26 Nov 2014 16:36:10 +0300 Subject: RFR: 8064909: FragmentMetaspace.java got OutOfMemoryError Message-ID: <5475D74A.2060907@oracle.com> Hi, Please review this simple fix for nightly test failure: Webrev: http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8064909 Problem: test fails because of OOME (not enough heap size). Solution: heap size were increased. Testing: jtreg Thanks, Michail From stefan.karlsson at oracle.com Wed Nov 26 13:42:55 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 26 Nov 2014 14:42:55 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <5475C303.5060402@oracle.com> References: <5475C303.5060402@oracle.com> Message-ID: <5475D8DF.80404@oracle.com> Hi Bengt, The changes looks good. There are a couple of follow-up cleanups that can be done as separate changes, but I don't think they are blocking this patch. http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/test/gc/startup_warnings/TestParNewSerialOld.java.frames.html I think the summary should be changed to: 28 * @summary Test that the unsupported ParNew+SerialOld combination does not start StefanK On 2014-11-26 13:09, Bengt Rutisson wrote: > > Hi everyone, > > Can I have a couple of reviews for this change to remove the support > for ParNew+SerialOld and DefNew+CMS? > > This work is part of JEP 214 (http://openjdk.java.net/jeps/214). > > The main change is in the hotspot repo: > http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ > > I also removed the ParNew runs from the jprt configuration: > http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ > > JBS bug: > https://bugs.openjdk.java.net/browse/JDK-8065972 > > Some notes about the change: > > - UseParNewGC is now a redundant flag. It can only be used with CMS, > so it will always have the same value as UseConcMarkSweepGC. Since > UseParNewGC was not one of the flags that were deprecated in JDK 8, we > decided to leave the flag in for JDK 9 but print a warning message > that it is deprecated. We plan on removing UseParNewGC in JDK 10. > > Here's how the VM will behave with different values of UseParNewGC > after my change: > > $ java -XX:+UseParNewGC -version > It is not possible to combine the ParNew young collector with the > Serial old collector. > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > > $ java -XX:-UseParNewGC -version > Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is > deprecated and will likely be removed in a future release > java version "1.9.0-internal-debug" > Java(TM) SE Runtime Environment (build > 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) > Java HotSpot(TM) 64-Bit Server VM (build > 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) > > $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version > Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is > deprecated and will likely be removed in a future release > java version "1.9.0-internal-debug" > Java(TM) SE Runtime Environment (build > 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) > Java HotSpot(TM) 64-Bit Server VM (build > 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) > > $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version > It is not possible to combine the DefNew young collector with the CMS > collector. > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > > - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() > methods will exit if they fail. No need to check for null after those > calls. This simplified the two implementations significantly. > > - At a few places I've changed from "DefNewGeneration* dng = > (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = > _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method > does the asserts that were often placed before the original casting code. > > - In concurrentMarkSweepGeneration.cpp I left the casting to > DefNewGeneration even though we could now use the more specific > ParNewGeneration. I prefer to do such an update later (if at all) > since that might come with its own complications regarding virtual > calls etc. > > - The two methods must_be_youngest() and must_be_oldest() were unused. > Removing them was not strictly needed for this change, but seemed like > a nice thing to do anyway and I don't think it clutters the webrev much. > > - UseParNewGC is now always true when UseConcMarkSweepGC is true. And > it is always false when UseConcMarkSweepGC is false. If statements > that used to test against both flags have been simplified to only test > against UseConcMarkSweepGC. This means that UseParNewGC is now only > used in arguments.cpp > > - Similarly JTreg tests that used UseParNewGC have been changed to use > UseConcMarkSweepGC instead. This should make it easy to remove in JDK 10. > > - The change to make/jprt.properties only affects the internal testing > system we have. It will no longer test the ParNew+SerialOld > combination (which is no longer supported). Note that DefNew+CMS was > not tested at all by JPRT. > > Thanks, > Bengt From bengt.rutisson at oracle.com Wed Nov 26 13:50:48 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 26 Nov 2014 14:50:48 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <5475D8DF.80404@oracle.com> References: <5475C303.5060402@oracle.com> <5475D8DF.80404@oracle.com> Message-ID: <5475DAB8.7000508@oracle.com> Hi Stefan, Thanks for looking at this! On 2014-11-26 14:42, Stefan Karlsson wrote: > Hi Bengt, > > The changes looks good. There are a couple of follow-up cleanups that > can be done as separate changes, but I don't think they are blocking > this patch. Agreed. I filed two cleanup bugs based on Mikael's review. > > http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/test/gc/startup_warnings/TestParNewSerialOld.java.frames.html > > > I think the summary should be changed to: > > 28 * @summary Test that the unsupported ParNew+SerialOld combination > does not start Good catch. Here's an updated webrev that fixes the summary and also removes the code that Mikael found: http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01/ Only the changes compared to the last webrev: http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00-01.diff/ Thanks, Bengt > > > StefanK > > On 2014-11-26 13:09, Bengt Rutisson wrote: >> >> Hi everyone, >> >> Can I have a couple of reviews for this change to remove the support >> for ParNew+SerialOld and DefNew+CMS? >> >> This work is part of JEP 214 (http://openjdk.java.net/jeps/214). >> >> The main change is in the hotspot repo: >> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ >> >> I also removed the ParNew runs from the jprt configuration: >> http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ >> >> JBS bug: >> https://bugs.openjdk.java.net/browse/JDK-8065972 >> >> Some notes about the change: >> >> - UseParNewGC is now a redundant flag. It can only be used with CMS, >> so it will always have the same value as UseConcMarkSweepGC. Since >> UseParNewGC was not one of the flags that were deprecated in JDK 8, >> we decided to leave the flag in for JDK 9 but print a warning message >> that it is deprecated. We plan on removing UseParNewGC in JDK 10. >> >> Here's how the VM will behave with different values of UseParNewGC >> after my change: >> >> $ java -XX:+UseParNewGC -version >> It is not possible to combine the ParNew young collector with the >> Serial old collector. >> Error: Could not create the Java Virtual Machine. >> Error: A fatal exception has occurred. Program will exit. >> >> $ java -XX:-UseParNewGC -version >> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >> deprecated and will likely be removed in a future release >> java version "1.9.0-internal-debug" >> Java(TM) SE Runtime Environment (build >> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >> Java HotSpot(TM) 64-Bit Server VM (build >> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >> >> $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version >> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >> deprecated and will likely be removed in a future release >> java version "1.9.0-internal-debug" >> Java(TM) SE Runtime Environment (build >> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >> Java HotSpot(TM) 64-Bit Server VM (build >> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >> >> $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version >> It is not possible to combine the DefNew young collector with the CMS >> collector. >> Error: Could not create the Java Virtual Machine. >> Error: A fatal exception has occurred. Program will exit. >> >> - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() >> methods will exit if they fail. No need to check for null after those >> calls. This simplified the two implementations significantly. >> >> - At a few places I've changed from "DefNewGeneration* dng = >> (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = >> _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method >> does the asserts that were often placed before the original casting >> code. >> >> - In concurrentMarkSweepGeneration.cpp I left the casting to >> DefNewGeneration even though we could now use the more specific >> ParNewGeneration. I prefer to do such an update later (if at all) >> since that might come with its own complications regarding virtual >> calls etc. >> >> - The two methods must_be_youngest() and must_be_oldest() were >> unused. Removing them was not strictly needed for this change, but >> seemed like a nice thing to do anyway and I don't think it clutters >> the webrev much. >> >> - UseParNewGC is now always true when UseConcMarkSweepGC is true. And >> it is always false when UseConcMarkSweepGC is false. If statements >> that used to test against both flags have been simplified to only >> test against UseConcMarkSweepGC. This means that UseParNewGC is now >> only used in arguments.cpp >> >> - Similarly JTreg tests that used UseParNewGC have been changed to >> use UseConcMarkSweepGC instead. This should make it easy to remove in >> JDK 10. >> >> - The change to make/jprt.properties only affects the internal >> testing system we have. It will no longer test the ParNew+SerialOld >> combination (which is no longer supported). Note that DefNew+CMS was >> not tested at all by JPRT. >> >> Thanks, >> Bengt > From jesper.wilhelmsson at oracle.com Wed Nov 26 14:19:28 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 26 Nov 2014 15:19:28 +0100 Subject: RFR: 8064909: FragmentMetaspace.java got OutOfMemoryError In-Reply-To: <5475D74A.2060907@oracle.com> References: <5475D74A.2060907@oracle.com> Message-ID: <5475E170.8010305@oracle.com> Looks good! One nit: Please add spaces around operators and after commas to follow the style in the rest of the test. Thanks! /Jesper Michail Chernov skrev 26/11/14 14:36: > Hi, > > Please review this simple fix for nightly test failure: > > Webrev: > http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.00/ > Bug: > https://bugs.openjdk.java.net/browse/JDK-8064909 > > Problem: test fails because of OOME (not enough heap size). > Solution: heap size were increased. > > Testing: > jtreg > > Thanks, > Michail From michail.chernov at oracle.com Wed Nov 26 14:44:56 2014 From: michail.chernov at oracle.com (Michail Chernov) Date: Wed, 26 Nov 2014 17:44:56 +0300 Subject: RFR: 8064909: FragmentMetaspace.java got OutOfMemoryError In-Reply-To: <5475E170.8010305@oracle.com> References: <5475D74A.2060907@oracle.com> <5475E170.8010305@oracle.com> Message-ID: <5475E768.8090209@oracle.com> Jesper, Thanks, was fixed: http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.01/ I hope nothing is forgotten! Thanks, Michail On 26.11.2014 17:19, Jesper Wilhelmsson wrote: > Looks good! > > One nit: Please add spaces around operators and after commas to follow > the style in the rest of the test. > > Thanks! > /Jesper > > > Michail Chernov skrev 26/11/14 14:36: >> Hi, >> >> Please review this simple fix for nightly test failure: >> >> Webrev: >> http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.00/ >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8064909 >> >> Problem: test fails because of OOME (not enough heap size). >> Solution: heap size were increased. >> >> Testing: >> jtreg >> >> Thanks, >> Michail > From bengt.rutisson at oracle.com Wed Nov 26 14:50:21 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 26 Nov 2014 15:50:21 +0100 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: <5474C821.1070109@oracle.com> References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> <54737E58.2040904@oracle.com> <5474C821.1070109@oracle.com> Message-ID: <5475E8AD.1010808@oracle.com> On 2014-11-25 19:19, Jon Masamitsu wrote: > > On 11/24/2014 11:37 AM, Kim Barrett wrote: >> On Nov 24, 2014, at 1:52 PM, Jon Masamitsu >> wrote: >>> >>> On 11/24/2014 10:06 AM, Kim Barrett wrote: >>>> [...] >>>> All that said, a more complete and cleaned up fix for the >>>> ParScanThreadState[Set] reuse problem should also suffice, e.g. >>>> either don't reuse or capture / report data and reinitialize before >>>> reuse. >>> I'd suggest that this to be a separate (from fixing this failure) >>> effort. >>> OK if this more extensive clean up is deferred. I can create a new >>> CR for it. >>> Priority? >> Is there an urgency to fixing the failure that makes it worth putting >> in code that >> we think we?ll be taking back out later? (Perhaps even sooner than >> later?) I?m >> not suggesting there isn?t, just wondering if there is. I also think >> it might not >> be all that difficult, but of course I haven?t actually done the work >> to prove that! > > I think the fix is simple so not much of a burden to fix the bug. The > only > urgency currently is getting the bug back log down and avoiding this > failure > is future testing. It was ranked P3 so higher that many bugs. > > This fix identifies the cause and is the fix that could be backported > if needed. > I don't think a larger change that deals with the ParScanThreadState > reuse > issue should be backported. There will be unknown risks in that. Agreed. I think the proposed patch looks good. Reviewed from my side. > > I would suggest starting the ParScanThreadState reuse improvement now > while we still remember what it is so I'm not suggesting a delay. Sounds like a good plan. Bengt > > Jon > >> >> >> > From Leonid.Mesnik at oracle.com Wed Nov 26 14:56:25 2014 From: Leonid.Mesnik at oracle.com (Leonid Mesnik) Date: Wed, 26 Nov 2014 17:56:25 +0300 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced In-Reply-To: <54732343.6000307@oracle.com> References: <546E0530.4060904@oracle.com> <54732343.6000307@oracle.com> Message-ID: <5475EA19.4080004@oracle.com> Stefan Thank you for reviewing fix. Here is update version: http://cr.openjdk.java.net/~sjohanss/lmesnik/8065579/hotspot.01/ I simplified test to be more robust. It tests only that we could start mark cycle after young/full GC and when concurrent mark cycle is not in progress. Leonid On 24.11.2014 15:23, Stefan Johansson wrote: > Hi Leonid, > > I think the changes for making it possible to trigger a marking cycle > through the WB API looks good. There are however some problems with > the test. > --- > 24 assertTrue(wb.g1StartConcMarkCycle()); > 25 wb.fullGC(); > 26 assertTrue(wb.g1StartConcMarkCycle()); > > Here the test assumes that it should be able to trigger a new marking > cycle since the Full-GC have aborted the previous one, but it is > possible that the concurrent mark thread hasn't yet finished the cycle > even if it has been aborted. That is, during cycle will return true > for some time after the concurrent mark has been aborted. > --- > 28 while (wb.g1InConcurrentMark()) { > 29 assertFalse(wb.g1StartConcMarkCycle()); > 30 } > 31 > 32 assertTrue(wb.g1StartConcMarkCycle()); > 33 > 34 while (wb.g1InConcurrentMark()) { > 35 assertFalse(wb.g1StartConcMarkCycle()); > 36 } > > Here the problem is that we can't guarantee that the concurrent cycle > won't finish just before the call to wb.g1StartConcMarkCycle(), and > therefor the assert might fail. > --- > > I'm not sure how you want to improve the test to avoid these problems, > the only really robust way I think is to only use the > g1InConcurrentMark() call to wait until marking is finished and then > assert that we can trigger a new cycle. > > Thanks, > Stefan > > On 2014-11-20 16:13, Leonid Mesnik wrote: >> Hi >> >> Here is the fix which introduce WB method which stars concurrent mark >> cycle explicitly. It is needed to make CM cycle related tests more >> stable. >> >> Changeset also includes test and very small fix of WB method >> g1InConcurrentMark . >> Fix pass JPRT with hotspot testset. >> >> JIRA: https://bugs.openjdk.java.net/browse/JDK-8065579 >> Webrev: http://cr.openjdk.java.net/~sjohanss/lmesnik/8065579/hotspot.00/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.johansson at oracle.com Wed Nov 26 16:58:25 2014 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 26 Nov 2014 17:58:25 +0100 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced In-Reply-To: <5475EA19.4080004@oracle.com> References: <546E0530.4060904@oracle.com> <54732343.6000307@oracle.com> <5475EA19.4080004@oracle.com> Message-ID: <547606B1.2070403@oracle.com> Hi Leonid, This looks good. I can sponsor and push the change but we need a Reviewer that looks at the change first. Thanks, Stefan On 2014-11-26 15:56, Leonid Mesnik wrote: > Stefan > > Thank you for reviewing fix. > > Here is update version: > http://cr.openjdk.java.net/~sjohanss/lmesnik/8065579/hotspot.01/ > > > I simplified test to be more robust. It tests only that we could start > mark cycle after young/full GC and when concurrent mark cycle is not > in progress. > > Leonid > > On 24.11.2014 15:23, Stefan Johansson wrote: >> Hi Leonid, >> >> I think the changes for making it possible to trigger a marking cycle >> through the WB API looks good. There are however some problems with >> the test. >> --- >> 24 assertTrue(wb.g1StartConcMarkCycle()); >> 25 wb.fullGC(); >> 26 assertTrue(wb.g1StartConcMarkCycle()); >> >> Here the test assumes that it should be able to trigger a new marking >> cycle since the Full-GC have aborted the previous one, but it is >> possible that the concurrent mark thread hasn't yet finished the >> cycle even if it has been aborted. That is, during cycle will return >> true for some time after the concurrent mark has been aborted. >> --- >> 28 while (wb.g1InConcurrentMark()) { >> 29 assertFalse(wb.g1StartConcMarkCycle()); >> 30 } >> 31 >> 32 assertTrue(wb.g1StartConcMarkCycle()); >> 33 >> 34 while (wb.g1InConcurrentMark()) { >> 35 assertFalse(wb.g1StartConcMarkCycle()); >> 36 } >> >> Here the problem is that we can't guarantee that the concurrent cycle >> won't finish just before the call to wb.g1StartConcMarkCycle(), and >> therefor the assert might fail. >> --- >> >> I'm not sure how you want to improve the test to avoid these >> problems, the only really robust way I think is to only use the >> g1InConcurrentMark() call to wait until marking is finished and then >> assert that we can trigger a new cycle. >> >> Thanks, >> Stefan >> >> On 2014-11-20 16:13, Leonid Mesnik wrote: >>> Hi >>> >>> Here is the fix which introduce WB method which stars concurrent >>> mark cycle explicitly. It is needed to make CM cycle related tests >>> more stable. >>> >>> Changeset also includes test and very small fix of WB method >>> g1InConcurrentMark . >>> Fix pass JPRT with hotspot testset. >>> >>> JIRA: https://bugs.openjdk.java.net/browse/JDK-8065579 >>> Webrev: http://cr.openjdk.java.net/~sjohanss/lmesnik/8065579/hotspot.00/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sangheon.kim at oracle.com Wed Nov 26 17:12:31 2014 From: sangheon.kim at oracle.com (Sangheon Kim) Date: Wed, 26 Nov 2014 09:12:31 -0800 Subject: RFR [1/2] 8065358: JDK-8065218 Move CMS-specific fields from Space to CompactibleFreeListSpace In-Reply-To: <5475A26B.90501@oracle.com> References: <546DAA4E.3080807@oracle.com> <546E1D48.3050507@oracle.com> <5475A26B.90501@oracle.com> Message-ID: <547609FF.8010307@oracle.com> Hi Mikael, Yes, I agree. Sangheon On 11/26/2014 01:50 AM, Mikael Gerdin wrote: > Hi Sangheon, > > On 2014-11-20 17:56, Sangheon Kim wrote: >> Hi Mikael, >> >> Looks good. >> >> One question about a function/variable name. >> I know the original names are using upper/lower-cases but according to >> hotspot coding style, only lower-case are recommended for >> function/variable names. >> As it seems easy to change, how about changing them? > > Although I agree with you in principle, most of > compactibleFreeListSpace uses camelCase rather than > underscore-separated lower-case so I'd rather keep it the way it is in > order to not make it even more inconsistent with itself. > > /Mikael > >> >> Thanks, >> Sangheon >> >> >> On 11/20/2014 12:46 AM, Mikael Gerdin wrote: >>> Hi all, >>> >>> This is the first patch of two to fix JDK-8065358. >>> >>> In this change I suggest that we move a field used only by CMS to >>> CompactibleFreeListSpace and introduce a virtual getter in the Space >>> base class. >>> >>> I estimate that the performance impact of the virtual call will be >>> small since the getter path already does several virtual calls. >>> >>> Moving the field will allow me to add another field to G1's >>> G1OffsetTableContigSpace in patch [2/2] without introducing a >>> footprint regression. >>> >>> Webrev: http://cr.openjdk.java.net/~mgerdin/8065218/webrev.0/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065218 >>> Parent bug: https://bugs.openjdk.java.net/browse/JDK-8065358 >>> Testing: JPRT >>> >>> Thanks >>> /Mikael >> From jesper.wilhelmsson at oracle.com Wed Nov 26 17:54:34 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 26 Nov 2014 18:54:34 +0100 Subject: RFR: 8064909: FragmentMetaspace.java got OutOfMemoryError In-Reply-To: <5475E768.8090209@oracle.com> References: <5475D74A.2060907@oracle.com> <5475E170.8010305@oracle.com> <5475E768.8090209@oracle.com> Message-ID: <547613DA.7090104@oracle.com> Looks good. Thanks! /Jesper Michail Chernov skrev 26/11/14 15:44: > Jesper, > > Thanks, was fixed: > http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.01/ > I hope nothing is forgotten! > > Thanks, > Michail > > On 26.11.2014 17:19, Jesper Wilhelmsson wrote: >> Looks good! >> >> One nit: Please add spaces around operators and after commas to follow the >> style in the rest of the test. >> >> Thanks! >> /Jesper >> >> >> Michail Chernov skrev 26/11/14 14:36: >>> Hi, >>> >>> Please review this simple fix for nightly test failure: >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.00/ >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8064909 >>> >>> Problem: test fails because of OOME (not enough heap size). >>> Solution: heap size were increased. >>> >>> Testing: >>> jtreg >>> >>> Thanks, >>> Michail >> > From jon.masamitsu at oracle.com Wed Nov 26 18:43:51 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 26 Nov 2014 10:43:51 -0800 Subject: RFR: 8065227: Report allocation context stats at end of cleanup In-Reply-To: <5475A66B.3030007@oracle.com> References: <5475A66B.3030007@oracle.com> Message-ID: <54761F67.8040003@oracle.com> Stefan, Still looks good. Reviewed. Jon On 11/26/2014 2:07 AM, Stefan Johansson wrote: > Hi, > > Please review this forward port to fix this issue in jdk9: > https://bugs.openjdk.java.net/browse/JDK-8065227 > > Webrev: > jdk9: http://cr.openjdk.java.net/~sjohanss/8065227/jdk9-hotspot.00/ > jdk8u40: http://cr.openjdk.java.net/~sjohanss/8065227/hotspot.00/ > > Old review thread: > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2014-November/011348.html > > > Summary: > The changeset didn't apply clean due to surrounding changes in > concurrentMark.cpp, but the idea for the fix remains the same. Move > the allocation_context_stats-update to the clean up phase instead of > remark. > > Testing: > * Built through JPRT > > Thanks, > Stefan From jon.masamitsu at oracle.com Wed Nov 26 19:04:49 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 26 Nov 2014 11:04:49 -0800 Subject: RFR: 8064909: FragmentMetaspace.java got OutOfMemoryError In-Reply-To: <5475D74A.2060907@oracle.com> References: <5475D74A.2060907@oracle.com> Message-ID: <54762451.3070802@oracle.com> Michail, Your change makes this test pass but it seems like at some future date 300m might not be big enough (for whatever reason). Could the test be make to caught an OOME, print out a message saying that an OOME doesn't mean the test failed but that the test needs a larger heap? Then pass an exception up (maybe some type of Runtime exception - sorry if that is vague but I don't what type of exception would make sense). That would mean we wouldn't have to spend time diagnosing what the OOME means again. Jon On 11/26/2014 5:36 AM, Michail Chernov wrote: > Hi, > > Please review this simple fix for nightly test failure: > > Webrev: > http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.00/ > Bug: > https://bugs.openjdk.java.net/browse/JDK-8064909 > > Problem: test fails because of OOME (not enough heap size). > Solution: heap size were increased. > > Testing: > jtreg > > Thanks, > Michail From kim.barrett at oracle.com Wed Nov 26 20:36:51 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 26 Nov 2014 15:36:51 -0500 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: <5475E8AD.1010808@oracle.com> References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> <54737E58.2040904@oracle.com> <5474C821.1070109@oracle.com> <5475E8AD.1010808@oracle.com> Message-ID: On Nov 26, 2014, at 9:50 AM, Bengt Rutisson wrote: > > > On 2014-11-25 19:19, Jon Masamitsu wrote: >> >> On 11/24/2014 11:37 AM, Kim Barrett wrote: >>> On Nov 24, 2014, at 1:52 PM, Jon Masamitsu wrote: >>>> >>>> On 11/24/2014 10:06 AM, Kim Barrett wrote: >>>>> [...] >>>>> All that said, a more complete and cleaned up fix for the ParScanThreadState[Set] reuse problem should also suffice, e.g. either don't reuse or capture / report data and reinitialize before reuse. >>>> I'd suggest that this to be a separate (from fixing this failure) effort. >>>> OK if this more extensive clean up is deferred. I can create a new CR for it. >>>> Priority? >>> Is there an urgency to fixing the failure that makes it worth putting in code that >>> we think we?ll be taking back out later? (Perhaps even sooner than later?) I?m >>> not suggesting there isn?t, just wondering if there is. I also think it might not >>> be all that difficult, but of course I haven?t actually done the work to prove that! >> >> I think the fix is simple so not much of a burden to fix the bug. The only >> urgency currently is getting the bug back log down and avoiding this failure >> is future testing. It was ranked P3 so higher that many bugs. >> >> This fix identifies the cause and is the fix that could be backported if needed. >> I don't think a larger change that deals with the ParScanThreadState reuse >> issue should be backported. There will be unknown risks in that. > > Agreed. I think the proposed patch looks good. > > Reviewed from my side. > >> >> I would suggest starting the ParScanThreadState reuse improvement now >> while we still remember what it is so I'm not suggesting a delay. > > Sounds like a good plan. Agreed. From Dmitry.Fazunenko at oracle.com Wed Nov 26 20:51:32 2014 From: Dmitry.Fazunenko at oracle.com (Dmitry Fazunenko) Date: Wed, 26 Nov 2014 23:51:32 +0300 Subject: RFR: 8064909: FragmentMetaspace.java got OutOfMemoryError In-Reply-To: <54762451.3070802@oracle.com> References: <5475D74A.2060907@oracle.com> <54762451.3070802@oracle.com> Message-ID: <54763D54.3070704@oracle.com> Hi Jon, The original version of test worked for 80 seconds trying to perform as many iterations as possible. The number of iterations performed depended on how fast is the machine. With each next iteration the size of generated and loaded classes is growing, so on fast enough machines 80 seconds is enough to run out of heap while generating a class. The fix not only sets the heap, but limits iterations. 300m heap is enough for 200 iterations. Your approach, with catching OOME(heap) and passing will also work, but it will reduce the test readability (and potentially could bring more problems). An alternative approach would be to limit metaspace and heap accordingly and load classes until we don't run out metaspace... But this might take awhile. So, I hope that Michael's fix is good. Thanks for looking and expressing comments. Dima On 26.11.2014 22:04, Jon Masamitsu wrote: > Michail, > > Your change makes this test pass but it seems like at > some future date 300m might not be big enough > (for whatever reason). Could the test be make to > caught an OOME, print out a message saying that > an OOME doesn't mean the test failed but that > the test needs a larger heap? Then pass an > exception up (maybe some type of Runtime > exception - sorry if that is vague but I don't > what type of exception would make sense). That > would mean we wouldn't have to spend time > diagnosing what the OOME means again. > > Jon > > On 11/26/2014 5:36 AM, Michail Chernov wrote: >> Hi, >> >> Please review this simple fix for nightly test failure: >> >> Webrev: >> http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.00/ >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8064909 >> >> Problem: test fails because of OOME (not enough heap size). >> Solution: heap size were increased. >> >> Testing: >> jtreg >> >> Thanks, >> Michail > From jon.masamitsu at oracle.com Wed Nov 26 23:00:00 2014 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 26 Nov 2014 15:00:00 -0800 Subject: RFR: 8064909: FragmentMetaspace.java got OutOfMemoryError In-Reply-To: <54763D54.3070704@oracle.com> References: <5475D74A.2060907@oracle.com> <54762451.3070802@oracle.com> <54763D54.3070704@oracle.com> Message-ID: <54765B70.10509@oracle.com> Dima, If this test fails with an OOME in the future, I would like it to be obvious that the failure is not that an OOME occurred. I cannot tell that from looking at the test. Can the test be changed so I don't have to spend time figuring out that the OOME is not a failure mode of the test? Jon On 11/26/2014 12:51 PM, Dmitry Fazunenko wrote: > Hi Jon, > > The original version of test worked for 80 seconds trying to perform > as many iterations as possible. The number of iterations performed > depended on how fast is the machine. With each next iteration the size > of generated and loaded classes is growing, so on fast enough machines > 80 seconds is enough to run out of heap while generating a class. > > The fix not only sets the heap, but limits iterations. 300m heap is > enough for 200 iterations. > > Your approach, with catching OOME(heap) and passing will also work, > but it will reduce the test readability (and potentially could bring > more problems). > > An alternative approach would be to limit metaspace and heap > accordingly and load classes until we don't run out metaspace... But > this might take awhile. > > So, I hope that Michael's fix is good. > > Thanks for looking and expressing comments. > Dima > > > > > On 26.11.2014 22:04, Jon Masamitsu wrote: >> Michail, >> >> Your change makes this test pass but it seems like at >> some future date 300m might not be big enough >> (for whatever reason). Could the test be make to >> caught an OOME, print out a message saying that >> an OOME doesn't mean the test failed but that >> the test needs a larger heap? Then pass an >> exception up (maybe some type of Runtime >> exception - sorry if that is vague but I don't >> what type of exception would make sense). That >> would mean we wouldn't have to spend time >> diagnosing what the OOME means again. >> >> Jon >> >> On 11/26/2014 5:36 AM, Michail Chernov wrote: >>> Hi, >>> >>> Please review this simple fix for nightly test failure: >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.00/ >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8064909 >>> >>> Problem: test fails because of OOME (not enough heap size). >>> Solution: heap size were increased. >>> >>> Testing: >>> jtreg >>> >>> Thanks, >>> Michail >> > From Leonid.Mesnik at oracle.com Wed Nov 26 23:10:53 2014 From: Leonid.Mesnik at oracle.com (Leonid Mesnik) Date: Thu, 27 Nov 2014 02:10:53 +0300 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced In-Reply-To: <547606B1.2070403@oracle.com> References: <546E0530.4060904@oracle.com> <54732343.6000307@oracle.com> <5475EA19.4080004@oracle.com> <547606B1.2070403@oracle.com> Message-ID: <54765DFD.5090105@oracle.com> Stefan Thank you for review. Could I have review from Reviewer for this fix? Leonid On 26.11.2014 19:58, Stefan Johansson wrote: > Hi Leonid, > > This looks good. I can sponsor and push the change but we need a > Reviewer that looks at the change first. > > Thanks, > Stefan > > On 2014-11-26 15:56, Leonid Mesnik wrote: >> Stefan >> >> Thank you for reviewing fix. >> >> Here is update version: >> http://cr.openjdk.java.net/~sjohanss/lmesnik/8065579/hotspot.01/ >> >> >> I simplified test to be more robust. It tests only that we could >> start mark cycle after young/full GC and when concurrent mark cycle >> is not in progress. >> >> Leonid >> >> On 24.11.2014 15:23, Stefan Johansson wrote: >>> Hi Leonid, >>> >>> I think the changes for making it possible to trigger a marking >>> cycle through the WB API looks good. There are however some problems >>> with the test. >>> --- >>> 24 assertTrue(wb.g1StartConcMarkCycle()); >>> 25 wb.fullGC(); >>> 26 assertTrue(wb.g1StartConcMarkCycle()); >>> >>> Here the test assumes that it should be able to trigger a new >>> marking cycle since the Full-GC have aborted the previous one, but >>> it is possible that the concurrent mark thread hasn't yet finished >>> the cycle even if it has been aborted. That is, during cycle will >>> return true for some time after the concurrent mark has been aborted. >>> --- >>> 28 while (wb.g1InConcurrentMark()) { >>> 29 assertFalse(wb.g1StartConcMarkCycle()); >>> 30 } >>> 31 >>> 32 assertTrue(wb.g1StartConcMarkCycle()); >>> 33 >>> 34 while (wb.g1InConcurrentMark()) { >>> 35 assertFalse(wb.g1StartConcMarkCycle()); >>> 36 } >>> >>> Here the problem is that we can't guarantee that the concurrent >>> cycle won't finish just before the call to >>> wb.g1StartConcMarkCycle(), and therefor the assert might fail. >>> --- >>> >>> I'm not sure how you want to improve the test to avoid these >>> problems, the only really robust way I think is to only use the >>> g1InConcurrentMark() call to wait until marking is finished and then >>> assert that we can trigger a new cycle. >>> >>> Thanks, >>> Stefan >>> >>> On 2014-11-20 16:13, Leonid Mesnik wrote: >>>> Hi >>>> >>>> Here is the fix which introduce WB method which stars concurrent >>>> mark cycle explicitly. It is needed to make CM cycle related tests >>>> more stable. >>>> >>>> Changeset also includes test and very small fix of WB method >>>> g1InConcurrentMark . >>>> Fix pass JPRT with hotspot testset. >>>> >>>> JIRA: https://bugs.openjdk.java.net/browse/JDK-8065579 >>>> Webrev: >>>> http://cr.openjdk.java.net/~sjohanss/lmesnik/8065579/hotspot.00/ >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sangheon.kim at oracle.com Thu Nov 27 06:13:06 2014 From: sangheon.kim at oracle.com (Sangheon Kim) Date: Wed, 26 Nov 2014 22:13:06 -0800 Subject: RFR(s): 8055239: assert(_thread == Thread::current()->osthread()) failed: The PromotionFailedInfo should be thread local. In-Reply-To: References: <546ECF05.8000504@oracle.com> <5472C657.2090501@oracle.com> <5472DF13.4020800@oracle.com> <5472F5FF.4040102@oracle.com> <54737E58.2040904@oracle.com> <5474C821.1070109@oracle.com> <5475E8AD.1010808@oracle.com> Message-ID: <5476C0F2.2010507@oracle.com> Hi, Thank you all for reviewing this. And I filed a new JDK-8066075 for enhancement of ParScanThreadStateSet. Sangheon On 11/26/2014 12:36 PM, Kim Barrett wrote: > On Nov 26, 2014, at 9:50 AM, Bengt Rutisson wrote: >> >> On 2014-11-25 19:19, Jon Masamitsu wrote: >>> On 11/24/2014 11:37 AM, Kim Barrett wrote: >>>> On Nov 24, 2014, at 1:52 PM, Jon Masamitsu wrote: >>>>> On 11/24/2014 10:06 AM, Kim Barrett wrote: >>>>>> [...] >>>>>> All that said, a more complete and cleaned up fix for the ParScanThreadState[Set] reuse problem should also suffice, e.g. either don't reuse or capture / report data and reinitialize before reuse. >>>>> I'd suggest that this to be a separate (from fixing this failure) effort. >>>>> OK if this more extensive clean up is deferred. I can create a new CR for it. >>>>> Priority? >>>> Is there an urgency to fixing the failure that makes it worth putting in code that >>>> we think we?ll be taking back out later? (Perhaps even sooner than later?) I?m >>>> not suggesting there isn?t, just wondering if there is. I also think it might not >>>> be all that difficult, but of course I haven?t actually done the work to prove that! >>> I think the fix is simple so not much of a burden to fix the bug. The only >>> urgency currently is getting the bug back log down and avoiding this failure >>> is future testing. It was ranked P3 so higher that many bugs. >>> >>> This fix identifies the cause and is the fix that could be backported if needed. >>> I don't think a larger change that deals with the ParScanThreadState reuse >>> issue should be backported. There will be unknown risks in that. >> Agreed. I think the proposed patch looks good. >> >> Reviewed from my side. >> >>> I would suggest starting the ParScanThreadState reuse improvement now >>> while we still remember what it is so I'm not suggesting a delay. >> Sounds like a good plan. > Agreed. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Nov 27 07:38:28 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 27 Nov 2014 08:38:28 +0100 Subject: RFR: JDK-8061259: ParNew promotion failed is serialized on a lock In-Reply-To: References: <5451E434.5030905@oracle.com> <5451E69F.7040004@oracle.com> <40C3FFB1-4108-4B85-A848-8850A8DE83F4@oracle.com> <54642CFC.7020609@oracle.com> <1693AB3A-6605-4A0A-A1B0-E997ECDEA78D@oracle.com> <5465D17B.3030209@oracle.com> <94863D3E-B732-453D-890E-EB89E1A9FA0A@oracle.com> <5469C9C1.7020306@oracle.com> <90D9148E-EC1F-446A-B312-B9FC01BF3070@oracle.com> <1A268F67-0A0C-457A-B158-C9C3D9444EC6@oracle.com> <737C76A3-6C09-4F12-9B34-FC52A89C! 82B4@oracle.com> Message-ID: <5476D4F4.3060701@oracle.com> Hi Kim and Jungwoo, Sorry for not replying earlier. On 2014-11-24 16:13, Kim Barrett wrote: > On Nov 23, 2014, at 10:08 PM, Kim Barrett wrote: >> On Nov 21, 2014, at 3:44 PM, Kim Barrett wrote: >>> On Nov 20, 2014, at 9:27 PM, Jungwoo Ha wrote: >>>> If we are to put a wrapper around, why not just go with the original change? >>>> I don't see adding a single field on a nearly singleton class is a bad thing. >>>> The changes are well wrapped inside par_promote. >>> There are multiple implementations of par_promote, for different >>> old-space collectors. (par_promote is a virtual function.) At least >>> some of the others may have similar issues (for example, at a quick >>> look, ParallelOld appears to have a similar locking structure), and it >>> seems like all could benefit from this short-circuiting. >>> >>> [One of the copy_to_survivor_space_XXX functions (the callers of >>> par_promote) is used exclusively when CMS is the old-space collector, >>> the other is used for other old-space collectors.] >> Except I?ve now been reminded that ParNew doesn?t get used with ParallelOld. >> In fact, several combinations of old/new collectors were deprecated in jdk8 and >> are slated to be removed in jdk9 (some already have), and it looks like ParNew >> will only remain in conjunction with CMS, making the split of ParNew?s >> copy_to_survivor_space into two variants no longer needed, and one of them >> redundant, making this wrapper vs comment duplication vs whatever issue moot. > To be clear, what I'm suggesting is that, in light of the deprecation > and expected removal of some combinations with ParNew, only > copy_to_survivor_space_avoiding_promotion_undo() really needs the > proposed change, and copy_to_survivor_space_with_undo() could be left > unmodified. Yes, this is correct. I sent out the review to remove the support for ParNew+SerialOld yesterday. That removes the copy_to_survivor_space_avoiding_promotion_undo() method and folds the copy_to_survivor_space_with_undo() method in to copy_to_survivor_space() since that is now the only use case. See the email thread "RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS" for more details. So, my suggestion would be to hold off with the patch for JDK-8061259 a couple of days until my cleanup has been pushed. That way there will only be one place to add the promotion failure check. I agree with Kim that a comment with a short explanation about the intentional (safe) race would be good. Thanks, Bengt > From thomas.schatzl at oracle.com Thu Nov 27 08:25:03 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 27 Nov 2014 09:25:03 +0100 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced In-Reply-To: <54765DFD.5090105@oracle.com> References: <546E0530.4060904@oracle.com> <54732343.6000307@oracle.com> <5475EA19.4080004@oracle.com> <547606B1.2070403@oracle.com> <54765DFD.5090105@oracle.com> Message-ID: <1417076703.3195.1.camel@oracle.com> Hi, On Thu, 2014-11-27 at 02:10 +0300, Leonid Mesnik wrote: > Stefan > > Thank you for review. > > Could I have review from Reviewer for this fix? - gcCause.cpp: Is it possible to let the human readable string be similar to the others? I.e. "WhiteBox Initiated Concurrent Mark" instead of "WhiteBox Initiated ConcurrentMarkCycle"? - WhiteBox.java: In the comment, please start sentences with capital letters. Also add punctuation if it is a full sentence. All but this and the "force xy GC" follow this style. - TestConcMarkCycleWB.java The test needs to install sun.hotspot.WhiteBox$WhiteBoxPermission too. There is a typo in the @summary line: relatd->related Thanks, Thomas From thomas.schatzl at oracle.com Thu Nov 27 08:31:18 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 27 Nov 2014 09:31:18 +0100 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced In-Reply-To: <1417076703.3195.1.camel@oracle.com> References: <546E0530.4060904@oracle.com> <54732343.6000307@oracle.com> <5475EA19.4080004@oracle.com> <547606B1.2070403@oracle.com> <54765DFD.5090105@oracle.com> <1417076703.3195.1.camel@oracle.com> Message-ID: <1417077078.3195.2.camel@oracle.com> Hi, from the "Add method to WhiteBox to get vm pagesize" thread: On 27/11/2014 3:36 AM, Yumin Qi wrote: > Thanks for the review. Yes, the test will build testlibrary with > > @library /testlibrary /testlibrary/whitebox No that won't necessarily build the testlibrary. From other email: >> I'm having a problem running a test in 8u25 that uses the testlibrary >> ProcessTools API. I get a ClassNotFoundException. Looking in the >> classes directory I only see two testlibrary classes - which map to >> two specific testlibrary classes that one test has on its @build >> line. The test in question simply has: >> >> @library /testlibrary >> >> Does it need an explicit: >> >> @build com.oracle.java.testlibrary.* > > Yes. It turns out that JTReg might not compile the library classes on > demand (but it does sometimes). So it is better to specify the > required build manually. > > -JB- David ----- > > Thanks I.e. you apparently also need to explicitly @build the testlibrary stuff. Thanks, Thomas From thomas.schatzl at oracle.com Thu Nov 27 13:17:54 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 27 Nov 2014 14:17:54 +0100 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced In-Reply-To: <1417076703.3195.1.camel@oracle.com> References: <546E0530.4060904@oracle.com> <54732343.6000307@oracle.com> <5475EA19.4080004@oracle.com> <547606B1.2070403@oracle.com> <54765DFD.5090105@oracle.com> <1417076703.3195.1.camel@oracle.com> Message-ID: <1417094274.21956.3.camel@oracle.com> Hi all, On Thu, 2014-11-27 at 09:25 +0100, Thomas Schatzl wrote: > Hi, > > On Thu, 2014-11-27 at 02:10 +0300, Leonid Mesnik wrote: > [...] > > Could I have review from Reviewer for this fix? > > - gcCause.cpp: > > Is it possible to let the human readable string be similar to the > others? I.e. "WhiteBox Initiated Concurrent Mark" instead of "WhiteBox > Initiated ConcurrentMarkCycle"? > > - WhiteBox.java: > > In the comment, please start sentences with capital letters. Also add > punctuation if it is a full sentence. All but this and the "force xy GC" > follow this style. > > - TestConcMarkCycleWB.java > > The test needs to install sun.hotspot.WhiteBox$WhiteBoxPermission too. > There is a typo in the @summary line: relatd->related > Leonid asked me to put up a new webrev with the suggested fixes plus fixes of the copyright header in the test, and addition of the @bug tag in the same. Here is the new webrev: http://cr.openjdk.java.net/~tschatzl/8065579-leonid/webrev/ I will sponsor the change. Thanks, Thomas From michail.chernov at oracle.com Thu Nov 27 13:28:49 2014 From: michail.chernov at oracle.com (Michail Chernov) Date: Thu, 27 Nov 2014 16:28:49 +0300 Subject: RFR: 8064909: FragmentMetaspace.java got OutOfMemoryError In-Reply-To: <54765B70.10509@oracle.com> References: <5475D74A.2060907@oracle.com> <54762451.3070802@oracle.com> <54763D54.3070704@oracle.com> <54765B70.10509@oracle.com> Message-ID: <54772711.6000003@oracle.com> Hi, CC'ed hotspot-runtime-dev. Here is not test failure - test works as expected. OOME is occurred in compiler instance. private JavaCompiler javac; ... javac = ToolProvider.getSystemJavaCompiler(); ... int exitcode = javac.run(null, null, null, file.getCanonicalPath()); if (exitcode != 0) { throw new RuntimeException("javac failure when compiling: " + file.getCanonicalPath()); Here is 2 ways - rewrite getGeneratedClass (runtime/testlibrary/GeneratedClassLoader.java) to allow them to throw not only RuntimeException, or to catch RuntimeException and check exception message comparing with "javac failure when compiling:". Both ways seem to me are not as clear as expected for this simple test. More - javac does not throw anything - it just returns exitcode (non-zero) and writes its messages to System.err. Also I can add comment to code like "OOME with message "java.lang.OutOfMemoryError: Java heap space" doesn't mean that something wrong with metaspace - need just to increase -Xmx". Thanks, Michail On 27.11.2014 2:00, Jon Masamitsu wrote: > Dima, > > If this test fails with an OOME in the future, I would like it to be > obvious that the failure is not that an OOME occurred. I cannot > tell that from looking at the test. Can the test be changed so > I don't have to spend time figuring out that the OOME is not > a failure mode of the test? > > Jon > > > On 11/26/2014 12:51 PM, Dmitry Fazunenko wrote: >> Hi Jon, >> >> The original version of test worked for 80 seconds trying to perform >> as many iterations as possible. The number of iterations performed >> depended on how fast is the machine. With each next iteration the >> size of generated and loaded classes is growing, so on fast enough >> machines 80 seconds is enough to run out of heap while generating a >> class. >> >> The fix not only sets the heap, but limits iterations. 300m heap is >> enough for 200 iterations. >> >> Your approach, with catching OOME(heap) and passing will also work, >> but it will reduce the test readability (and potentially could bring >> more problems). >> >> An alternative approach would be to limit metaspace and heap >> accordingly and load classes until we don't run out metaspace... But >> this might take awhile. >> >> So, I hope that Michael's fix is good. >> >> Thanks for looking and expressing comments. >> Dima >> >> >> >> >> On 26.11.2014 22:04, Jon Masamitsu wrote: >>> Michail, >>> >>> Your change makes this test pass but it seems like at >>> some future date 300m might not be big enough >>> (for whatever reason). Could the test be make to >>> caught an OOME, print out a message saying that >>> an OOME doesn't mean the test failed but that >>> the test needs a larger heap? Then pass an >>> exception up (maybe some type of Runtime >>> exception - sorry if that is vague but I don't >>> what type of exception would make sense). That >>> would mean we wouldn't have to spend time >>> diagnosing what the OOME means again. >>> >>> Jon >>> >>> On 11/26/2014 5:36 AM, Michail Chernov wrote: >>>> Hi, >>>> >>>> Please review this simple fix for nightly test failure: >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eistepan/~mchernov/8064909/webrev.00/ >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8064909 >>>> >>>> Problem: test fails because of OOME (not enough heap size). >>>> Solution: heap size were increased. >>>> >>>> Testing: >>>> jtreg >>>> >>>> Thanks, >>>> Michail >>> >> > > > From Leonid.Mesnik at oracle.com Thu Nov 27 13:38:51 2014 From: Leonid.Mesnik at oracle.com (Leonid Mesnik) Date: Thu, 27 Nov 2014 16:38:51 +0300 Subject: RFR 8065579: WB method to start G1 concurrent mark cycle should be introduced In-Reply-To: <1417094274.21956.3.camel@oracle.com> References: <546E0530.4060904@oracle.com> <54732343.6000307@oracle.com> <5475EA19.4080004@oracle.com> <547606B1.2070403@oracle.com> <54765DFD.5090105@oracle.com> <1417076703.3195.1.camel@oracle.com> <1417094274.21956.3.camel@oracle.com> Message-ID: <5477296B.5060107@oracle.com> Thank you for reviewing and pushing fix. Leonid On 27.11.2014 16:17, Thomas Schatzl wrote: > Hi all, > > On Thu, 2014-11-27 at 09:25 +0100, Thomas Schatzl wrote: >> Hi, >> >> On Thu, 2014-11-27 at 02:10 +0300, Leonid Mesnik wrote: >> [...] >>> Could I have review from Reviewer for this fix? >> - gcCause.cpp: >> >> Is it possible to let the human readable string be similar to the >> others? I.e. "WhiteBox Initiated Concurrent Mark" instead of "WhiteBox >> Initiated ConcurrentMarkCycle"? >> >> - WhiteBox.java: >> >> In the comment, please start sentences with capital letters. Also add >> punctuation if it is a full sentence. All but this and the "force xy GC" >> follow this style. >> >> - TestConcMarkCycleWB.java >> >> The test needs to install sun.hotspot.WhiteBox$WhiteBoxPermission too. >> There is a typo in the @summary line: relatd->related >> > Leonid asked me to put up a new webrev with the suggested fixes plus > fixes of the copyright header in the test, and addition of the @bug tag > in the same. > > Here is the new webrev: > http://cr.openjdk.java.net/~tschatzl/8065579-leonid/webrev/ > > I will sponsor the change. > > Thanks, > Thomas > > From mikael.gerdin at oracle.com Thu Nov 27 13:56:56 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 27 Nov 2014 14:56:56 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <5475DAB8.7000508@oracle.com> References: <5475C303.5060402@oracle.com> <5475D8DF.80404@oracle.com> <5475DAB8.7000508@oracle.com> Message-ID: <54772DA8.3000206@oracle.com> Hi Bengt, On 2014-11-26 14:50, Bengt Rutisson wrote: > > Hi Stefan, > > Thanks for looking at this! > > On 2014-11-26 14:42, Stefan Karlsson wrote: >> Hi Bengt, >> >> The changes looks good. There are a couple of follow-up cleanups that >> can be done as separate changes, but I don't think they are blocking >> this patch. > > Agreed. I filed two cleanup bugs based on Mikael's review. > >> >> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/test/gc/startup_warnings/TestParNewSerialOld.java.frames.html >> >> >> I think the summary should be changed to: >> >> 28 * @summary Test that the unsupported ParNew+SerialOld combination >> does not start > > Good catch. > > Here's an updated webrev that fixes the summary and also removes the > code that Mikael found: > > http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01/ Excellent! 600+ deleted lines! Looks good. /Mikael > > Only the changes compared to the last webrev: > > http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00-01.diff/ > > Thanks, > Bengt > >> >> >> StefanK >> >> On 2014-11-26 13:09, Bengt Rutisson wrote: >>> >>> Hi everyone, >>> >>> Can I have a couple of reviews for this change to remove the support >>> for ParNew+SerialOld and DefNew+CMS? >>> >>> This work is part of JEP 214 (http://openjdk.java.net/jeps/214). >>> >>> The main change is in the hotspot repo: >>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ >>> >>> I also removed the ParNew runs from the jprt configuration: >>> http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ >>> >>> JBS bug: >>> https://bugs.openjdk.java.net/browse/JDK-8065972 >>> >>> Some notes about the change: >>> >>> - UseParNewGC is now a redundant flag. It can only be used with CMS, >>> so it will always have the same value as UseConcMarkSweepGC. Since >>> UseParNewGC was not one of the flags that were deprecated in JDK 8, >>> we decided to leave the flag in for JDK 9 but print a warning message >>> that it is deprecated. We plan on removing UseParNewGC in JDK 10. >>> >>> Here's how the VM will behave with different values of UseParNewGC >>> after my change: >>> >>> $ java -XX:+UseParNewGC -version >>> It is not possible to combine the ParNew young collector with the >>> Serial old collector. >>> Error: Could not create the Java Virtual Machine. >>> Error: A fatal exception has occurred. Program will exit. >>> >>> $ java -XX:-UseParNewGC -version >>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>> deprecated and will likely be removed in a future release >>> java version "1.9.0-internal-debug" >>> Java(TM) SE Runtime Environment (build >>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>> Java HotSpot(TM) 64-Bit Server VM (build >>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>> >>> $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version >>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>> deprecated and will likely be removed in a future release >>> java version "1.9.0-internal-debug" >>> Java(TM) SE Runtime Environment (build >>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>> Java HotSpot(TM) 64-Bit Server VM (build >>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>> >>> $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version >>> It is not possible to combine the DefNew young collector with the CMS >>> collector. >>> Error: Could not create the Java Virtual Machine. >>> Error: A fatal exception has occurred. Program will exit. >>> >>> - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() >>> methods will exit if they fail. No need to check for null after those >>> calls. This simplified the two implementations significantly. >>> >>> - At a few places I've changed from "DefNewGeneration* dng = >>> (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = >>> _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method >>> does the asserts that were often placed before the original casting >>> code. >>> >>> - In concurrentMarkSweepGeneration.cpp I left the casting to >>> DefNewGeneration even though we could now use the more specific >>> ParNewGeneration. I prefer to do such an update later (if at all) >>> since that might come with its own complications regarding virtual >>> calls etc. >>> >>> - The two methods must_be_youngest() and must_be_oldest() were >>> unused. Removing them was not strictly needed for this change, but >>> seemed like a nice thing to do anyway and I don't think it clutters >>> the webrev much. >>> >>> - UseParNewGC is now always true when UseConcMarkSweepGC is true. And >>> it is always false when UseConcMarkSweepGC is false. If statements >>> that used to test against both flags have been simplified to only >>> test against UseConcMarkSweepGC. This means that UseParNewGC is now >>> only used in arguments.cpp >>> >>> - Similarly JTreg tests that used UseParNewGC have been changed to >>> use UseConcMarkSweepGC instead. This should make it easy to remove in >>> JDK 10. >>> >>> - The change to make/jprt.properties only affects the internal >>> testing system we have. It will no longer test the ParNew+SerialOld >>> combination (which is no longer supported). Note that DefNew+CMS was >>> not tested at all by JPRT. >>> >>> Thanks, >>> Bengt >> > From bengt.rutisson at oracle.com Thu Nov 27 14:01:40 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 27 Nov 2014 15:01:40 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <54772DA8.3000206@oracle.com> References: <5475C303.5060402@oracle.com> <5475D8DF.80404@oracle.com> <5475DAB8.7000508@oracle.com> <54772DA8.3000206@oracle.com> Message-ID: <54772EC4.8030108@oracle.com> Hi Mikael, Thanks for the review! StefanK found some more code to delete. :) Here's an updated webrev: http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.02/ And here's the diff compared to the last one: cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01-02.diff/ Thanks, Bengt On 2014-11-27 14:56, Mikael Gerdin wrote: > Hi Bengt, > > On 2014-11-26 14:50, Bengt Rutisson wrote: >> >> Hi Stefan, >> >> Thanks for looking at this! >> >> On 2014-11-26 14:42, Stefan Karlsson wrote: >>> Hi Bengt, >>> >>> The changes looks good. There are a couple of follow-up cleanups that >>> can be done as separate changes, but I don't think they are blocking >>> this patch. >> >> Agreed. I filed two cleanup bugs based on Mikael's review. >> >>> >>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/test/gc/startup_warnings/TestParNewSerialOld.java.frames.html >>> >>> >>> >>> I think the summary should be changed to: >>> >>> 28 * @summary Test that the unsupported ParNew+SerialOld combination >>> does not start >> >> Good catch. >> >> Here's an updated webrev that fixes the summary and also removes the >> code that Mikael found: >> >> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01/ > > Excellent! 600+ deleted lines! > > Looks good. > > /Mikael > >> >> Only the changes compared to the last webrev: >> >> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00-01.diff/ >> >> Thanks, >> Bengt >> >>> >>> >>> StefanK >>> >>> On 2014-11-26 13:09, Bengt Rutisson wrote: >>>> >>>> Hi everyone, >>>> >>>> Can I have a couple of reviews for this change to remove the support >>>> for ParNew+SerialOld and DefNew+CMS? >>>> >>>> This work is part of JEP 214 (http://openjdk.java.net/jeps/214). >>>> >>>> The main change is in the hotspot repo: >>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ >>>> >>>> I also removed the ParNew runs from the jprt configuration: >>>> http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ >>>> >>>> JBS bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8065972 >>>> >>>> Some notes about the change: >>>> >>>> - UseParNewGC is now a redundant flag. It can only be used with CMS, >>>> so it will always have the same value as UseConcMarkSweepGC. Since >>>> UseParNewGC was not one of the flags that were deprecated in JDK 8, >>>> we decided to leave the flag in for JDK 9 but print a warning message >>>> that it is deprecated. We plan on removing UseParNewGC in JDK 10. >>>> >>>> Here's how the VM will behave with different values of UseParNewGC >>>> after my change: >>>> >>>> $ java -XX:+UseParNewGC -version >>>> It is not possible to combine the ParNew young collector with the >>>> Serial old collector. >>>> Error: Could not create the Java Virtual Machine. >>>> Error: A fatal exception has occurred. Program will exit. >>>> >>>> $ java -XX:-UseParNewGC -version >>>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>>> deprecated and will likely be removed in a future release >>>> java version "1.9.0-internal-debug" >>>> Java(TM) SE Runtime Environment (build >>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>>> Java HotSpot(TM) 64-Bit Server VM (build >>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>>> >>>> $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version >>>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>>> deprecated and will likely be removed in a future release >>>> java version "1.9.0-internal-debug" >>>> Java(TM) SE Runtime Environment (build >>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>>> Java HotSpot(TM) 64-Bit Server VM (build >>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>>> >>>> $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version >>>> It is not possible to combine the DefNew young collector with the CMS >>>> collector. >>>> Error: Could not create the Java Virtual Machine. >>>> Error: A fatal exception has occurred. Program will exit. >>>> >>>> - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() >>>> methods will exit if they fail. No need to check for null after those >>>> calls. This simplified the two implementations significantly. >>>> >>>> - At a few places I've changed from "DefNewGeneration* dng = >>>> (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = >>>> _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method >>>> does the asserts that were often placed before the original casting >>>> code. >>>> >>>> - In concurrentMarkSweepGeneration.cpp I left the casting to >>>> DefNewGeneration even though we could now use the more specific >>>> ParNewGeneration. I prefer to do such an update later (if at all) >>>> since that might come with its own complications regarding virtual >>>> calls etc. >>>> >>>> - The two methods must_be_youngest() and must_be_oldest() were >>>> unused. Removing them was not strictly needed for this change, but >>>> seemed like a nice thing to do anyway and I don't think it clutters >>>> the webrev much. >>>> >>>> - UseParNewGC is now always true when UseConcMarkSweepGC is true. And >>>> it is always false when UseConcMarkSweepGC is false. If statements >>>> that used to test against both flags have been simplified to only >>>> test against UseConcMarkSweepGC. This means that UseParNewGC is now >>>> only used in arguments.cpp >>>> >>>> - Similarly JTreg tests that used UseParNewGC have been changed to >>>> use UseConcMarkSweepGC instead. This should make it easy to remove in >>>> JDK 10. >>>> >>>> - The change to make/jprt.properties only affects the internal >>>> testing system we have. It will no longer test the ParNew+SerialOld >>>> combination (which is no longer supported). Note that DefNew+CMS was >>>> not tested at all by JPRT. >>>> >>>> Thanks, >>>> Bengt >>> >> From stefan.karlsson at oracle.com Thu Nov 27 14:03:14 2014 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 27 Nov 2014 15:03:14 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <54772EC4.8030108@oracle.com> References: <5475C303.5060402@oracle.com> <5475D8DF.80404@oracle.com> <5475DAB8.7000508@oracle.com> <54772DA8.3000206@oracle.com> <54772EC4.8030108@oracle.com> Message-ID: <54772F22.7040900@oracle.com> On 2014-11-27 15:01, Bengt Rutisson wrote: > > Hi Mikael, > > Thanks for the review! > > StefanK found some more code to delete. :) > > Here's an updated webrev: > http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.02/ > > And here's the diff compared to the last one: > cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01-02.diff/ Looks good. :) StefanK > > Thanks, > Bengt > > > On 2014-11-27 14:56, Mikael Gerdin wrote: >> Hi Bengt, >> >> On 2014-11-26 14:50, Bengt Rutisson wrote: >>> >>> Hi Stefan, >>> >>> Thanks for looking at this! >>> >>> On 2014-11-26 14:42, Stefan Karlsson wrote: >>>> Hi Bengt, >>>> >>>> The changes looks good. There are a couple of follow-up cleanups that >>>> can be done as separate changes, but I don't think they are blocking >>>> this patch. >>> >>> Agreed. I filed two cleanup bugs based on Mikael's review. >>> >>>> >>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/test/gc/startup_warnings/TestParNewSerialOld.java.frames.html >>>> >>>> >>>> >>>> I think the summary should be changed to: >>>> >>>> 28 * @summary Test that the unsupported ParNew+SerialOld combination >>>> does not start >>> >>> Good catch. >>> >>> Here's an updated webrev that fixes the summary and also removes the >>> code that Mikael found: >>> >>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01/ >> >> Excellent! 600+ deleted lines! >> >> Looks good. >> >> /Mikael >> >>> >>> Only the changes compared to the last webrev: >>> >>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00-01.diff/ >>> >>> Thanks, >>> Bengt >>> >>>> >>>> >>>> StefanK >>>> >>>> On 2014-11-26 13:09, Bengt Rutisson wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> Can I have a couple of reviews for this change to remove the support >>>>> for ParNew+SerialOld and DefNew+CMS? >>>>> >>>>> This work is part of JEP 214 (http://openjdk.java.net/jeps/214). >>>>> >>>>> The main change is in the hotspot repo: >>>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ >>>>> >>>>> I also removed the ParNew runs from the jprt configuration: >>>>> http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ >>>>> >>>>> JBS bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8065972 >>>>> >>>>> Some notes about the change: >>>>> >>>>> - UseParNewGC is now a redundant flag. It can only be used with CMS, >>>>> so it will always have the same value as UseConcMarkSweepGC. Since >>>>> UseParNewGC was not one of the flags that were deprecated in JDK 8, >>>>> we decided to leave the flag in for JDK 9 but print a warning message >>>>> that it is deprecated. We plan on removing UseParNewGC in JDK 10. >>>>> >>>>> Here's how the VM will behave with different values of UseParNewGC >>>>> after my change: >>>>> >>>>> $ java -XX:+UseParNewGC -version >>>>> It is not possible to combine the ParNew young collector with the >>>>> Serial old collector. >>>>> Error: Could not create the Java Virtual Machine. >>>>> Error: A fatal exception has occurred. Program will exit. >>>>> >>>>> $ java -XX:-UseParNewGC -version >>>>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>>>> deprecated and will likely be removed in a future release >>>>> java version "1.9.0-internal-debug" >>>>> Java(TM) SE Runtime Environment (build >>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>>>> Java HotSpot(TM) 64-Bit Server VM (build >>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>>>> >>>>> $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version >>>>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>>>> deprecated and will likely be removed in a future release >>>>> java version "1.9.0-internal-debug" >>>>> Java(TM) SE Runtime Environment (build >>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>>>> Java HotSpot(TM) 64-Bit Server VM (build >>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>>>> >>>>> $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version >>>>> It is not possible to combine the DefNew young collector with the CMS >>>>> collector. >>>>> Error: Could not create the Java Virtual Machine. >>>>> Error: A fatal exception has occurred. Program will exit. >>>>> >>>>> - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() >>>>> methods will exit if they fail. No need to check for null after those >>>>> calls. This simplified the two implementations significantly. >>>>> >>>>> - At a few places I've changed from "DefNewGeneration* dng = >>>>> (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = >>>>> _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method >>>>> does the asserts that were often placed before the original casting >>>>> code. >>>>> >>>>> - In concurrentMarkSweepGeneration.cpp I left the casting to >>>>> DefNewGeneration even though we could now use the more specific >>>>> ParNewGeneration. I prefer to do such an update later (if at all) >>>>> since that might come with its own complications regarding virtual >>>>> calls etc. >>>>> >>>>> - The two methods must_be_youngest() and must_be_oldest() were >>>>> unused. Removing them was not strictly needed for this change, but >>>>> seemed like a nice thing to do anyway and I don't think it clutters >>>>> the webrev much. >>>>> >>>>> - UseParNewGC is now always true when UseConcMarkSweepGC is true. And >>>>> it is always false when UseConcMarkSweepGC is false. If statements >>>>> that used to test against both flags have been simplified to only >>>>> test against UseConcMarkSweepGC. This means that UseParNewGC is now >>>>> only used in arguments.cpp >>>>> >>>>> - Similarly JTreg tests that used UseParNewGC have been changed to >>>>> use UseConcMarkSweepGC instead. This should make it easy to remove in >>>>> JDK 10. >>>>> >>>>> - The change to make/jprt.properties only affects the internal >>>>> testing system we have. It will no longer test the ParNew+SerialOld >>>>> combination (which is no longer supported). Note that DefNew+CMS was >>>>> not tested at all by JPRT. >>>>> >>>>> Thanks, >>>>> Bengt >>>> >>> > From mikael.gerdin at oracle.com Thu Nov 27 14:05:28 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 27 Nov 2014 15:05:28 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <54772EC4.8030108@oracle.com> References: <5475C303.5060402@oracle.com> <5475D8DF.80404@oracle.com> <5475DAB8.7000508@oracle.com> <54772DA8.3000206@oracle.com> <54772EC4.8030108@oracle.com> Message-ID: <54772FA8.4080707@oracle.com> Bengt, On 2014-11-27 15:01, Bengt Rutisson wrote: > > Hi Mikael, > > Thanks for the review! > > StefanK found some more code to delete. :) > > Here's an updated webrev: > http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.02/ > > And here's the diff compared to the last one: > cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01-02.diff/ Looks good. /Mikael > > Thanks, > Bengt > > > On 2014-11-27 14:56, Mikael Gerdin wrote: >> Hi Bengt, >> >> On 2014-11-26 14:50, Bengt Rutisson wrote: >>> >>> Hi Stefan, >>> >>> Thanks for looking at this! >>> >>> On 2014-11-26 14:42, Stefan Karlsson wrote: >>>> Hi Bengt, >>>> >>>> The changes looks good. There are a couple of follow-up cleanups that >>>> can be done as separate changes, but I don't think they are blocking >>>> this patch. >>> >>> Agreed. I filed two cleanup bugs based on Mikael's review. >>> >>>> >>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/test/gc/startup_warnings/TestParNewSerialOld.java.frames.html >>>> >>>> >>>> >>>> I think the summary should be changed to: >>>> >>>> 28 * @summary Test that the unsupported ParNew+SerialOld combination >>>> does not start >>> >>> Good catch. >>> >>> Here's an updated webrev that fixes the summary and also removes the >>> code that Mikael found: >>> >>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01/ >> >> Excellent! 600+ deleted lines! >> >> Looks good. >> >> /Mikael >> >>> >>> Only the changes compared to the last webrev: >>> >>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00-01.diff/ >>> >>> Thanks, >>> Bengt >>> >>>> >>>> >>>> StefanK >>>> >>>> On 2014-11-26 13:09, Bengt Rutisson wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> Can I have a couple of reviews for this change to remove the support >>>>> for ParNew+SerialOld and DefNew+CMS? >>>>> >>>>> This work is part of JEP 214 (http://openjdk.java.net/jeps/214). >>>>> >>>>> The main change is in the hotspot repo: >>>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ >>>>> >>>>> I also removed the ParNew runs from the jprt configuration: >>>>> http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ >>>>> >>>>> JBS bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8065972 >>>>> >>>>> Some notes about the change: >>>>> >>>>> - UseParNewGC is now a redundant flag. It can only be used with CMS, >>>>> so it will always have the same value as UseConcMarkSweepGC. Since >>>>> UseParNewGC was not one of the flags that were deprecated in JDK 8, >>>>> we decided to leave the flag in for JDK 9 but print a warning message >>>>> that it is deprecated. We plan on removing UseParNewGC in JDK 10. >>>>> >>>>> Here's how the VM will behave with different values of UseParNewGC >>>>> after my change: >>>>> >>>>> $ java -XX:+UseParNewGC -version >>>>> It is not possible to combine the ParNew young collector with the >>>>> Serial old collector. >>>>> Error: Could not create the Java Virtual Machine. >>>>> Error: A fatal exception has occurred. Program will exit. >>>>> >>>>> $ java -XX:-UseParNewGC -version >>>>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>>>> deprecated and will likely be removed in a future release >>>>> java version "1.9.0-internal-debug" >>>>> Java(TM) SE Runtime Environment (build >>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>>>> Java HotSpot(TM) 64-Bit Server VM (build >>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>>>> >>>>> $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version >>>>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>>>> deprecated and will likely be removed in a future release >>>>> java version "1.9.0-internal-debug" >>>>> Java(TM) SE Runtime Environment (build >>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>>>> Java HotSpot(TM) 64-Bit Server VM (build >>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>>>> >>>>> $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version >>>>> It is not possible to combine the DefNew young collector with the CMS >>>>> collector. >>>>> Error: Could not create the Java Virtual Machine. >>>>> Error: A fatal exception has occurred. Program will exit. >>>>> >>>>> - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() >>>>> methods will exit if they fail. No need to check for null after those >>>>> calls. This simplified the two implementations significantly. >>>>> >>>>> - At a few places I've changed from "DefNewGeneration* dng = >>>>> (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = >>>>> _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() method >>>>> does the asserts that were often placed before the original casting >>>>> code. >>>>> >>>>> - In concurrentMarkSweepGeneration.cpp I left the casting to >>>>> DefNewGeneration even though we could now use the more specific >>>>> ParNewGeneration. I prefer to do such an update later (if at all) >>>>> since that might come with its own complications regarding virtual >>>>> calls etc. >>>>> >>>>> - The two methods must_be_youngest() and must_be_oldest() were >>>>> unused. Removing them was not strictly needed for this change, but >>>>> seemed like a nice thing to do anyway and I don't think it clutters >>>>> the webrev much. >>>>> >>>>> - UseParNewGC is now always true when UseConcMarkSweepGC is true. And >>>>> it is always false when UseConcMarkSweepGC is false. If statements >>>>> that used to test against both flags have been simplified to only >>>>> test against UseConcMarkSweepGC. This means that UseParNewGC is now >>>>> only used in arguments.cpp >>>>> >>>>> - Similarly JTreg tests that used UseParNewGC have been changed to >>>>> use UseConcMarkSweepGC instead. This should make it easy to remove in >>>>> JDK 10. >>>>> >>>>> - The change to make/jprt.properties only affects the internal >>>>> testing system we have. It will no longer test the ParNew+SerialOld >>>>> combination (which is no longer supported). Note that DefNew+CMS was >>>>> not tested at all by JPRT. >>>>> >>>>> Thanks, >>>>> Bengt >>>> >>> > From bengt.rutisson at oracle.com Thu Nov 27 20:19:34 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 27 Nov 2014 21:19:34 +0100 Subject: RFR (L): 8065972: Remove support for ParNew+SerialOld and DefNew+CMS In-Reply-To: <54772F22.7040900@oracle.com> References: <5475C303.5060402@oracle.com> <5475D8DF.80404@oracle.com> <5475DAB8.7000508@oracle.com> <54772DA8.3000206@oracle.com> <54772EC4.8030108@oracle.com> <54772F22.7040900@oracle.com> Message-ID: <54778756.3070104@oracle.com> Thanks for the reviews Mikael and Stefan! Pushing this now. Bengt On 11/27/14 3:03 PM, Stefan Karlsson wrote: > On 2014-11-27 15:01, Bengt Rutisson wrote: >> >> Hi Mikael, >> >> Thanks for the review! >> >> StefanK found some more code to delete. :) >> >> Here's an updated webrev: >> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.02/ >> >> And here's the diff compared to the last one: >> cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01-02.diff/ > > Looks good. :) > > StefanK > >> >> Thanks, >> Bengt >> >> >> On 2014-11-27 14:56, Mikael Gerdin wrote: >>> Hi Bengt, >>> >>> On 2014-11-26 14:50, Bengt Rutisson wrote: >>>> >>>> Hi Stefan, >>>> >>>> Thanks for looking at this! >>>> >>>> On 2014-11-26 14:42, Stefan Karlsson wrote: >>>>> Hi Bengt, >>>>> >>>>> The changes looks good. There are a couple of follow-up cleanups that >>>>> can be done as separate changes, but I don't think they are blocking >>>>> this patch. >>>> >>>> Agreed. I filed two cleanup bugs based on Mikael's review. >>>> >>>>> >>>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/test/gc/startup_warnings/TestParNewSerialOld.java.frames.html >>>>> >>>>> >>>>> >>>>> I think the summary should be changed to: >>>>> >>>>> 28 * @summary Test that the unsupported ParNew+SerialOld >>>>> combination >>>>> does not start >>>> >>>> Good catch. >>>> >>>> Here's an updated webrev that fixes the summary and also removes the >>>> code that Mikael found: >>>> >>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.01/ >>> >>> Excellent! 600+ deleted lines! >>> >>> Looks good. >>> >>> /Mikael >>> >>>> >>>> Only the changes compared to the last webrev: >>>> >>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00-01.diff/ >>>> >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> >>>>> >>>>> StefanK >>>>> >>>>> On 2014-11-26 13:09, Bengt Rutisson wrote: >>>>>> >>>>>> Hi everyone, >>>>>> >>>>>> Can I have a couple of reviews for this change to remove the support >>>>>> for ParNew+SerialOld and DefNew+CMS? >>>>>> >>>>>> This work is part of JEP 214 (http://openjdk.java.net/jeps/214). >>>>>> >>>>>> The main change is in the hotspot repo: >>>>>> http://cr.openjdk.java.net/~brutisso/8065972/hotspot-webrev.00/ >>>>>> >>>>>> I also removed the ParNew runs from the jprt configuration: >>>>>> http://cr.openjdk.java.net/~brutisso/8065972/root-webrev.00/ >>>>>> >>>>>> JBS bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8065972 >>>>>> >>>>>> Some notes about the change: >>>>>> >>>>>> - UseParNewGC is now a redundant flag. It can only be used with CMS, >>>>>> so it will always have the same value as UseConcMarkSweepGC. Since >>>>>> UseParNewGC was not one of the flags that were deprecated in JDK 8, >>>>>> we decided to leave the flag in for JDK 9 but print a warning >>>>>> message >>>>>> that it is deprecated. We plan on removing UseParNewGC in JDK 10. >>>>>> >>>>>> Here's how the VM will behave with different values of UseParNewGC >>>>>> after my change: >>>>>> >>>>>> $ java -XX:+UseParNewGC -version >>>>>> It is not possible to combine the ParNew young collector with the >>>>>> Serial old collector. >>>>>> Error: Could not create the Java Virtual Machine. >>>>>> Error: A fatal exception has occurred. Program will exit. >>>>>> >>>>>> $ java -XX:-UseParNewGC -version >>>>>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>>>>> deprecated and will likely be removed in a future release >>>>>> java version "1.9.0-internal-debug" >>>>>> Java(TM) SE Runtime Environment (build >>>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>>>>> Java HotSpot(TM) 64-Bit Server VM (build >>>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>>>>> >>>>>> $ java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -version >>>>>> Java HotSpot(TM) 64-Bit Server VM warning: The UseParNewGC flag is >>>>>> deprecated and will likely be removed in a future release >>>>>> java version "1.9.0-internal-debug" >>>>>> Java(TM) SE Runtime Environment (build >>>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00) >>>>>> Java HotSpot(TM) 64-Bit Server VM (build >>>>>> 1.9.0-internal-debug-brutisso_2014_11_05_09_40-b00, mixed mode) >>>>>> >>>>>> $ java -XX:-UseParNewGC -XX:+UseConcMarkSweepGC -version >>>>>> It is not possible to combine the DefNew young collector with the >>>>>> CMS >>>>>> collector. >>>>>> Error: Could not create the Java Virtual Machine. >>>>>> Error: A fatal exception has occurred. Program will exit. >>>>>> >>>>>> - The NEW_C_HEAP_ARRAY and "new" calls in initialize_generations() >>>>>> methods will exit if they fail. No need to check for null after >>>>>> those >>>>>> calls. This simplified the two implementations significantly. >>>>>> >>>>>> - At a few places I've changed from "DefNewGeneration* dng = >>>>>> (DefNewGeneration*)_young_gen;" to "DefNewGeneration* dng = >>>>>> _young_gen->as_DefNewGeneration();". The as_DefNewGeneration() >>>>>> method >>>>>> does the asserts that were often placed before the original casting >>>>>> code. >>>>>> >>>>>> - In concurrentMarkSweepGeneration.cpp I left the casting to >>>>>> DefNewGeneration even though we could now use the more specific >>>>>> ParNewGeneration. I prefer to do such an update later (if at all) >>>>>> since that might come with its own complications regarding virtual >>>>>> calls etc. >>>>>> >>>>>> - The two methods must_be_youngest() and must_be_oldest() were >>>>>> unused. Removing them was not strictly needed for this change, but >>>>>> seemed like a nice thing to do anyway and I don't think it clutters >>>>>> the webrev much. >>>>>> >>>>>> - UseParNewGC is now always true when UseConcMarkSweepGC is true. >>>>>> And >>>>>> it is always false when UseConcMarkSweepGC is false. If statements >>>>>> that used to test against both flags have been simplified to only >>>>>> test against UseConcMarkSweepGC. This means that UseParNewGC is now >>>>>> only used in arguments.cpp >>>>>> >>>>>> - Similarly JTreg tests that used UseParNewGC have been changed to >>>>>> use UseConcMarkSweepGC instead. This should make it easy to >>>>>> remove in >>>>>> JDK 10. >>>>>> >>>>>> - The change to make/jprt.properties only affects the internal >>>>>> testing system we have. It will no longer test the ParNew+SerialOld >>>>>> combination (which is no longer supported). Note that DefNew+CMS was >>>>>> not tested at all by JPRT. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>> >>>> >> > From thomas.schatzl at oracle.com Fri Nov 28 08:46:25 2014 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 28 Nov 2014 09:46:25 +0100 Subject: RFR (S): 8066102: Clean up HeapRegionRemSet files Message-ID: <1417164385.3364.2.camel@oracle.com> Hi all, can I have reviews for this small cleanup? I went over heap region remembered set files recently and found dead code, useless code, tightened the public interface and tried to document the methods a little. I also moved FromCardCache accessor methods from OtherRegionTable to HeapRegionRemSet to do away with that needless indirection. Imo it fits better that way too. CR: https://bugs.openjdk.java.net/browse/JDK-8066102 Webrev: http://cr.openjdk.java.net/~tschatzl/8066102/webrev/ Testing: jprt Thanks, Thomas From mikael.gerdin at oracle.com Fri Nov 28 12:10:00 2014 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 28 Nov 2014 13:10:00 +0100 Subject: RFR (S): 8066102: Clean up HeapRegionRemSet files In-Reply-To: <1417164385.3364.2.camel@oracle.com> References: <1417164385.3364.2.camel@oracle.com> Message-ID: <54786618.7070008@oracle.com> Hi Thomas, On 2014-11-28 09:46, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small cleanup? I went over heap region > remembered set files recently and found dead code, useless code, > tightened the public interface and tried to document the methods a > little. I also moved FromCardCache accessor methods from > OtherRegionTable to HeapRegionRemSet to do away with that needless > indirection. Imo it fits better that way too. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8066102 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8066102/webrev/ In heapRegionRemSet.hpp: + // One bits in the bitmaps indicate that the given region or card is live. typo: "One bit in one of the bitmaps indicate that the.." Otherwise the change looks good to me. /Mikael > > Testing: > jprt > > Thanks, > Thomas > From jesper.wilhelmsson at oracle.com Fri Nov 28 12:32:13 2014 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 28 Nov 2014 13:32:13 +0100 Subject: RFR (S): 8066102: Clean up HeapRegionRemSet files In-Reply-To: <1417164385.3364.2.camel@oracle.com> References: <1417164385.3364.2.camel@oracle.com> Message-ID: <54786B4D.8070200@oracle.com> Looks good! /Jesper Thomas Schatzl skrev 28/11/14 09:46: > Hi all, > > can I have reviews for this small cleanup? I went over heap region > remembered set files recently and found dead code, useless code, > tightened the public interface and tried to document the methods a > little. I also moved FromCardCache accessor methods from > OtherRegionTable to HeapRegionRemSet to do away with that needless > indirection. Imo it fits better that way too. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8066102 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8066102/webrev/ > > Testing: > jprt > > Thanks, > Thomas > From kim.barrett at oracle.com Fri Nov 28 21:18:50 2014 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 28 Nov 2014 16:18:50 -0500 Subject: RFR (S): 8066102: Clean up HeapRegionRemSet files In-Reply-To: <1417164385.3364.2.camel@oracle.com> References: <1417164385.3364.2.camel@oracle.com> Message-ID: <5BEE7051-32F8-4D15-8EB9-C413947FA524@oracle.com> [resending from proper from address, so it will reach mailing list] On Nov 28, 2014, at 3:46 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this small cleanup? I went over heap region > remembered set files recently and found dead code, useless code, > tightened the public interface and tried to document the methods a > little. I also moved FromCardCache accessor methods from > OtherRegionTable to HeapRegionRemSet to do away with that needless > indirection. Imo it fits better that way too. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8066102 > > Webrev: > http://cr.openjdk.java.net/~tschatzl/8066102/webrev/ Looks good. This change surprised me though: src/share/vm/gc_implementation/g1/heapRegionRemSet.cpp 978 _bosa(hrrs->bosa()), => 943 _bosa(hrrs->_bosa), Violates the Hotspot coding guidelines item about member access from outside the class be through accessor functions. OTOH, the class being accessed has befriended the class doing this access, and the (removed by this changeset) bosa() accessor was private. Regarding Mikael's comment: > In heapRegionRemSet.hpp: > + // One bits in the bitmaps indicate that the given region or card is live. > typo: "One bit in one of the bitmaps indicate that the.." I think the existing comment is correct, but might be clearer if it said "Set bits in the bitmaps ..."