From vladimir.kozlov at oracle.com Sat Nov 1 00:55:14 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 31 Oct 2014 17:55:14 -0700 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms Message-ID: <54542F72.3000609@oracle.com> http://cr.openjdk.java.net/~kvn/8059780/webrev/ https://bugs.openjdk.java.net/browse/JDK-8059780 We found performance regression (3-5%) on x64 after changes 8052081 [1] were pushed. It is caused by changes in lcm.cpp. Those changes were done because Atom is in-order cpu and some benchmarks on Atom showed benefit from these changes. But Atom is not our core platform and the regression is big. Backout changes made in lcm.cpp in jdk9 and 8u40 (8052081 was backported to 8u40). We have RFEs to improve block's instructions order which may help both platforms: https://bugs.openjdk.java.net/browse/JDK-6958833 https://bugs.openjdk.java.net/browse/JDK-7101232 Tested with SPECjvm2008-mpegaudio. Thanks, Vladimir [1] https://bugs.openjdk.java.net/browse/JDK-8052081 From vladimir.kozlov at oracle.com Sat Nov 1 00:58:39 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 31 Oct 2014 17:58:39 -0700 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <54542F72.3000609@oracle.com> References: <54542F72.3000609@oracle.com> Message-ID: <5454303F.8060805@oracle.com> I forgot to say that lcm.cpp changes were small part of 8052081 changes. The rest of changes will stay. Thanks, Vladimir On 10/31/14 5:55 PM, Vladimir Kozlov wrote: > http://cr.openjdk.java.net/~kvn/8059780/webrev/ > https://bugs.openjdk.java.net/browse/JDK-8059780 > > We found performance regression (3-5%) on x64 after changes 8052081 [1] > were pushed. It is caused by changes in lcm.cpp. Those changes were done > because Atom is in-order cpu and some benchmarks on Atom showed benefit > from these changes. > > But Atom is not our core platform and the regression is big. > > Backout changes made in lcm.cpp in jdk9 and 8u40 (8052081 was backported > to 8u40). > > We have RFEs to improve block's instructions order which may help both > platforms: > > https://bugs.openjdk.java.net/browse/JDK-6958833 > https://bugs.openjdk.java.net/browse/JDK-7101232 > > Tested with SPECjvm2008-mpegaudio. > > Thanks, > Vladimir > > [1] https://bugs.openjdk.java.net/browse/JDK-8052081 From igor.veresov at oracle.com Sat Nov 1 07:29:01 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Fri, 31 Oct 2014 21:29:01 -1000 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <54542F72.3000609@oracle.com> References: <54542F72.3000609@oracle.com> Message-ID: <32703F35-7EA5-42F3-B99A-DE88677D0C0D@oracle.com> Looks fine to me. igor On Oct 31, 2014, at 2:55 PM, Vladimir Kozlov wrote: > http://cr.openjdk.java.net/~kvn/8059780/webrev/ > https://bugs.openjdk.java.net/browse/JDK-8059780 > > We found performance regression (3-5%) on x64 after changes 8052081 [1] were pushed. It is caused by changes in lcm.cpp. Those changes were done because Atom is in-order cpu and some benchmarks on Atom showed benefit from these changes. > > But Atom is not our core platform and the regression is big. > > Backout changes made in lcm.cpp in jdk9 and 8u40 (8052081 was backported to 8u40). > > We have RFEs to improve block's instructions order which may help both platforms: > > https://bugs.openjdk.java.net/browse/JDK-6958833 > https://bugs.openjdk.java.net/browse/JDK-7101232 > > Tested with SPECjvm2008-mpegaudio. > > Thanks, > Vladimir > > [1] https://bugs.openjdk.java.net/browse/JDK-8052081 From david.r.chase at oracle.com Sat Nov 1 17:03:28 2014 From: david.r.chase at oracle.com (David Chase) Date: Sat, 1 Nov 2014 13:03:28 -0400 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <54550CE2.9090605@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <545406BC.20005@gmail.com> <54550CE2.9090605@gmail.com> Message-ID: <13FF3B65-027D-4152-8CEF-0F31773976EA@oracle.com> Hello Peter, I think it is expected that inserting-interns will be asymptotically rare ? classes have a finite number of methods, after all. I?m not sure if that is worth doing right now, since this is also a bug fix ? maybe the performance enhancements go in as an RFE. Maybe other reviewers will have an opinion? David On 2014-11-01, at 12:40 PM, Peter Levart wrote: > > On 10/31/2014 11:59 PM, David Chase wrote: >> Thanks very much, I shall attend to these irregularities. >> >> David >> > > Hi David, > > Just a nit (in Class.ClassData): > > 2537 if (oldCapacity > 0) { > 2538 element_data[oldCapacity] = element_data[oldCapacity - 1]; > 2539 // all array elements are non-null and sorted, increase size. > 2540 // if store to element_data above floats below > 2541 // store to size on the next line, that will be > 2542 // inconsistent to the VM if a safepoint occurs here. > 2543 size += 1; > 2544 for (int i = oldCapacity; i > index; i--) { > 2545 // pre: element_data[i] is duplicated at [i+1] > 2546 element_data[i] = element_data[i - 1]; > 2547 // post: element_data[i-1] is duplicated at [i] > 2548 } > 2549 // element_data[index] is duplicated at [index+1] > 2550 element_data[index] = (Comparable) e; > 2551 } else { > > In line 2544, you could start the for loop with (int i = oldCapacity - 1; ...), since you have already moved the last element before incrementing the size. Also, I would more quickly grasp the code if "oldCapacity" was called "oldSize". > > Now just a though... > > What is the expected ratio of intern() calls that insert new element to those that just return existing interned element? If those that return existing element are frequent and since you already carefully arrange insertion so that VM can at any safepoint see the "consistent" state without null elements, I wonder if intern() could itself perform an optimistic search without holding an exclusive lock. > > This is just a speculation, but would the following code work? > > private Comparable[] elementData() { > Comparable[] elementData = this.elementData; > if (elementData == null) { > synchronized (this) { > elementData = this.elementData; > if (elementData == null) { > this.elementData = elementData = new Comparable[1]; > } > } > } > return elementData; > } > > private final StampedLock lock = new StampedLock(); > > public > E intern(Class klass, E memberName, int redefined_count) { > int size, index = 0; > Comparable[] elementData; > // try to take an optimistic-read stamp > long rstamp = lock.tryOptimisticRead(); > long wstamp = 0L; > > if (rstamp != 0L) { // successfull > // 1st read size so that it doesn't overshoot the actual elementData.length > size = this.size; > // 2nd read elementData > elementData = elementData(); > > index = Arrays.binarySearch(elementData, 0, size, memberName); > if (index >= 0) { > E element = (E) elementData[index]; > // validate that our reads were not disturbed by any writes > if (lock.validate(rstamp)) { > return element; > } > } > > // try to convert to write lock > wstamp = lock.tryConvertToWriteLock(rstamp); > } > > if (wstamp == 0L) { > // either tryOptimisticRead or tryConvertToWriteLock failed - > // must acquire write lock and re-read/re-try search > wstamp = lock.writeLock(); > size = this.size; > elementData = elementData(); > index = Arrays.binarySearch(elementData, 0, size, memberName); > if (index >= 0) { > E element = (E) elementData[index]; > lock.unlockWrite(wstamp); > return element; > } > } > > // we have a write lock and are sure there was no element found > E element = add(klass, ~index, memberName, redefined_count); > > lock.unlockWrite(wstamp); > return element; > } > > > > The only thing that will have to be done to add() method is to publish new elements safely. Code doing binary-search under optimistic read could observe an unsafely published MemberName and comparing with such instance could lead to a NPE for example. To remedy this, the newly inserted MemberName would have to be published using a volatile write to the array slot (using Unsafe) - moving existing elements up and down the array does not have to be performed with volatile writes, since they have already been published. > > Do you think this would be worth the effort? > > Regards, Peter > > >> On 2014-10-31, at 6:01 PM, Peter Levart >> wrote: >> >> >>> On 10/31/2014 07:11 PM, David Chase wrote: >>> >>>> I found a lurking bug and updated the webrevs ? I was mistaken >>>> about this version having passed the ute tests (but now, for real, it does). >>>> >>>> I also added converted Christian?s test code into a jtreg test (which passes): >>>> >>>> >>>> >>>> http://cr.openjdk.java.net/~drchase/8013267/hotspot.05/ >>>> http://cr.openjdk.java.net/~drchase/8013267/jdk.05/ >>> Hi David, >>> >>> I'll just comment on the JDK side of things. >>> >>> In Class.ClassData.intern(), ther is a part that synchronizes on the elementData (volatile field holding array of Comparable(s)): >>> >>> 2500 synchronized (elementData) { >>> 2501 final int index = Arrays.binarySearch(elementData, 0, size, memberName); >>> 2502 if (index >= 0) { >>> 2503 return (E) elementData[index]; >>> 2504 } >>> 2505 // Not found, add carefully. >>> 2506 return add(klass, ~index, memberName, redefined_count); >>> 2507 } >>> >>> Inside this synchronized block, add() method is called, which can call grow() method: >>> >>> 2522 if (oldCapacity + 1 > element_data.length ) { >>> 2523 // Replacing array with a copy is safe; elements are identical. >>> 2524 grow(oldCapacity + 1); >>> 2525 element_data = elementData; >>> 2526 } >>> >>> grow() method creates a copy of elementData array and replaces it on this volatile field (line 2584): >>> >>> 2577 private void grow(int minCapacity) { >>> 2578 // overflow-conscious code >>> 2579 int oldCapacity = elementData.length; >>> 2580 int newCapacity = oldCapacity + (oldCapacity >> 1); >>> 2581 if (newCapacity - minCapacity < 0) >>> 2582 newCapacity = minCapacity; >>> 2583 // minCapacity is usually close to size, so this is a win: >>> 2584 elementData = Arrays.copyOf(elementData, newCapacity); >>> 2585 } >>> >>> A concurrent call to intern() can therefore synchronize on a different monitor, so two threads will be inserting the element into the same array at the same time, Auch! >>> >>> >>> >>> Also, lazy construction of ClassData instance: >>> >>> 2593 private ClassData classData() { >>> 2594 if (this.classData != null) { >>> 2595 return this.classData; >>> 2596 } >>> 2597 synchronized (this) { >>> 2598 if (this.classData == null) { >>> 2599 this.classData = new ClassData<>(); >>> 2600 } >>> 2601 } >>> 2602 return this.classData; >>> 2603 } >>> >>> Synchronizes on the j.l.Class instance, which can interfere with user synchronization (think synchronized static methods). This dangerous. >>> >>> Theres an inner class Class.Atomic which is a home for Unsafe machinery in j.l.Class. You can add a casClassData method to it and use it to atomically install the ClassData instance without synchronized blocks. >>> >>> >>> >>> Regards, Peter >>> >>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From peter.levart at gmail.com Sat Nov 1 16:40:02 2014 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 01 Nov 2014 17:40:02 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <545406BC.20005@gmail.com> Message-ID: <54550CE2.9090605@gmail.com> On 10/31/2014 11:59 PM, David Chase wrote: > Thanks very much, I shall attend to these irregularities. > > David Hi David, Just a nit (in Class.ClassData): 2537 if (oldCapacity > 0) { 2538 element_data[oldCapacity] = element_data[oldCapacity - 1]; 2539 // all array elements are non-null and sorted, increase size. 2540 // if store to element_data above floats below 2541 // store to size on the next line, that will be 2542 // inconsistent to the VM if a safepoint occurs here. 2543 size += 1; 2544 for (int i = oldCapacity; i > index; i--) { 2545 // pre: element_data[i] is duplicated at [i+1] 2546 element_data[i] = element_data[i - 1]; 2547 // post: element_data[i-1] is duplicated at [i] 2548 } 2549 // element_data[index] is duplicated at [index+1] 2550 element_data[index] = (Comparable) e; 2551 } else { In line 2544, you could start the for loop with (int i = oldCapacity - 1; ...), since you have already moved the last element before incrementing the size. Also, I would more quickly grasp the code if "oldCapacity" was called "oldSize". Now just a though... What is the expected ratio of intern() calls that insert new element to those that just return existing interned element? If those that return existing element are frequent and since you already carefully arrange insertion so that VM can at any safepoint see the "consistent" state without null elements, I wonder if intern() could itself perform an optimistic search without holding an exclusive lock. This is just a speculation, but would the following code work? private Comparable[] elementData() { Comparable[] elementData = this.elementData; if (elementData == null) { synchronized (this) { elementData = this.elementData; if (elementData == null) { this.elementData = elementData = new Comparable[1]; } } } return elementData; } private final StampedLock lock = new StampedLock(); public > E intern(Class klass, E memberName, int redefined_count) { int size, index = 0; Comparable[] elementData; // try to take an optimistic-read stamp long rstamp = lock.tryOptimisticRead(); long wstamp = 0L; if (rstamp != 0L) { // successfull // 1st read size so that it doesn't overshoot the actual elementData.length size = this.size; // 2nd read elementData elementData = elementData(); index = Arrays.binarySearch(elementData, 0, size, memberName); if (index >= 0) { E element = (E) elementData[index]; // validate that our reads were not disturbed by any writes if (lock.validate(rstamp)) { return element; } } // try to convert to write lock wstamp = lock.tryConvertToWriteLock(rstamp); } if (wstamp == 0L) { // either tryOptimisticRead or tryConvertToWriteLock failed - // must acquire write lock and re-read/re-try search wstamp = lock.writeLock(); size = this.size; elementData = elementData(); index = Arrays.binarySearch(elementData, 0, size, memberName); if (index >= 0) { E element = (E) elementData[index]; lock.unlockWrite(wstamp); return element; } } // we have a write lock and are sure there was no element found E element = add(klass, ~index, memberName, redefined_count); lock.unlockWrite(wstamp); return element; } The only thing that will have to be done to add() method is to publish new elements safely. Code doing binary-search under optimistic read could observe an unsafely published MemberName and comparing with such instance could lead to a NPE for example. To remedy this, the newly inserted MemberName would have to be published using a volatile write to the array slot (using Unsafe) - moving existing elements up and down the array does not have to be performed with volatile writes, since they have already been published. Do you think this would be worth the effort? Regards, Peter > On 2014-10-31, at 6:01 PM, Peter Levart wrote: > >> On 10/31/2014 07:11 PM, David Chase wrote: >>> I found a lurking bug and updated the webrevs ? I was mistaken >>> about this version having passed the ute tests (but now, for real, it does). >>> >>> I also added converted Christian?s test code into a jtreg test (which passes): >>> >>> >>> http://cr.openjdk.java.net/~drchase/8013267/hotspot.05/ >>> http://cr.openjdk.java.net/~drchase/8013267/jdk.05/ >> Hi David, >> >> I'll just comment on the JDK side of things. >> >> In Class.ClassData.intern(), ther is a part that synchronizes on the elementData (volatile field holding array of Comparable(s)): >> >> 2500 synchronized (elementData) { >> 2501 final int index = Arrays.binarySearch(elementData, 0, size, memberName); >> 2502 if (index >= 0) { >> 2503 return (E) elementData[index]; >> 2504 } >> 2505 // Not found, add carefully. >> 2506 return add(klass, ~index, memberName, redefined_count); >> 2507 } >> >> Inside this synchronized block, add() method is called, which can call grow() method: >> >> 2522 if (oldCapacity + 1 > element_data.length ) { >> 2523 // Replacing array with a copy is safe; elements are identical. >> 2524 grow(oldCapacity + 1); >> 2525 element_data = elementData; >> 2526 } >> >> grow() method creates a copy of elementData array and replaces it on this volatile field (line 2584): >> >> 2577 private void grow(int minCapacity) { >> 2578 // overflow-conscious code >> 2579 int oldCapacity = elementData.length; >> 2580 int newCapacity = oldCapacity + (oldCapacity >> 1); >> 2581 if (newCapacity - minCapacity < 0) >> 2582 newCapacity = minCapacity; >> 2583 // minCapacity is usually close to size, so this is a win: >> 2584 elementData = Arrays.copyOf(elementData, newCapacity); >> 2585 } >> >> A concurrent call to intern() can therefore synchronize on a different monitor, so two threads will be inserting the element into the same array at the same time, Auch! >> >> >> >> Also, lazy construction of ClassData instance: >> >> 2593 private ClassData classData() { >> 2594 if (this.classData != null) { >> 2595 return this.classData; >> 2596 } >> 2597 synchronized (this) { >> 2598 if (this.classData == null) { >> 2599 this.classData = new ClassData<>(); >> 2600 } >> 2601 } >> 2602 return this.classData; >> 2603 } >> >> Synchronizes on the j.l.Class instance, which can interfere with user synchronization (think synchronized static methods). This dangerous. >> >> Theres an inner class Class.Atomic which is a home for Unsafe machinery in j.l.Class. You can add a casClassData method to it and use it to atomically install the ClassData instance without synchronized blocks. >> >> >> >> Regards, Peter >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.r.chase at oracle.com Mon Nov 3 00:05:16 2014 From: david.r.chase at oracle.com (David Chase) Date: Sun, 2 Nov 2014 19:05:16 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> Message-ID: <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> On 2014-10-31, at 5:45 PM, Vitaly Davidovich wrote: > The volatile load prevents subsequent loads and stores from reordering with it, but that doesn't stop C from moving before the B store. So breaking B into the load (call it BL) and store (BS) you can still get this ordering: A, BL, C, BS I think this should do the trick. element_data[oldCapacity] = element_data[oldCapacity - 1]; // all array elements are non-null and sorted, increase size. // if store to element_data above floats below // store to size on the next line, that will be // inconsistent to the VM if a safepoint occurs here. size += 1; // Load of volatile size prevents movement of element_data store for (int i = size - 1; i > index; i--) { The change is to load the volatile size for the loop bound; this stops the stores in the loop from moving earlier, right? David -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From david.holmes at oracle.com Mon Nov 3 03:49:45 2014 From: david.holmes at oracle.com (David Holmes) Date: Mon, 03 Nov 2014 13:49:45 +1000 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> Message-ID: <5456FB59.60905@oracle.com> On 3/11/2014 10:05 AM, David Chase wrote: > > On 2014-10-31, at 5:45 PM, Vitaly Davidovich wrote: > >> The volatile load prevents subsequent loads and stores from reordering with it, but that doesn't stop C from moving before the B store. So breaking B into the load (call it BL) and store (BS) you can still get this ordering: A, BL, C, BS > > I think this should do the trick. > > element_data[oldCapacity] = element_data[oldCapacity - 1]; > // all array elements are non-null and sorted, increase size. > // if store to element_data above floats below > // store to size on the next line, that will be > // inconsistent to the VM if a safepoint occurs here. > size += 1; > // Load of volatile size prevents movement of element_data store > for (int i = size - 1; i > index; i--) { > > The change is to load the volatile size for the loop bound; this stops the stores > in the loop from moving earlier, right? Treating volatile accesses like memory barriers is playing a bit fast-and-loose with the spec+implementation. The basic happens-before relationship for volatiles states that if a volatile read sees a value X, then the volatile write that wrote X happened-before the read [1]. But in this code there are no checks of the values of the volatile fields. Instead you are relying on a volatile read "acting like acquire()" and a volatile write "acting like release()". That said you are trying to "synchronize" the hotspot code with the JDK code so you have stepped outside the JMM in any case and reasoning about what is and is not allowed is somewhat moot - unless the hotspot code always uses Java-style accesses to the Java-level variables. BTW the Java side of this needs to be reviewed on core-libs-dev at openjdk.java.net David H. [1] http://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.4 > David > From david.r.chase at oracle.com Mon Nov 3 12:49:55 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 3 Nov 2014 07:49:55 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5456FB59.60905@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> Message-ID: <632A5C98-B386-4625-BE12-355241581955@oracle.com> On 2014-11-02, at 10:49 PM, David Holmes wrote: >> The change is to load the volatile size for the loop bound; this stops the stores >> in the loop from moving earlier, right? > > Treating volatile accesses like memory barriers is playing a bit fast-and-loose with the spec+implementation. The basic happens-before relationship for volatiles states that if a volatile read sees a value X, then the volatile write that wrote X happened-before the read [1]. But in this code there are no checks of the values of the volatile fields. Instead you are relying on a volatile read "acting like acquire()" and a volatile write "acting like release()". > > That said you are trying to "synchronize" the hotspot code with the JDK code so you have stepped outside the JMM in any case and reasoning about what is and is not allowed is somewhat moot - unless the hotspot code always uses Java-style accesses to the Java-level variables. My main concern is that the compiler is inhibited from any peculiar code motion; I assume that taking a safe point has a bit of barrier built into it anyway, especially given that the worry case is safepoint + JVMTI. Given the worry, what?s the best way to spell ?barrier? here? I could synchronize on classData (it would be a recursive lock in the current version of the code) synchronized (this) { size++; } or I could synchronize on elementData (no longer used for a lock elsewhere, so always uncontended) synchronized (elementData) { size++; } or is there some Unsafe thing that would be better? (core-libs-dev ? there will be another webrev coming. This is a runtime+jdk patch.) David > BTW the Java side of this needs to be reviewed on core-libs-dev at openjdk.java.net > > David H. > > [1] http://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.4 > > >> David -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From roland.westrelin at oracle.com Mon Nov 3 12:52:16 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Mon, 3 Nov 2014 13:52:16 +0100 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <54542F72.3000609@oracle.com> References: <54542F72.3000609@oracle.com> Message-ID: <178002AA-8A1E-4B7D-A0E5-3B6CCDE38DE7@oracle.com> > http://cr.openjdk.java.net/~kvn/8059780/webrev/ Looks good to me. Roland. From roland.westrelin at oracle.com Mon Nov 3 13:08:12 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Mon, 3 Nov 2014 14:08:12 +0100 Subject: RFR(S) : 8043125 : compiler/types/correctness/CorrectnessTest.java: assert(layout->tag() == DataLayout::speculative_trap_data_tag) failed: wrong type In-Reply-To: <5451A9F5.2060207@oracle.com> References: <5451A9F5.2060207@oracle.com> Message-ID: <0FBF83F9-4D3C-4D1A-B8D0-3A81F9B9600C@oracle.com> > http://cr.openjdk.java.net/~iignatyev/8043125/webrev.00/ Wouldn?t it be better to keep the VM operation? If there?s java code running concurrently with the ClearMethodState call, profile will be updated and cleared concurrently. I?m not sure that would be safe. Or are we sure that ClearMethodState will never be run concurrently with other code? Roland. From roland.westrelin at oracle.com Mon Nov 3 13:19:41 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Mon, 3 Nov 2014 14:19:41 +0100 Subject: RFR(M): 6700100: optimize inline_native_clone() for small objects with exact klass In-Reply-To: <544F67A4.3060801@oracle.com> References: <5436E2C3.7060008@oracle.com> <5436FCF6.7080309@oracle.com> <543814CC.2080905@oracle.com> <5C32D21B-9A72-4367-A5F7-3C4FB2326FE0@oracle.com> <543833F6.5050002@oracle.com> <71726E40-F4AF-43A4-A213-1A60F5687AB0@oracle.com> <025487DA-04CA-49BB-885C-39446D0C9BF7@oracle.com> <544F67A4.3060801@oracle.com> Message-ID: <02C11976-5D45-40F7-A852-5BBA274CA963@oracle.com> Hi Aleksey, Thanks for the feedback. > A minor pet peeve about the code style in Hotspot, which adds up to > accidental code complexity: pro-actively blowing up more methods than > required to solve the task at hand. > > This change provides the example: both compute_injected_fields() and > has_injected_fields() do dances around _has_injected_fields value. It > would seem more readable to just have has_injected_fields() to lazily > compute the value on demand, inlining both compute_injected_fields() and > compute_injected_fields_helper(). That would spare two additional > methods, and make the code generally more readable, and maintainable. has_injected_fields() is small so it can fit in the .hpp and be inlined which pays off if it?s called repeatedly once _has_injected_fields is computed. The compute_injected_fields() stuff is in its own method to keep has_injected_fields() small. compute_injected_fields_helper() is its own method because we have to enter the VM to do that computation and it?s clearer (to me at least) that whatever we do once we have entered the VM is together in a separate method. So it doesn?t look that bad to me. Roland. From bengt.rutisson at oracle.com Mon Nov 3 13:31:13 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 03 Nov 2014 14:31:13 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <545386E1.2050402@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> Message-ID: <545783A1.3050300@oracle.com> Hi Dima, Answers inline. On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: > Hi Bengt, > > Thanks a lot for your detailed feedback, we appreciate it very much! > > See comments inline. > > On 31.10.2014 1:09, Bengt Rutisson wrote: >> >> Hi Evgeniya, >> >> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>> Hi, >>> >>> Please review changes for 8062537, the OpenJDK/hotspot part of the >>> JDK-8019361 >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>> >>> Problem: Some tests explicitly set GC and fail when jtreg set >>> another GC. >>> Solution: Such tests marked with the jtreg tag "requires" to skip >>> test if there is a conflict >> >> Thanks for fixing this! It is really great that we finally start >> sorting this out. >> >> First a general comment. The @requires tag has been developed without >> much cooperation with the GC team. We did have a lot of feedback when >> it was first presented a year ago, but it does not seem like this >> feedback was incorporated into the @requires that was eventually built. > > We tried to implement as much developer's wishes as possible. But not > everything is possible, sorry for that. Yes, I'm sure you have done your best. It's just that we have been requesting this feature for 3 years and I was expecting us to be able to influence the feature much more than was the case now. > >> >> I think this change that gets proposed now is a big step forward and >> I won't object to it. But I am pretty convinced that we will soon run >> in to the limitations of the current @requires implementation and we >> will have to redo this work. >> >> Some of the points I don't really like about the @requires tag are: >> >> - the "vm.gc" abstraction is more limiting than helping. It would >> have been better to just "require" any command line flag. > "vm.gc" is an alias to a very popular flag. It's also possible to use: > vm.opt.UseG1GC == true instead. > > The table with all vars available in jtreg: > http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names The problem with having this matching built in to JTreg is that it makes it very hard to change. When we discussed this a year ago I think we said that JTreg should only provide a means to test against the command line and a hook for running some java code in the @requires tag. That way we could put logic like this in a test library that is under our control. This would make it easy for us to change and also enables us to use different logic for different versions. > >> - the requirement should be per @run tag. Right now we have to do >> what you did in this change and use vm.gc=null even when some tests >> could actually have been run when a GC was specified. > it would be great, but it will unlikely happen in jtreg, as well as > test case support. what do you mean with test case support? > >> - there are many tests that require more than just a specific GC. >> Often there are other flags that can't be changed either for the test >> to work properly. > > yes. conflicting GC is just the most popular problem caused by > conflicting options. > If we address this issue and we are satisfied with solution, we could > move further. Yes, I agree that taking one step at the time is good. Personally I would have preferred that the first step was a "just run the command line as specified in the @run tag" step. > >> >> Maybe this is not the right place to discuss the current >> implementation of the @requires tag. I just want to say that I'm not >> too happy about how the @requires tag turned out. But assuming we >> have to use it the way it is now I guess the proposed changeset looks >> good. > > yes, this thread is about change made by Evgeniya, not about jtreg :) > And thanks for reviewing it! Agreed. And as I said, I think the patch looks ok. I have not looked at all tests. But if they now pass with the combinations that we test with I guess they should be ok. > >> >>> Tested locally with different GC flags (-XX:+UseG1GC, >>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>> without any GC flag). Tests are being excluded as expected. No tests >>> failed because of the conflict. >> Have you tested with -Xconcgc too? It's an alias for >> -XX:+UseConcMarkSweepGC. > > '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) Ok. Thanks. > >> >> I think some of the test, like >> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >> with -XX:+UseParNewGC. Others, like >> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >> with -XX:-UseParNewGC. Could you test these two cases too? > > These two tests ignore vm flags. > Add @requires here is not necessary, but it will allow not execute the > tests when not needed. > So, if we run HS tests with 4 GC, we don't need to run these tests 4 > times, 1 should be enough. Do we really want to use the @requires functionality for this purpose? It seems like a way of misusing @requires. If we just want the tests to be run once I think Leonid's approach with tests lists seems more suitable. But are you sure that this is the reason for the @requires in this case? TestDefNewCMS does sound like a test that is DefNew specific. I don't see a reason to run it with ParNew. If it doesn't fail today it should probably be changed so that it does fail if it is run with the wrong GC. > >> Similarly it looks to me like there are tests that will fail if you >> run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >> >> Just a heads up. These two tests will soon be removed. I'm about to >> push a changeset that removes them: >> >> test/gc/startup_warnings/TestCMSIncrementalMode.java >> test/gc/startup_warnings/TestCMSNoIncrementalMode.java > okay, thank for letting us know. > >> >> Is there some way of making sure that all tests are run at one time >> or another. With this change there is a risk that some tests are >> never run and always skipped. Will we somehow be tracking what gets >> skipped and make sure that all tests are at least run once with the >> correct GC so that it is not skipped all the time? > > This is a very good question! > jtreg now doesn't report skipped tests, hopefully it will do soon, > after getting fix of: > https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 > > And yes, tracking tests which are not run is important thing. > @requires - is not the only to exclude test from execution. > > Other examples: > > /* > *@ignore > *@test > */ > ... > > /*@bug 4445555 > *@test > */ > ... > Such tests will never be run, because jtreg treats as test only files > with @test on the first place... > > So, making sure that tests do not disappear is important SQE task, we > know about that, we're thinking on solution (may be very actively). > But this subject for another discussion, not within RFR :) Right. Glad to hear that you are actively working on this! Bengt > > Thanks, > Dima > > > >> >> Thanks, >> Bengt >> >>> >>> Thanks, >>> Evgeniya Stepanova >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zoltan.majo at oracle.com Mon Nov 3 13:58:45 2014 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Mon, 03 Nov 2014 14:58:45 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) Message-ID: <54578A15.7060605@oracle.com> Hi, please review the following patch. Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 Problem: We have five tests that fail with SIGSEGV in our nightlies: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java All tests fail when executing the C2-compiled version of java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): 489: private void inflateSpine() { 490: if (spine == null) { 491: spine = newArrayArray(MIN_SPINE_SIZE); 492: priorElementCount = new long[MIN_SPINE_SIZE]; 493: spine[0] = curChunk; 494: } 495: } The failure is due to the 'aastore' at line 493; the failure is caused by the monomorphic array check optimization (-XX:+MonomorphicArrayCheck, enabled by default). In InfiniteStreamWithLimitOpTest.java, C2 determines (based on profiling information) that inflateSpine() has two hot receiver types, SpinedBuffer.OfInt and SpinedBuffer.OfDouble: - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] - in SpinedBuffer.ofDouble, the variable 'spine' is of type double[][]. Please consider the following pseudo Java code that illustrates how C2 sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on the two hot receiver types SpinedBuffer.OfInt and SpinedBuffer.OfDouble: static void inflateSpine(boolean isOfDouble, Object curChunk) { Object[] spine = isOfDouble ? new double[8][] : new int[8][]; spine[0] = curChunk; } If MonomorphicArrayCheck is disabled, C2 checks the element type of 'spine' *at runtime*. The check looks something like: if (!spine.getClass().getComponentType().isInstance(curChunk)) { throw new ArrayStoreException(); } If MonomorphicArrayCheck is enabled (our case), C2 creates an array check like this: if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { throw new ArrayStoreException(); } where TYPE is the type of the 'spine' variable, as it is determined by C2 *at compile time*. The optimization treats TYPE as a constant and thus saves a load from memory + an offset calculation (I think). The problem is that due to 'spine' being either of type int[][] or of type double[][], C2 determines that TYPE==java/lang/Object. As a result, when the inflateSpine() method is executed, it first loads the Klass* corresponding to java/lang/Object (TYPE). Let us call this Klass* array_klass. Then, the method obtains a Klass* from array_klass->_element_klass. Then, the method reads from offset 20 from array_klass->_element_klass, which results in a SIGSEGV. The reason for the SIGSEGV is that the Klass* array_klass is of type java/lang/Object and is therefore represented as an 'InstanceKlass'. But _element_klass is defined only in objects of type 'ObjArrayKlass'. The compiler reads array_klass->_element_klass because it expects the destination of an 'aastore' to be an array, which is not true if TYPE==java/lang/Object. Solution: The compiler already checks if the compile-time type of the array (TYPE) is the same as the array's type at runtime. If that check fails, the method branches to an uncommon trap. In our case the SIGSEGV happens because the load of array_klass->_element_klass "floats" above the check. I propose two solutions: Solution 1. Add a control edge between the IfTrue branch of the check and the load of _array_klass->_element_klass. That prohibits the load of _array_klass->element_klass floating above the check. Solution 2. In addition to the changes in Solution 1, the compiler checks if the compile-time type of the array (TYPE) is java/lang/Object. If TYPE==java/lang/Object, the compiler should not assume array_type to be TYPE, but should determine array type at runtime instead. The reason is for Solution 2 is: We can't do an array store into a java/lang/Object. So, if we see TYPE==java/lang/Object at compile time, it is pretty sure that the method will deoptimize at runtime. Instead of assuming TYPE to be java/lang/Object, we read at runtime the type of the array store's destination. That still gives us some chance that the compiled version of the method will successfully execute. The additional check is in parseHelper.cpp in Parse::array_store_check() on line 171. Webrev: Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ Testing (for both solutions): JPRT, manual testing with failing test cases. Please note that both solutions are somewhat "hacky", because that allowed me to produce a solution within reasonable time and with reasonable effort. Any suggestions or feedback is welcome. Thank you and best regards, Zoltan From roland.westrelin at oracle.com Mon Nov 3 14:16:49 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Mon, 3 Nov 2014 15:16:49 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <54578A15.7060605@oracle.com> References: <54578A15.7060605@oracle.com> Message-ID: <73185467-C424-4AED-BA52-5AB406FFC218@oracle.com> Hi Zoltan, > Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ Instead: 1505 && attempt_remove_control_input in LoadNode::Ideal_helper() you should be able to do something like this: Opcode() != Op_LoadKlass And then you don't need most of the changes in memnode.cpp Roland. From david.r.chase at oracle.com Mon Nov 3 16:36:59 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 3 Nov 2014 11:36:59 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5457AA75.8090103@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> Message-ID: >> My main concern is that the compiler is inhibited from any peculiar code motion; I assume that taking a safe point has a bit of barrier built into it anyway, especially given that the worry case is safepoint + JVMTI. >> >> Given the worry, what?s the best way to spell ?barrier? here? >> I could synchronize on classData (it would be a recursive lock in the current version of the code) >> synchronized (this) { size++; } >> or I could synchronize on elementData (no longer used for a lock elsewhere, so always uncontended) >> synchronized (elementData) { size++; } >> or is there some Unsafe thing that would be better? > > You're worried that writes moving array elements up for one slot would bubble up before write of size = size+1, right? If that happens, VM could skip an existing (last) element and not update it. exactly, with the restriction that it would be compiler-induced bubbling, not architectural. Which is both better, and worse ? I don?t have to worry about crazy hardware, but the rules of java/jvm "memory model" are not as thoroughly defined as those for java itself. I added a method to Atomic (.storeFence() ). New webrev to come after I rebuild and retest. Thanks much, David > It seems that Unsafe.storeFence() between size++ and moving of elements could do, as the javadoc for it says: > > /** > * Ensures lack of reordering of stores before the fence > * with loads or stores after the fence. > * @since 1.8 > */ > public native void storeFence(); -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From peter.levart at gmail.com Mon Nov 3 16:16:53 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 03 Nov 2014 17:16:53 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <632A5C98-B386-4625-BE12-355241581955@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> Message-ID: <5457AA75.8090103@gmail.com> On 11/03/2014 01:49 PM, David Chase wrote: > On 2014-11-02, at 10:49 PM, David Holmes wrote: >>> The change is to load the volatile size for the loop bound; this stops the stores >>> in the loop from moving earlier, right? >> Treating volatile accesses like memory barriers is playing a bit fast-and-loose with the spec+implementation. The basic happens-before relationship for volatiles states that if a volatile read sees a value X, then the volatile write that wrote X happened-before the read [1]. But in this code there are no checks of the values of the volatile fields. Instead you are relying on a volatile read "acting like acquire()" and a volatile write "acting like release()". >> >> That said you are trying to "synchronize" the hotspot code with the JDK code so you have stepped outside the JMM in any case and reasoning about what is and is not allowed is somewhat moot - unless the hotspot code always uses Java-style accesses to the Java-level variables. > My main concern is that the compiler is inhibited from any peculiar code motion; I assume that taking a safe point has a bit of barrier built into it anyway, especially given that the worry case is safepoint + JVMTI. > > Given the worry, what?s the best way to spell ?barrier? here? > I could synchronize on classData (it would be a recursive lock in the current version of the code) > synchronized (this) { size++; } > or I could synchronize on elementData (no longer used for a lock elsewhere, so always uncontended) > synchronized (elementData) { size++; } > or is there some Unsafe thing that would be better? > > (core-libs-dev ? there will be another webrev coming. This is a runtime+jdk patch.) > > David Hi David, You're worried that writes moving array elements up for one slot would bubble up before write of size = size+1, right? If that happens, VM could skip an existing (last) element and not update it. It seems that Unsafe.storeFence() between size++ and moving of elements could do, as the javadoc for it says: /** * Ensures lack of reordering of stores before the fence * with loads or stores after the fence. * @since 1.8 */ public native void storeFence(); Regards, Peter > >> BTW the Java side of this needs to be reviewed on core-libs-dev at openjdk.java.net >> >> David H. >> >> [1] http://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.4 >> >> >>> David From peter.levart at gmail.com Mon Nov 3 16:42:30 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 03 Nov 2014 17:42:30 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5457AA75.8090103@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> Message-ID: <5457B076.10205@gmail.com> On 11/03/2014 05:16 PM, Peter Levart wrote: > On 11/03/2014 01:49 PM, David Chase wrote: >> On 2014-11-02, at 10:49 PM, David Holmes >> wrote: >>>> The change is to load the volatile size for the loop bound; this >>>> stops the stores >>>> in the loop from moving earlier, right? >>> Treating volatile accesses like memory barriers is playing a bit >>> fast-and-loose with the spec+implementation. The basic >>> happens-before relationship for volatiles states that if a volatile >>> read sees a value X, then the volatile write that wrote X >>> happened-before the read [1]. But in this code there are no checks >>> of the values of the volatile fields. Instead you are relying on a >>> volatile read "acting like acquire()" and a volatile write "acting >>> like release()". >>> >>> That said you are trying to "synchronize" the hotspot code with the >>> JDK code so you have stepped outside the JMM in any case and >>> reasoning about what is and is not allowed is somewhat moot - unless >>> the hotspot code always uses Java-style accesses to the Java-level >>> variables. >> My main concern is that the compiler is inhibited from any peculiar >> code motion; I assume that taking a safe point has a bit of barrier >> built into it anyway, especially given that the worry case is >> safepoint + JVMTI. >> >> Given the worry, what?s the best way to spell ?barrier? here? >> I could synchronize on classData (it would be a recursive lock in the >> current version of the code) >> synchronized (this) { size++; } >> or I could synchronize on elementData (no longer used for a lock >> elsewhere, so always uncontended) >> synchronized (elementData) { size++; } >> or is there some Unsafe thing that would be better? >> >> (core-libs-dev ? there will be another webrev coming. This is a >> runtime+jdk patch.) >> >> David > > Hi David, > > You're worried that writes moving array elements up for one slot would > bubble up before write of size = size+1, right? If that happens, VM > could skip an existing (last) element and not update it. > > It seems that Unsafe.storeFence() between size++ and moving of > elements could do, as the javadoc for it says: > > /** > * Ensures lack of reordering of stores before the fence > * with loads or stores after the fence. > * @since 1.8 > */ > public native void storeFence(); You might need a storeFence() between each two writes into the array too. Your moving loop is the following: 2544 for (int i = oldCapacity; i > index; i--) { 2545 // pre: element_data[i] is duplicated at [i+1] 2546 element_data[i] = element_data[i - 1]; 2547 // post: element_data[i-1] is duplicated at [i] 2548 } If we start unrolling, it becomes: w1: element_data[old_capacity - 0] = element_data[old_capacity - 1]; w2: element_data[old_capacity - 1] = element_data[old_capacity - 2]; w3: element_data[old_capacity - 2] = element_data[old_capacity - 3]; ... Can compiler reorder w2 and w3 (just writes - not the whole statements)? Say that it reads a chunk of elements into the registers and then writes them out, but in different order, and a check for safepoint comes inside this chunk of writes... This is hypothetical, but it could do it without breaking the local semantics... Peter > > > Regards, Peter > > > >> >>> BTW the Java side of this needs to be reviewed on >>> core-libs-dev at openjdk.java.net >>> >>> David H. >>> >>> [1] >>> http://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.4 >>> >>> >>>> David > From vladimir.kozlov at oracle.com Mon Nov 3 17:18:50 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Nov 2014 09:18:50 -0800 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <32703F35-7EA5-42F3-B99A-DE88677D0C0D@oracle.com> References: <54542F72.3000609@oracle.com> <32703F35-7EA5-42F3-B99A-DE88677D0C0D@oracle.com> Message-ID: <5457B8FA.2020208@oracle.com> Thank you, Igor Vladimir K On 11/1/14 12:29 AM, Igor Veresov wrote: > Looks fine to me. > > igor > > On Oct 31, 2014, at 2:55 PM, Vladimir Kozlov wrote: > >> http://cr.openjdk.java.net/~kvn/8059780/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8059780 >> >> We found performance regression (3-5%) on x64 after changes 8052081 [1] were pushed. It is caused by changes in lcm.cpp. Those changes were done because Atom is in-order cpu and some benchmarks on Atom showed benefit from these changes. >> >> But Atom is not our core platform and the regression is big. >> >> Backout changes made in lcm.cpp in jdk9 and 8u40 (8052081 was backported to 8u40). >> >> We have RFEs to improve block's instructions order which may help both platforms: >> >> https://bugs.openjdk.java.net/browse/JDK-6958833 >> https://bugs.openjdk.java.net/browse/JDK-7101232 >> >> Tested with SPECjvm2008-mpegaudio. >> >> Thanks, >> Vladimir >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8052081 > From vladimir.kozlov at oracle.com Mon Nov 3 17:20:03 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Nov 2014 09:20:03 -0800 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <178002AA-8A1E-4B7D-A0E5-3B6CCDE38DE7@oracle.com> References: <54542F72.3000609@oracle.com> <178002AA-8A1E-4B7D-A0E5-3B6CCDE38DE7@oracle.com> Message-ID: <5457B943.8040101@oracle.com> Thank you, Roland Vladimir On 11/3/14 4:52 AM, Roland Westrelin wrote: >> http://cr.openjdk.java.net/~kvn/8059780/webrev/ > > Looks good to me. > > Roland. > From david.r.chase at oracle.com Mon Nov 3 17:28:00 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 3 Nov 2014 12:28:00 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5457B076.10205@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457B076.10205@gmail.com> Message-ID: <73CF1882-22F1-4D9A-B37A-EAC9BCF675B0@oracle.com> On 2014-11-03, at 11:42 AM, Peter Levart wrote: >> You're worried that writes moving array elements up for one slot would bubble up before write of size = size+1, right? If that happens, VM could skip an existing (last) element and not update it. >> >> It seems that Unsafe.storeFence() between size++ and moving of elements could do, as the javadoc for it says: >> >> /** >> * Ensures lack of reordering of stores before the fence >> * with loads or stores after the fence. >> * @since 1.8 >> */ >> public native void storeFence(); > You might need a storeFence() between each two writes into the array too. Your moving loop is the following: > > 2544 for (int i = oldCapacity; i > index; i--) { > 2545 // pre: element_data[i] is duplicated at [i+1] > 2546 element_data[i] = element_data[i - 1]; > 2547 // post: element_data[i-1] is duplicated at [i] > 2548 } > > > If we start unrolling, it becomes: > > w1: element_data[old_capacity - 0] = element_data[old_capacity - 1]; > w2: element_data[old_capacity - 1] = element_data[old_capacity - 2]; > w3: element_data[old_capacity - 2] = element_data[old_capacity - 3]; > ... > > Can compiler reorder w2 and w3 (just writes - not the whole statements)? Say that it reads a chunk of elements into the registers and then writes them out, but in different order, and a check for safepoint comes inside this chunk of writes... This is hypothetical, but it could do it without breaking the local semantics? I think you are right, certainly in theory, and if I don?t hear someone else declaring that in practice we?re both just being paranoid, I?ll do that too. Seems like it might eventually slow things down to do all those fences. David -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dean.long at oracle.com Mon Nov 3 17:40:49 2014 From: dean.long at oracle.com (Dean Long) Date: Mon, 03 Nov 2014 09:40:49 -0800 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <5454303F.8060805@oracle.com> References: <54542F72.3000609@oracle.com> <5454303F.8060805@oracle.com> Message-ID: <5457BE21.9010402@oracle.com> By the way, do we know why the lcm change caused a regression? Is reordering the worklist using pop() a design feature? dl On 10/31/2014 5:58 PM, Vladimir Kozlov wrote: > I forgot to say that lcm.cpp changes were small part of 8052081 > changes. The rest of changes will stay. > > Thanks, > Vladimir > > On 10/31/14 5:55 PM, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8059780/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8059780 >> >> We found performance regression (3-5%) on x64 after changes 8052081 [1] >> were pushed. It is caused by changes in lcm.cpp. Those changes were done >> because Atom is in-order cpu and some benchmarks on Atom showed benefit >> from these changes. >> >> But Atom is not our core platform and the regression is big. >> >> Backout changes made in lcm.cpp in jdk9 and 8u40 (8052081 was backported >> to 8u40). >> >> We have RFEs to improve block's instructions order which may help both >> platforms: >> >> https://bugs.openjdk.java.net/browse/JDK-6958833 >> https://bugs.openjdk.java.net/browse/JDK-7101232 >> >> Tested with SPECjvm2008-mpegaudio. >> >> Thanks, >> Vladimir >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8052081 From vladimir.kozlov at oracle.com Mon Nov 3 18:00:53 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Nov 2014 10:00:53 -0800 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <54578A15.7060605@oracle.com> References: <54578A15.7060605@oracle.com> Message-ID: <5457C2D5.9080401@oracle.com> I agree with additional check tak != TypeKlassPtr::OBJECT (solution 2). Instead of creating new LoadKlassNode::make() method you can set control in parseHelper.cpp since it is the only place which set it: Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), p2, tak); if (always_see_exact_class) { a_e_klass->init_req(MemNode::Control, control()); } a_e_klass = _gvn.transform(a_e_klass); Also an other reason to have control edge in this place is the address of this klass load is based on ConP constant. In a normal case after class check you should have CheckCastPP attached to control and used as address's base. And agree with Roland's suggestion about Op_LoadKlass check. Thanks, Vladimir On 11/3/14 5:58 AM, Zolt?n Maj? wrote: > Hi, > > > please review the following patch. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 > > > Problem: > > > We have five tests that fail with SIGSEGV in our nightlies: > > java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java > java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java > java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java > java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java > java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java > > All tests fail when executing the C2-compiled version of > java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): > > > 489: private void inflateSpine() { > 490: if (spine == null) { > 491: spine = newArrayArray(MIN_SPINE_SIZE); > 492: priorElementCount = new long[MIN_SPINE_SIZE]; > 493: spine[0] = curChunk; > 494: } > 495: } > > > The failure is due to the 'aastore' at line 493; the failure is caused > by the monomorphic array check optimization > (-XX:+MonomorphicArrayCheck, enabled by default). > > In InfiniteStreamWithLimitOpTest.java, C2 determines (based on > profiling information) that inflateSpine() has two hot receiver types, > SpinedBuffer.OfInt and SpinedBuffer.OfDouble: > > - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] > > - in SpinedBuffer.ofDouble, the variable 'spine' is of type > double[][]. > > Please consider the following pseudo Java code that illustrates how C2 > sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on > the two hot receiver types SpinedBuffer.OfInt and > SpinedBuffer.OfDouble: > > > static void inflateSpine(boolean isOfDouble, Object curChunk) { > Object[] spine = isOfDouble ? new double[8][] : new int[8][]; > spine[0] = curChunk; > } > > > If MonomorphicArrayCheck is disabled, C2 checks the element type > of 'spine' *at runtime*. The check looks something like: > > > if (!spine.getClass().getComponentType().isInstance(curChunk)) { > throw new ArrayStoreException(); > } > > > If MonomorphicArrayCheck is enabled (our case), C2 creates an array > check like this: > > > if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { > throw new ArrayStoreException(); > } > > > where TYPE is the type of the 'spine' variable, as it is determined by > C2 *at compile time*. The optimization treats TYPE as a constant and > thus saves a load from memory + an offset calculation (I think). > > The problem is that due to 'spine' being either of type int[][] or of > type double[][], C2 determines that TYPE==java/lang/Object. > > As a result, when the inflateSpine() method is executed, it first loads the > Klass* corresponding to java/lang/Object (TYPE). Let us call this Klass* > array_klass. Then, the method obtains a Klass* from array_klass->_element_klass. > Then, the method reads from offset 20 from array_klass->_element_klass, which > results in a SIGSEGV. > > The reason for the SIGSEGV is that the Klass* array_klass is of type > java/lang/Object and is therefore represented as an 'InstanceKlass'. > But _element_klass is defined only in objects of type 'ObjArrayKlass'. > The compiler reads array_klass->_element_klass because it expects the > destination of an 'aastore' to be an array, which is not true if > TYPE==java/lang/Object. > > > Solution: > > > The compiler already checks if the compile-time type of the array (TYPE) is > the same as the array's type at runtime. If that check fails, the method > branches to an uncommon trap. In our case the SIGSEGV happens because > the load of array_klass->_element_klass "floats" above the check. > > I propose two solutions: > > Solution 1. Add a control edge between the IfTrue branch of the check and > the load of _array_klass->_element_klass. That prohibits the load of > _array_klass->element_klass floating above the check. > > Solution 2. In addition to the changes in Solution 1, the compiler checks if > the compile-time type of the array (TYPE) is java/lang/Object. If > TYPE==java/lang/Object, the compiler should not assume array_type to be > TYPE, but should determine array type at runtime instead. > > The reason is for Solution 2 is: We can't do an array store into a java/lang/Object. > So, if we see TYPE==java/lang/Object at compile time, it is pretty sure that the > method will deoptimize at runtime. Instead of assuming TYPE to be java/lang/Object, > we read at runtime the type of the array store's destination. That still gives us some > chance that the compiled version of the method will successfully execute. The > additional check is in parseHelper.cpp in Parse::array_store_check() on line 171. > > > Webrev: > > Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ > Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ > > > Testing (for both solutions): > > JPRT, manual testing with failing test cases. > > > Please note that both solutions are somewhat "hacky", because that allowed me to > produce a solution within reasonable time and with reasonable effort. Any suggestions > or feedback is welcome. > > > Thank you and best regards, > > > Zoltan > From john.r.rose at oracle.com Mon Nov 3 18:07:13 2014 From: john.r.rose at oracle.com (John Rose) Date: Mon, 3 Nov 2014 10:07:13 -0800 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <5457BE21.9010402@oracle.com> References: <54542F72.3000609@oracle.com> <5454303F.8060805@oracle.com> <5457BE21.9010402@oracle.com> Message-ID: <2740E980-A265-4D7A-A872-24635C72D982@oracle.com> On Nov 3, 2014, at 9:40 AM, Dean Long wrote: > By the way, do we know why the lcm change caused a regression? Is reordering the worklist > using pop() a design feature? It's simple a data structure trick. The worklist is logically unordered, but physically packed into an array. If you remove an item from an array, preserving order, you have to move O(N) items to close up the gap. If you don't care about order, you can remove an item with O(1) move, as shown in the restored code. It is possible (though unlikely) that the performance regression was due to JIT compilation slowdown. In the JIT we try hard to use O(1) algorithms instead of O(N) ones, O(N) instead of O(N^2), etc. It's obvious once you state it, but it is non-trivial to avoid adding an exponent (turning O(1) into O(N), etc.). Generally speaking, if there is a problem in the JIT due to optimization order (as there seems to have been on ARM) the solution is almost certainly not to add to the exponent of our algorithm, but to find some sort of targeted extraction of items from the worklist in the needed order. ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kozlov at oracle.com Mon Nov 3 18:10:46 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Nov 2014 10:10:46 -0800 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <5457BE21.9010402@oracle.com> References: <54542F72.3000609@oracle.com> <5454303F.8060805@oracle.com> <5457BE21.9010402@oracle.com> Message-ID: <5457C526.2030205@oracle.com> On 11/3/14 9:40 AM, Dean Long wrote: > By the way, do we know why the lcm change caused a regression? Is reordering the worklist > using pop() a design feature? No, putting last element in a place of removed one is not used here by "generate optimal code" design but because it is fastest way to remove element from array. If we only used selection code at lines 534-548 it would not matter which is the order. The problem in special nodes (lines 462-469) which depends on the order and this, I think caused, the problem. Unfortunately this code in lcm does not produce as optimal code as we would like (see examples in RFEs). We were unlucky that reordering of nodes due to Atom changes caused this regression. Thanks, Vladimir > > dl > > On 10/31/2014 5:58 PM, Vladimir Kozlov wrote: >> I forgot to say that lcm.cpp changes were small part of 8052081 changes. The rest of changes will stay. >> >> Thanks, >> Vladimir >> >> On 10/31/14 5:55 PM, Vladimir Kozlov wrote: >>> http://cr.openjdk.java.net/~kvn/8059780/webrev/ >>> https://bugs.openjdk.java.net/browse/JDK-8059780 >>> >>> We found performance regression (3-5%) on x64 after changes 8052081 [1] >>> were pushed. It is caused by changes in lcm.cpp. Those changes were done >>> because Atom is in-order cpu and some benchmarks on Atom showed benefit >>> from these changes. >>> >>> But Atom is not our core platform and the regression is big. >>> >>> Backout changes made in lcm.cpp in jdk9 and 8u40 (8052081 was backported >>> to 8u40). >>> >>> We have RFEs to improve block's instructions order which may help both >>> platforms: >>> >>> https://bugs.openjdk.java.net/browse/JDK-6958833 >>> https://bugs.openjdk.java.net/browse/JDK-7101232 >>> >>> Tested with SPECjvm2008-mpegaudio. >>> >>> Thanks, >>> Vladimir >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8052081 > From david.r.chase at oracle.com Mon Nov 3 20:41:30 2014 From: david.r.chase at oracle.com (David Chase) Date: Mon, 3 Nov 2014 15:41:30 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5457E0F9.8090004@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> Message-ID: On 2014-11-03, at 3:09 PM, Peter Levart wrote: > Hi David, > > I was thinking about the fact that java.lang.invoke code is leaking into java.lang.Class. Perhaps, If you don't mind rewriting the code, a better code structure would be, if j.l.Class changes only consisted of adding a simple: > ? > This way all the worries about ordering of writes into array and/or size are gone. The array is still used to quickly search for an element, but VM only scans the linked-list. > > What do you think of this? I?m not sure. I know Coleen Ph would like to see that happen. A couple of people have vague plans to move more of the MemberName resolution into core libs. (Years ago I worked on a VM where *all* of this occurred in Java, but some of it was ahead of time.) I heard mention of ?we want to put more stuff in there? but I got the impression that already happened (there?s reflection data, for example) so I?m not sure that makes sense. There?s also a proposal from people in the runtime to just use a jmethodid, take the hit of an extra indirection, and no need to for this worrisome jvm/java concurrency. And if we instead wrote a hash table that only grew, and never relocated elements, we could (I think) allow non-synchronized O(1) probes of the table from the Java side, synchronized O(1) insertions from the Java side, and because nothing moves, a smaller dance with the VM. I?m rather tempted to look into this ? given the amount of work it would take to do the benchmarking to see if (a) jmethodid would have acceptable performance or (b) the existing costs are too high, I could instead just write fast code and be done. And another way to view this is that we?re now quibbling about performance, when we still have an existing correctness problem that this patch solves, so maybe we should just get this done and then file an RFE. David -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From dean.long at oracle.com Mon Nov 3 20:58:23 2014 From: dean.long at oracle.com (Dean Long) Date: Mon, 03 Nov 2014 12:58:23 -0800 Subject: [9] RFR(XS) 8059780: SPECjvm2008-MPEG performance regressions on x64 platforms In-Reply-To: <5457C526.2030205@oracle.com> References: <54542F72.3000609@oracle.com> <5454303F.8060805@oracle.com> <5457BE21.9010402@oracle.com> <5457C526.2030205@oracle.com> Message-ID: <5457EC6F.2020009@oracle.com> I see now. Thanks Vladimir and John. dl On 11/3/2014 10:10 AM, Vladimir Kozlov wrote: > On 11/3/14 9:40 AM, Dean Long wrote: >> By the way, do we know why the lcm change caused a regression? Is >> reordering the worklist >> using pop() a design feature? > > No, putting last element in a place of removed one is not used here by > "generate optimal code" design but because it is fastest way to remove > element from array. > If we only used selection code at lines 534-548 it would not matter > which is the order. > The problem in special nodes (lines 462-469) which depends on the > order and this, I think caused, the problem. > > Unfortunately this code in lcm does not produce as optimal code as we > would like (see examples in RFEs). > We were unlucky that reordering of nodes due to Atom changes caused > this regression. > > Thanks, > Vladimir > >> >> dl >> >> On 10/31/2014 5:58 PM, Vladimir Kozlov wrote: >>> I forgot to say that lcm.cpp changes were small part of 8052081 >>> changes. The rest of changes will stay. >>> >>> Thanks, >>> Vladimir >>> >>> On 10/31/14 5:55 PM, Vladimir Kozlov wrote: >>>> http://cr.openjdk.java.net/~kvn/8059780/webrev/ >>>> https://bugs.openjdk.java.net/browse/JDK-8059780 >>>> >>>> We found performance regression (3-5%) on x64 after changes 8052081 >>>> [1] >>>> were pushed. It is caused by changes in lcm.cpp. Those changes were >>>> done >>>> because Atom is in-order cpu and some benchmarks on Atom showed >>>> benefit >>>> from these changes. >>>> >>>> But Atom is not our core platform and the regression is big. >>>> >>>> Backout changes made in lcm.cpp in jdk9 and 8u40 (8052081 was >>>> backported >>>> to 8u40). >>>> >>>> We have RFEs to improve block's instructions order which may help both >>>> platforms: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-6958833 >>>> https://bugs.openjdk.java.net/browse/JDK-7101232 >>>> >>>> Tested with SPECjvm2008-mpegaudio. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8052081 >> From igor.veresov at oracle.com Mon Nov 3 21:03:36 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Mon, 3 Nov 2014 11:03:36 -1000 Subject: RFR(S) 8062591: SPARC PICL causes significantly longer startup times Message-ID: <0D2F0C0A-A0F4-4D8B-B60E-098BCB418635@oracle.com> Querying cache line sizes using PICL was noticed to take a lot of time mostly because of the depth-first search in a big string-keyed tree. I did the following optimizations that take care of the regression: - Combined L1 and L2 traversals in a single pass. - Based on the observation that the CPU-related part of the tree is closer to the side the traversal starts on, and the records about CPUs are all siblings we can stop the traversal once we?ve seen all the CPUs (os::processor_count()), leaving a significant part of the tree untouched. - Avoid double traversal on Fujitsu boxes, just look once for the ?core? class. JBS: https://bugs.openjdk.java.net/browse/JDK-8062591 Webrev: http://cr.openjdk.java.net/~iveresov/8062591/webrev.00 igor From Dmitry.Fazunenko at oracle.com Mon Nov 3 21:12:06 2014 From: Dmitry.Fazunenko at oracle.com (Dmitry Fazunenko) Date: Tue, 04 Nov 2014 00:12:06 +0300 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <545783A1.3050300@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> Message-ID: <5457EFA6.7050404@oracle.com> Hi Bengt, That's great that we have very closed visions! The general comment: currently, jtreg doesn't support any sort of plugins, so you can't provide a VM specific handler of the @requires or another tag. This is very annoying limitation and we have to live with it. A few more comments inline. On 03.11.2014 16:31, Bengt Rutisson wrote: > > > Hi Dima, > > Answers inline. > > On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >> Hi Bengt, >> >> Thanks a lot for your detailed feedback, we appreciate it very much! >> >> See comments inline. >> >> On 31.10.2014 1:09, Bengt Rutisson wrote: >>> >>> Hi Evgeniya, >>> >>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>> Hi, >>>> >>>> Please review changes for 8062537, the OpenJDK/hotspot part of the >>>> JDK-8019361 >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>> >>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>> another GC. >>>> Solution: Such tests marked with the jtreg tag "requires" to skip >>>> test if there is a conflict >>> >>> Thanks for fixing this! It is really great that we finally start >>> sorting this out. >>> >>> First a general comment. The @requires tag has been developed >>> without much cooperation with the GC team. We did have a lot of >>> feedback when it was first presented a year ago, but it does not >>> seem like this feedback was incorporated into the @requires that was >>> eventually built. >> >> We tried to implement as much developer's wishes as possible. But not >> everything is possible, sorry for that. > > Yes, I'm sure you have done your best. It's just that we have been > requesting this feature for 3 years and I was expecting us to be able > to influence the feature much more than was the case now. My personal hope: @requires will address ~90% of existing issues. > >> >>> >>> I think this change that gets proposed now is a big step forward and >>> I won't object to it. But I am pretty convinced that we will soon >>> run in to the limitations of the current @requires implementation >>> and we will have to redo this work. >>> >>> Some of the points I don't really like about the @requires tag are: >>> >>> - the "vm.gc" abstraction is more limiting than helping. It would >>> have been better to just "require" any command line flag. >> "vm.gc" is an alias to a very popular flag. It's also possible to use: >> vm.opt.UseG1GC == true instead. >> >> The table with all vars available in jtreg: >> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names > > The problem with having this matching built in to JTreg is that it > makes it very hard to change. When we discussed this a year ago I > think we said that JTreg should only provide a means to test against > the command line and a hook for running some java code in the > @requires tag. That way we could put logic like this in a test library > that is under our control. This would make it easy for us to change > and also enables us to use different logic for different versions. I would be glad to have own harness... > >> >>> - the requirement should be per @run tag. Right now we have to do >>> what you did in this change and use vm.gc=null even when some tests >>> could actually have been run when a GC was specified. >> it would be great, but it will unlikely happen in jtreg, as well as >> test case support. > > what do you mean with test case support? Hi Evgeniya, Under test case support I mean ability to treat each @run as a separate test. Now @test @run -XX:g1RegSize=1m MyTest @run -XX:g1RegSize=2m MyTest @run -XX:g1RegSize=4m MyTest class MyTest { } is always a single test. You can't exclude, or re-run a part of it. > >> >>> - there are many tests that require more than just a specific GC. >>> Often there are other flags that can't be changed either for the >>> test to work properly. >> >> yes. conflicting GC is just the most popular problem caused by >> conflicting options. >> If we address this issue and we are satisfied with solution, we could >> move further. > > Yes, I agree that taking one step at the time is good. Personally I > would have preferred that the first step was a "just run the command > line as specified in the @run tag" step. > >> >>> >>> Maybe this is not the right place to discuss the current >>> implementation of the @requires tag. I just want to say that I'm not >>> too happy about how the @requires tag turned out. But assuming we >>> have to use it the way it is now I guess the proposed changeset >>> looks good. >> >> yes, this thread is about change made by Evgeniya, not about jtreg :) >> And thanks for reviewing it! > > Agreed. And as I said, I think the patch looks ok. I have not looked > at all tests. But if they now pass with the combinations that we test > with I guess they should be ok. Excellent! Thanks a lot! > >> >>> >>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>> without any GC flag). Tests are being excluded as expected. No >>>> tests failed because of the conflict. >>> Have you tested with -Xconcgc too? It's an alias for >>> -XX:+UseConcMarkSweepGC. >> >> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) > > Ok. Thanks. > >> >>> >>> I think some of the test, like >>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >>> with -XX:+UseParNewGC. Others, like >>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >>> with -XX:-UseParNewGC. Could you test these two cases too? >> >> These two tests ignore vm flags. >> Add @requires here is not necessary, but it will allow not execute >> the tests when not needed. >> So, if we run HS tests with 4 GC, we don't need to run these tests 4 >> times, 1 should be enough. > > Do we really want to use the @requires functionality for this purpose? > It seems like a way of misusing @requires. If we just want the tests > to be run once I think Leonid's approach with tests lists seems more > suitable. No, it's not a purpose of course, it's just side effect :) > But are you sure that this is the reason for the @requires in this > case? TestDefNewCMS does sound like a test that is DefNew specific. I > don't see a reason to run it with ParNew. If it doesn't fail today it > should probably be changed so that it does fail if it is run with the > wrong GC. @requires - is not the silver bullet, but it's quite easy way to solve a lot of issues. I hope, @requires will allow to reduce the number of "selfish" tests, which produce a new java process to ignore vm flags coming from outside. No @requires, no other mechanism could 100% protect a test from running with conflicting options, but this is not the goal. If one runs tests with an exotic option, like a new G2 collector, there shouldn't mass failures caused by options conflicts. But a few failures could be handled manually. > >> >>> Similarly it looks to me like there are tests that will fail if you >>> run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>> >>> Just a heads up. These two tests will soon be removed. I'm about to >>> push a changeset that removes them: >>> >>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >> okay, thank for letting us know. >> >>> >>> Is there some way of making sure that all tests are run at one time >>> or another. With this change there is a risk that some tests are >>> never run and always skipped. Will we somehow be tracking what gets >>> skipped and make sure that all tests are at least run once with the >>> correct GC so that it is not skipped all the time? >> >> This is a very good question! >> jtreg now doesn't report skipped tests, hopefully it will do soon, >> after getting fix of: >> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >> >> And yes, tracking tests which are not run is important thing. >> @requires - is not the only to exclude test from execution. >> >> Other examples: >> >> /* >> *@ignore >> *@test >> */ >> ... >> >> /*@bug 4445555 >> *@test >> */ >> ... >> Such tests will never be run, because jtreg treats as test only files >> with @test on the first place... >> >> So, making sure that tests do not disappear is important SQE task, >> we know about that, we're thinking on solution (may be very >> actively). But this subject for another discussion, not within RFR :) > > Right. Glad to hear that you are actively working on this! I was going to say "not very actively", but never mind, we know about this problem. With introducing @requires mechanism it will become more important! Thanks for your comments! -- Dima > > Bengt > >> >> Thanks, >> Dima >> >> >> >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.thalinger at oracle.com Mon Nov 3 21:30:07 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 3 Nov 2014 13:30:07 -0800 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> Message-ID: <00703487-00EB-4E43-9613-01EE9EE64147@oracle.com> > On Nov 3, 2014, at 12:41 PM, David Chase wrote: > > > On 2014-11-03, at 3:09 PM, Peter Levart wrote: > >> Hi David, >> >> I was thinking about the fact that java.lang.invoke code is leaking into java.lang.Class. Perhaps, If you don't mind rewriting the code, a better code structure would be, if j.l.Class changes only consisted of adding a simple: >> ? > >> This way all the worries about ordering of writes into array and/or size are gone. The array is still used to quickly search for an element, but VM only scans the linked-list. >> >> What do you think of this? > > I?m not sure. I know Coleen Ph would like to see that happen. > > A couple of people have vague plans to move more of the MemberName resolution into core libs. > (Years ago I worked on a VM where *all* of this occurred in Java, but some of it was ahead of time.) > > I heard mention of ?we want to put more stuff in there? but I got the impression that already happened > (there?s reflection data, for example) so I?m not sure that makes sense. > > There?s also a proposal from people in the runtime to just use a jmethodid, take the hit of an extra indirection, > and no need to for this worrisome jvm/java concurrency. > > And if we instead wrote a hash table that only grew, and never relocated elements, we could > (I think) allow non-synchronized O(1) probes of the table from the Java side, synchronized > O(1) insertions from the Java side, and because nothing moves, a smaller dance with the > VM. I?m rather tempted to look into this ? given the amount of work it would take to do the > benchmarking to see if (a) jmethodid would have acceptable performance or (b) the existing > costs are too high, I could instead just write fast code and be done. ?but you still have to do the benchmarking. Let?s not forget that there was a performance regression with the first C++ implementation of this. > > And another way to view this is that we?re now quibbling about performance, when we still > have an existing correctness problem that this patch solves, so maybe we should just get this > done and then file an RFE. > > David > From vladimir.kozlov at oracle.com Mon Nov 3 21:47:21 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Nov 2014 13:47:21 -0800 Subject: RFR(S) 8062591: SPARC PICL causes significantly longer startup times In-Reply-To: <0D2F0C0A-A0F4-4D8B-B60E-098BCB418635@oracle.com> References: <0D2F0C0A-A0F4-4D8B-B60E-098BCB418635@oracle.com> Message-ID: <5457F7E9.4010002@oracle.com> Next check should use != 0 (since we can use 31st bit): (features & sparc64_family_m) > 0 Otherwise it looks good. Thanks, Vladimir On 11/3/14 1:03 PM, Igor Veresov wrote: > Querying cache line sizes using PICL was noticed to take a lot of time mostly because of the depth-first search in a big string-keyed tree. > I did the following optimizations that take care of the regression: > - Combined L1 and L2 traversals in a single pass. > - Based on the observation that the CPU-related part of the tree is closer to the side the traversal starts on, and the records about CPUs are all siblings we can stop the traversal once we?ve seen all the CPUs (os::processor_count()), leaving a significant part of the tree untouched. > - Avoid double traversal on Fujitsu boxes, just look once for the ?core? class. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8062591 > Webrev: http://cr.openjdk.java.net/~iveresov/8062591/webrev.00 > > igor > From igor.veresov at oracle.com Mon Nov 3 22:34:57 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Mon, 3 Nov 2014 12:34:57 -1000 Subject: RFR(S) 8062591: SPARC PICL causes significantly longer startup times In-Reply-To: <5457F7E9.4010002@oracle.com> References: <0D2F0C0A-A0F4-4D8B-B60E-098BCB418635@oracle.com> <5457F7E9.4010002@oracle.com> Message-ID: You?re right, thanks for noticing that. Webrev: http://cr.openjdk.java.net/~iveresov/8062591/webrev.01 igor On Nov 3, 2014, at 11:47 AM, Vladimir Kozlov wrote: > Next check should use != 0 (since we can use 31st bit): > > (features & sparc64_family_m) > 0 > > Otherwise it looks good. > > Thanks, > Vladimir > > On 11/3/14 1:03 PM, Igor Veresov wrote: >> Querying cache line sizes using PICL was noticed to take a lot of time mostly because of the depth-first search in a big string-keyed tree. >> I did the following optimizations that take care of the regression: >> - Combined L1 and L2 traversals in a single pass. >> - Based on the observation that the CPU-related part of the tree is closer to the side the traversal starts on, and the records about CPUs are all siblings we can stop the traversal once we?ve seen all the CPUs (os::processor_count()), leaving a significant part of the tree untouched. >> - Avoid double traversal on Fujitsu boxes, just look once for the ?core? class. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8062591 >> Webrev: http://cr.openjdk.java.net/~iveresov/8062591/webrev.00 >> >> igor >> From vladimir.kozlov at oracle.com Mon Nov 3 23:07:53 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Nov 2014 15:07:53 -0800 Subject: RFR(S) 8062591: SPARC PICL causes significantly longer startup times In-Reply-To: References: <0D2F0C0A-A0F4-4D8B-B60E-098BCB418635@oracle.com> <5457F7E9.4010002@oracle.com> Message-ID: <54580AC9.1070606@oracle.com> Good. Thanks, Vladimir On 11/3/14 2:34 PM, Igor Veresov wrote: > You?re right, thanks for noticing that. > > Webrev: http://cr.openjdk.java.net/~iveresov/8062591/webrev.01 > > igor > > On Nov 3, 2014, at 11:47 AM, Vladimir Kozlov wrote: > >> Next check should use != 0 (since we can use 31st bit): >> >> (features & sparc64_family_m) > 0 >> >> Otherwise it looks good. >> >> Thanks, >> Vladimir >> >> On 11/3/14 1:03 PM, Igor Veresov wrote: >>> Querying cache line sizes using PICL was noticed to take a lot of time mostly because of the depth-first search in a big string-keyed tree. >>> I did the following optimizations that take care of the regression: >>> - Combined L1 and L2 traversals in a single pass. >>> - Based on the observation that the CPU-related part of the tree is closer to the side the traversal starts on, and the records about CPUs are all siblings we can stop the traversal once we?ve seen all the CPUs (os::processor_count()), leaving a significant part of the tree untouched. >>> - Avoid double traversal on Fujitsu boxes, just look once for the ?core? class. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8062591 >>> Webrev: http://cr.openjdk.java.net/~iveresov/8062591/webrev.00 >>> >>> igor >>> > From igor.veresov at oracle.com Mon Nov 3 23:25:15 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Mon, 3 Nov 2014 13:25:15 -1000 Subject: RFR(S) 8062591: SPARC PICL causes significantly longer startup times In-Reply-To: <54580AC9.1070606@oracle.com> References: <0D2F0C0A-A0F4-4D8B-B60E-098BCB418635@oracle.com> <5457F7E9.4010002@oracle.com> <54580AC9.1070606@oracle.com> Message-ID: <75CA80C7-5E51-410A-95EF-0597E1AF1580@oracle.com> Thanks, Vladimir! igor > On Nov 3, 2014, at 1:07 PM, Vladimir Kozlov wrote: > > Good. > > Thanks, > Vladimir > >> On 11/3/14 2:34 PM, Igor Veresov wrote: >> You?re right, thanks for noticing that. >> >> Webrev: http://cr.openjdk.java.net/~iveresov/8062591/webrev.01 >> >> igor >> >>> On Nov 3, 2014, at 11:47 AM, Vladimir Kozlov wrote: >>> >>> Next check should use != 0 (since we can use 31st bit): >>> >>> (features & sparc64_family_m) > 0 >>> >>> Otherwise it looks good. >>> >>> Thanks, >>> Vladimir >>> >>>> On 11/3/14 1:03 PM, Igor Veresov wrote: >>>> Querying cache line sizes using PICL was noticed to take a lot of time mostly because of the depth-first search in a big string-keyed tree. >>>> I did the following optimizations that take care of the regression: >>>> - Combined L1 and L2 traversals in a single pass. >>>> - Based on the observation that the CPU-related part of the tree is closer to the side the traversal starts on, and the records about CPUs are all siblings we can stop the traversal once we?ve seen all the CPUs (os::processor_count()), leaving a significant part of the tree untouched. >>>> - Avoid double traversal on Fujitsu boxes, just look once for the ?core? class. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8062591 >>>> Webrev: http://cr.openjdk.java.net/~iveresov/8062591/webrev.00 >>>> >>>> igor >>>> >> From vladimir.kozlov at oracle.com Tue Nov 4 00:27:10 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 03 Nov 2014 16:27:10 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> Message-ID: <54581D5E.2090706@oracle.com> Sorry, it took so much time. I was confused by number of files you changed. In general it seems good to me. Do you know why GCM scheduled the load into previous block? Usually loads are placed near their uses in: 0c7 B13: # B17 B14 <- B11 Freq: 0.768468 0c7 movzwl R8, [RCX + #24 + R10 << #1] # ushort/char << SEGV 0cd addl RDI, #-2 # int 0d0 movl R9, #-2 # int 0d6 testl R10, R10 0d9 jle,s B17 P=0.000001 C=-1.000000 0d9 0db B14: # B21 B15 <- B13 Freq: 0.768467 0db testl R8, R8 0de je,s B21 P=0.100000 C=-1.000000 castnode.hpp: Fields names: _required --> _carry_dependency. I am not convinced that you need _inexact. Based on your code _inexact can be converted from 'true' to 'false' only when phase->type(cmp->in(2))->is_int()->is_con() and all conditions at the beginning of CastIINode::Ideal() are true. But if input's type is narrowed later we can still improve CastII's type. Do you think we will get worse code if we don't do Identity for _carry_dependency CastII nodes? If _carry_dependency is set we can't replace it with an other node, I think, (unless all inputs are matching - hash() and cmp()). castnode.cpp The type change belongs to Value() not Ideal(). Especially if you don't need to set _inexact field and create new CastII node for that. Use fatal with message which includes BoolTest::mask name instead of ShouldNotReachHere(). loopTransform.cpp insert_castii_before_loop() --> cast_incr_before_loop() You missed registration of new CastII nodes - register_new_node(). opaquenode.cpp What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. There is loop_limit_check predicate before a counted loop which has opaqued limit. It . Those opaque nodes are removed by cleanup_loop_predicates(). Why Phi is not put on worklist at that time? Thanks, Vladimir On 10/6/14 2:49 AM, Roland Westrelin wrote: > > Anyone? > > Roland. > > On Sep 17, 2014, at 7:03 PM, Roland Westrelin wrote: > >> http://cr.openjdk.java.net/~roland/8054478/webrev.00/ >> >> The IR after parsing is this in pseudo code: >> >> i_main = 0; >> while(true) { >> if (i_main >= 1000000) return; >> i_main++; >> char[] array = new char[1]; >> if (pattern1 != null) { >> i = 0; >> while(true) { >> if (i >= 0) { >> do { >> if (pattern0 == null) uncommon_trap; >> if (pattern0.length u< i) uncommon_trap; >> if (pattern0[i] != 0) { >> if (pattern1.length u< i) uncommon_trap; >> if (pattern1[i] != 0) { >> goto out1; >> } >> } >> i?; >> pos?; >> if (pos != -1) { >> goto out2; >> } >> } while (i >= 0); >> goto out1; >> } >> out2: >> c = array[pos]; >> } >> } >> out1: >> } >> >> The do {} while loop is a CountedLoop. The null check & range check are moved out of the loop. Then a pre and a post loops are created and the body is unrolled. The pattern0[i] LoadNode nodes in the unrolled body have a control that is set to the range check before the pre loop. In the unrolled loop, because of the if (pos != -1) test in the first copy of the loop, the compiler finds that in the second copy of the loop body pos is != -1 and so the loop is exited before reaching the end of the unrolled loop. The back branch of the unrolled loop is dead. The compiler optimizes the CountedLoopNode of the unrolled loop out because it doesn?t have a back branch anymore, PhiNodes are removed and the LoadNode for pattern0[i] in the unroll loop body is now independent of the input control of the CountedLoop. Its control is still set to the range check before the pre loop. In the generated code, the pre loop is followed by the pattern1[i] access which is incorrect because it happens before the if (i >= 0) that dominated the unrolled loop before it was removed. >> >> The graph is good as long as the CountedLoop is not removed because the LoadNode depends on a PhiNode of the CountedLoop. But as soon as the CountedLoop is removed and the PhiNode is disconnected, the LoadNode no longer depends on the check that guarded the CountedLoop. >> >> Vladimir suggested (in a private conversation) that I use a CastII (control dependent on the if that guards the main loop with as input the input value to the induction variable in the loop) as a way to guarantee correct dependencies. There are 2 problems with this: >> >> 1) the CastII nodes are removed after CCP. I added a flag to the CastII nodes so we can mark CastII nodes that can?t be removed and should be left as they are post-CCP. >> 2) we don?t know the range of values that are possible on the if branch that enters the main loop right away (it can be adjusted as loop opts are applied). We may actually never know it if it depends on non constants. If we use a type for a range that is too wide in the CastII the CastII may be optimized out incorrectly. I added another flag to the CastII to mark nodes that cannot be optimized. In CastII:Ideal I added code that tries to figure out a better type for the CastII so we may be able to optimize the CastII as the compiler optimizes the IR. >> >> Another independent issue here is that the main loop is actually never executed because the loop of the test is a single iteration loop so the compiler should be able to optimize out the main loop entirely but it doesn?t. PhiNode::Value() has this code: >> >> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >> if( l && l->can_be_counted_loop(phase) && >> ((const Node*)l->phi() == this) ) { // Trip counted loop! >> >> ... >> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >> return TypeInt::make(lo->_lo,hi->_hi,3); >> } >> } >> } >> >> that returns an accurate value for the Phi node of a CountedLoop. In this case we want the pre loop?s Phi to return an accurate type that would then make the guard of the main loop optimize. The pre loop is guarded by a test on an opaque node which is later removed. If PhiNode::Value() is called before the Opaque1 node is removed then, the limit type is int and it cannot return anything accurate. If it?s called after, then limit type is a constant and PhiNode::Value() also returns a constant for the pre-loop. Then the main loop is optimized out. To fix this: I added code to Opaque1Node::Ideal so it reenqueues the Phi when the opaque node is removed. >> >> Roland. > From albert.noll at oracle.com Tue Nov 4 08:38:38 2014 From: albert.noll at oracle.com (Albert Noll) Date: Tue, 04 Nov 2014 09:38:38 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <5457C2D5.9080401@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> Message-ID: <5458908E.1050909@oracle.com> Hi, please see comments inline. On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: > I agree with additional check tak != TypeKlassPtr::OBJECT (solution 2). > > Instead of creating new LoadKlassNode::make() method you can set > control in parseHelper.cpp since it is the only place which set it: > > Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), > p2, tak); > if (always_see_exact_class) { > a_e_klass->init_req(MemNode::Control, control()); > } > a_e_klass = _gvn.transform(a_e_klass); > An alternative solution is to make a default parameter for control. This way we can set the control in make() without having to add a new make() function. > Also an other reason to have control edge in this place is the address > of this klass load is based on ConP constant. In a normal case after > class check you should have CheckCastPP attached to control and used > as address's base. > > And agree with Roland's suggestion about Op_LoadKlass check. > What you are suggesting is to make a super class (LoadNode) aware of its subclasses (LoadPNode and LoadKlassNode). From a software engineering point of view, we should try to avoid doing that. Another solution would be to add a method (e.g., can_remove_control()) to LoadNode, LoadPNode, and LoadKlassNode. The function returns 'false' for LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What do you think? Best, Albert > Thanks, > Vladimir > > On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >> Hi, >> >> >> please review the following patch. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >> >> >> Problem: >> >> >> We have five tests that fail with SIGSEGV in our nightlies: >> >> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >> >> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >> >> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >> >> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >> >> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >> >> All tests fail when executing the C2-compiled version of >> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >> >> >> 489: private void inflateSpine() { >> 490: if (spine == null) { >> 491: spine = newArrayArray(MIN_SPINE_SIZE); >> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >> 493: spine[0] = curChunk; >> 494: } >> 495: } >> >> >> The failure is due to the 'aastore' at line 493; the failure is caused >> by the monomorphic array check optimization >> (-XX:+MonomorphicArrayCheck, enabled by default). >> >> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >> profiling information) that inflateSpine() has two hot receiver types, >> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >> >> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >> >> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >> double[][]. >> >> Please consider the following pseudo Java code that illustrates how C2 >> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on >> the two hot receiver types SpinedBuffer.OfInt and >> SpinedBuffer.OfDouble: >> >> >> static void inflateSpine(boolean isOfDouble, Object curChunk) { >> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >> spine[0] = curChunk; >> } >> >> >> If MonomorphicArrayCheck is disabled, C2 checks the element type >> of 'spine' *at runtime*. The check looks something like: >> >> >> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >> throw new ArrayStoreException(); >> } >> >> >> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >> check like this: >> >> >> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >> throw new ArrayStoreException(); >> } >> >> >> where TYPE is the type of the 'spine' variable, as it is determined by >> C2 *at compile time*. The optimization treats TYPE as a constant and >> thus saves a load from memory + an offset calculation (I think). >> >> The problem is that due to 'spine' being either of type int[][] or of >> type double[][], C2 determines that TYPE==java/lang/Object. >> >> As a result, when the inflateSpine() method is executed, it first >> loads the >> Klass* corresponding to java/lang/Object (TYPE). Let us call this Klass* >> array_klass. Then, the method obtains a Klass* from >> array_klass->_element_klass. >> Then, the method reads from offset 20 from >> array_klass->_element_klass, which >> results in a SIGSEGV. >> >> The reason for the SIGSEGV is that the Klass* array_klass is of type >> java/lang/Object and is therefore represented as an 'InstanceKlass'. >> But _element_klass is defined only in objects of type 'ObjArrayKlass'. >> The compiler reads array_klass->_element_klass because it expects the >> destination of an 'aastore' to be an array, which is not true if >> TYPE==java/lang/Object. >> >> >> Solution: >> >> >> The compiler already checks if the compile-time type of the array >> (TYPE) is >> the same as the array's type at runtime. If that check fails, the method >> branches to an uncommon trap. In our case the SIGSEGV happens because >> the load of array_klass->_element_klass "floats" above the check. >> >> I propose two solutions: >> >> Solution 1. Add a control edge between the IfTrue branch of the check >> and >> the load of _array_klass->_element_klass. That prohibits the load of >> _array_klass->element_klass floating above the check. >> >> Solution 2. In addition to the changes in Solution 1, the compiler >> checks if >> the compile-time type of the array (TYPE) is java/lang/Object. If >> TYPE==java/lang/Object, the compiler should not assume array_type to be >> TYPE, but should determine array type at runtime instead. >> >> The reason is for Solution 2 is: We can't do an array store into a >> java/lang/Object. >> So, if we see TYPE==java/lang/Object at compile time, it is pretty >> sure that the >> method will deoptimize at runtime. Instead of assuming TYPE to be >> java/lang/Object, >> we read at runtime the type of the array store's destination. That >> still gives us some >> chance that the compiled version of the method will successfully >> execute. The >> additional check is in parseHelper.cpp in Parse::array_store_check() >> on line 171. >> >> >> Webrev: >> >> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >> >> >> Testing (for both solutions): >> >> JPRT, manual testing with failing test cases. >> >> >> Please note that both solutions are somewhat "hacky", because that >> allowed me to >> produce a solution within reasonable time and with reasonable effort. >> Any suggestions >> or feedback is welcome. >> >> >> Thank you and best regards, >> >> >> Zoltan >> From bengt.rutisson at oracle.com Tue Nov 4 08:40:43 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 04 Nov 2014 09:40:43 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5457EFA6.7050404@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> Message-ID: <5458910B.2070100@oracle.com> Hi Dima, Thanks for the answers. I think the currently proposed patch is a good start. We will have to evolve the @requires tag in the future, but let's have that discussion separate from this review. And we can start that discussion later when we have more experience with the current version of @requires. Thanks for doing this! Bengt On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: > Hi Bengt, > > That's great that we have very closed visions! > > The general comment: currently, jtreg doesn't support any sort of > plugins, so you can't provide a VM specific handler of the @requires > or another tag. This is very annoying limitation and we have to live > with it. > > A few more comments inline. > > > On 03.11.2014 16:31, Bengt Rutisson wrote: >> >> >> Hi Dima, >> >> Answers inline. >> >> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>> Hi Bengt, >>> >>> Thanks a lot for your detailed feedback, we appreciate it very much! >>> >>> See comments inline. >>> >>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>> >>>> Hi Evgeniya, >>>> >>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>> Hi, >>>>> >>>>> Please review changes for 8062537, the OpenJDK/hotspot part of the >>>>> JDK-8019361 >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>> >>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>> another GC. >>>>> Solution: Such tests marked with the jtreg tag "requires" to skip >>>>> test if there is a conflict >>>> >>>> Thanks for fixing this! It is really great that we finally start >>>> sorting this out. >>>> >>>> First a general comment. The @requires tag has been developed >>>> without much cooperation with the GC team. We did have a lot of >>>> feedback when it was first presented a year ago, but it does not >>>> seem like this feedback was incorporated into the @requires that >>>> was eventually built. >>> >>> We tried to implement as much developer's wishes as possible. But >>> not everything is possible, sorry for that. >> >> Yes, I'm sure you have done your best. It's just that we have been >> requesting this feature for 3 years and I was expecting us to be able >> to influence the feature much more than was the case now. > > My personal hope: @requires will address ~90% of existing issues. > >> >>> >>>> >>>> I think this change that gets proposed now is a big step forward >>>> and I won't object to it. But I am pretty convinced that we will >>>> soon run in to the limitations of the current @requires >>>> implementation and we will have to redo this work. >>>> >>>> Some of the points I don't really like about the @requires tag are: >>>> >>>> - the "vm.gc" abstraction is more limiting than helping. It would >>>> have been better to just "require" any command line flag. >>> "vm.gc" is an alias to a very popular flag. It's also possible to use: >>> vm.opt.UseG1GC == true instead. >>> >>> The table with all vars available in jtreg: >>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >> >> The problem with having this matching built in to JTreg is that it >> makes it very hard to change. When we discussed this a year ago I >> think we said that JTreg should only provide a means to test against >> the command line and a hook for running some java code in the >> @requires tag. That way we could put logic like this in a test >> library that is under our control. This would make it easy for us to >> change and also enables us to use different logic for different versions. > > I would be glad to have own harness... > >> >>> >>>> - the requirement should be per @run tag. Right now we have to do >>>> what you did in this change and use vm.gc=null even when some tests >>>> could actually have been run when a GC was specified. >>> it would be great, but it will unlikely happen in jtreg, as well as >>> test case support. >> >> what do you mean with test case support? Hi Evgeniya, > > Under test case support I mean ability to treat each @run as a > separate test. Now > > @test > @run -XX:g1RegSize=1m MyTest > @run -XX:g1RegSize=2m MyTest > @run -XX:g1RegSize=4m MyTest > class MyTest { > } > > is always a single test. You can't exclude, or re-run a part of it. > > >> >>> >>>> - there are many tests that require more than just a specific GC. >>>> Often there are other flags that can't be changed either for the >>>> test to work properly. >>> >>> yes. conflicting GC is just the most popular problem caused by >>> conflicting options. >>> If we address this issue and we are satisfied with solution, we >>> could move further. >> >> Yes, I agree that taking one step at the time is good. Personally I >> would have preferred that the first step was a "just run the command >> line as specified in the @run tag" step. >> >>> >>>> >>>> Maybe this is not the right place to discuss the current >>>> implementation of the @requires tag. I just want to say that I'm >>>> not too happy about how the @requires tag turned out. But assuming >>>> we have to use it the way it is now I guess the proposed changeset >>>> looks good. >>> >>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>> And thanks for reviewing it! >> >> Agreed. And as I said, I think the patch looks ok. I have not looked >> at all tests. But if they now pass with the combinations that we test >> with I guess they should be ok. > > Excellent! Thanks a lot! > >> >>> >>>> >>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>> without any GC flag). Tests are being excluded as expected. No >>>>> tests failed because of the conflict. >>>> Have you tested with -Xconcgc too? It's an alias for >>>> -XX:+UseConcMarkSweepGC. >>> >>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >> >> Ok. Thanks. >> >>> >>>> >>>> I think some of the test, like >>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >>>> with -XX:+UseParNewGC. Others, like >>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >>>> with -XX:-UseParNewGC. Could you test these two cases too? >>> >>> These two tests ignore vm flags. >>> Add @requires here is not necessary, but it will allow not execute >>> the tests when not needed. >>> So, if we run HS tests with 4 GC, we don't need to run these tests 4 >>> times, 1 should be enough. >> >> Do we really want to use the @requires functionality for this >> purpose? It seems like a way of misusing @requires. If we just want >> the tests to be run once I think Leonid's approach with tests lists >> seems more suitable. > > No, it's not a purpose of course, it's just side effect :) > > >> But are you sure that this is the reason for the @requires in this >> case? TestDefNewCMS does sound like a test that is DefNew specific. I >> don't see a reason to run it with ParNew. If it doesn't fail today it >> should probably be changed so that it does fail if it is run with the >> wrong GC. > > @requires - is not the silver bullet, but it's quite easy way to solve > a lot of issues. > > I hope, @requires will allow to reduce the number of "selfish" tests, > which produce a new java process to ignore vm flags coming from > outside. No @requires, no other mechanism could 100% protect a test > from running with conflicting options, but this is not the goal. > > If one runs tests with an exotic option, like a new G2 collector, > there shouldn't mass failures caused by options conflicts. But a few > failures could be handled manually. > > >> >>> >>>> Similarly it looks to me like there are tests that will fail if you >>>> run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>> >>>> Just a heads up. These two tests will soon be removed. I'm about to >>>> push a changeset that removes them: >>>> >>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>> okay, thank for letting us know. >>> >>>> >>>> Is there some way of making sure that all tests are run at one time >>>> or another. With this change there is a risk that some tests are >>>> never run and always skipped. Will we somehow be tracking what gets >>>> skipped and make sure that all tests are at least run once with the >>>> correct GC so that it is not skipped all the time? >>> >>> This is a very good question! >>> jtreg now doesn't report skipped tests, hopefully it will do soon, >>> after getting fix of: >>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>> >>> And yes, tracking tests which are not run is important thing. >>> @requires - is not the only to exclude test from execution. >>> >>> Other examples: >>> >>> /* >>> *@ignore >>> *@test >>> */ >>> ... >>> >>> /*@bug 4445555 >>> *@test >>> */ >>> ... >>> Such tests will never be run, because jtreg treats as test only >>> files with @test on the first place... >>> >>> So, making sure that tests do not disappear is important SQE task, >>> we know about that, we're thinking on solution (may be very >>> actively). But this subject for another discussion, not within RFR :) >> >> Right. Glad to hear that you are actively working on this! > > I was going to say "not very actively", but never mind, we know about > this problem. With introducing @requires mechanism it will become more > important! > > > Thanks for your comments! > > -- Dima > > >> >> Bengt >> >>> >>> Thanks, >>> Dima >>> >>> >>> >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dmitry.Fazunenko at oracle.com Tue Nov 4 11:32:48 2014 From: Dmitry.Fazunenko at oracle.com (Dmitry Fazunenko) Date: Tue, 04 Nov 2014 14:32:48 +0300 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5458910B.2070100@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> Message-ID: <5458B960.8000305@oracle.com> Nice plan! Please feel free to send me any feedback/questions regarding @requires Thanks, Dima On 04.11.2014 11:40, Bengt Rutisson wrote: > > Hi Dima, > > Thanks for the answers. I think the currently proposed patch is a good > start. We will have to evolve the @requires tag in the future, but > let's have that discussion separate from this review. And we can start > that discussion later when we have more experience with the current > version of @requires. > > Thanks for doing this! > Bengt > > > > On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >> Hi Bengt, >> >> That's great that we have very closed visions! >> >> The general comment: currently, jtreg doesn't support any sort of >> plugins, so you can't provide a VM specific handler of the @requires >> or another tag. This is very annoying limitation and we have to live >> with it. >> >> A few more comments inline. >> >> >> On 03.11.2014 16:31, Bengt Rutisson wrote: >>> >>> >>> Hi Dima, >>> >>> Answers inline. >>> >>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>> Hi Bengt, >>>> >>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>> >>>> See comments inline. >>>> >>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>> >>>>> Hi Evgeniya, >>>>> >>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>> Hi, >>>>>> >>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of >>>>>> the JDK-8019361 >>>>>> >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>> >>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>> another GC. >>>>>> Solution: Such tests marked with the jtreg tag "requires" to skip >>>>>> test if there is a conflict >>>>> >>>>> Thanks for fixing this! It is really great that we finally start >>>>> sorting this out. >>>>> >>>>> First a general comment. The @requires tag has been developed >>>>> without much cooperation with the GC team. We did have a lot of >>>>> feedback when it was first presented a year ago, but it does not >>>>> seem like this feedback was incorporated into the @requires that >>>>> was eventually built. >>>> >>>> We tried to implement as much developer's wishes as possible. But >>>> not everything is possible, sorry for that. >>> >>> Yes, I'm sure you have done your best. It's just that we have been >>> requesting this feature for 3 years and I was expecting us to be >>> able to influence the feature much more than was the case now. >> >> My personal hope: @requires will address ~90% of existing issues. >> >>> >>>> >>>>> >>>>> I think this change that gets proposed now is a big step forward >>>>> and I won't object to it. But I am pretty convinced that we will >>>>> soon run in to the limitations of the current @requires >>>>> implementation and we will have to redo this work. >>>>> >>>>> Some of the points I don't really like about the @requires tag are: >>>>> >>>>> - the "vm.gc" abstraction is more limiting than helping. It would >>>>> have been better to just "require" any command line flag. >>>> "vm.gc" is an alias to a very popular flag. It's also possible to use: >>>> vm.opt.UseG1GC == true instead. >>>> >>>> The table with all vars available in jtreg: >>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>> >>> The problem with having this matching built in to JTreg is that it >>> makes it very hard to change. When we discussed this a year ago I >>> think we said that JTreg should only provide a means to test against >>> the command line and a hook for running some java code in the >>> @requires tag. That way we could put logic like this in a test >>> library that is under our control. This would make it easy for us to >>> change and also enables us to use different logic for different >>> versions. >> >> I would be glad to have own harness... >> >>> >>>> >>>>> - the requirement should be per @run tag. Right now we have to do >>>>> what you did in this change and use vm.gc=null even when some >>>>> tests could actually have been run when a GC was specified. >>>> it would be great, but it will unlikely happen in jtreg, as well as >>>> test case support. >>> >>> what do you mean with test case support? Hi Evgeniya, >> >> Under test case support I mean ability to treat each @run as a >> separate test. Now >> >> @test >> @run -XX:g1RegSize=1m MyTest >> @run -XX:g1RegSize=2m MyTest >> @run -XX:g1RegSize=4m MyTest >> class MyTest { >> } >> >> is always a single test. You can't exclude, or re-run a part of it. >> >> >>> >>>> >>>>> - there are many tests that require more than just a specific GC. >>>>> Often there are other flags that can't be changed either for the >>>>> test to work properly. >>>> >>>> yes. conflicting GC is just the most popular problem caused by >>>> conflicting options. >>>> If we address this issue and we are satisfied with solution, we >>>> could move further. >>> >>> Yes, I agree that taking one step at the time is good. Personally I >>> would have preferred that the first step was a "just run the command >>> line as specified in the @run tag" step. >>> >>>> >>>>> >>>>> Maybe this is not the right place to discuss the current >>>>> implementation of the @requires tag. I just want to say that I'm >>>>> not too happy about how the @requires tag turned out. But assuming >>>>> we have to use it the way it is now I guess the proposed changeset >>>>> looks good. >>>> >>>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>>> And thanks for reviewing it! >>> >>> Agreed. And as I said, I think the patch looks ok. I have not looked >>> at all tests. But if they now pass with the combinations that we >>> test with I guess they should be ok. >> >> Excellent! Thanks a lot! >> >>> >>>> >>>>> >>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>> without any GC flag). Tests are being excluded as expected. No >>>>>> tests failed because of the conflict. >>>>> Have you tested with -Xconcgc too? It's an alias for >>>>> -XX:+UseConcMarkSweepGC. >>>> >>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>> >>> Ok. Thanks. >>> >>>> >>>>> >>>>> I think some of the test, like >>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >>>>> with -XX:+UseParNewGC. Others, like >>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >>>>> with -XX:-UseParNewGC. Could you test these two cases too? >>>> >>>> These two tests ignore vm flags. >>>> Add @requires here is not necessary, but it will allow not execute >>>> the tests when not needed. >>>> So, if we run HS tests with 4 GC, we don't need to run these tests >>>> 4 times, 1 should be enough. >>> >>> Do we really want to use the @requires functionality for this >>> purpose? It seems like a way of misusing @requires. If we just want >>> the tests to be run once I think Leonid's approach with tests lists >>> seems more suitable. >> >> No, it's not a purpose of course, it's just side effect :) >> >> >>> But are you sure that this is the reason for the @requires in this >>> case? TestDefNewCMS does sound like a test that is DefNew specific. >>> I don't see a reason to run it with ParNew. If it doesn't fail today >>> it should probably be changed so that it does fail if it is run with >>> the wrong GC. >> >> @requires - is not the silver bullet, but it's quite easy way to >> solve a lot of issues. >> >> I hope, @requires will allow to reduce the number of "selfish" tests, >> which produce a new java process to ignore vm flags coming from >> outside. No @requires, no other mechanism could 100% protect a test >> from running with conflicting options, but this is not the goal. >> >> If one runs tests with an exotic option, like a new G2 collector, >> there shouldn't mass failures caused by options conflicts. But a few >> failures could be handled manually. >> >> >>> >>>> >>>>> Similarly it looks to me like there are tests that will fail if >>>>> you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>> >>>>> Just a heads up. These two tests will soon be removed. I'm about >>>>> to push a changeset that removes them: >>>>> >>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>> okay, thank for letting us know. >>>> >>>>> >>>>> Is there some way of making sure that all tests are run at one >>>>> time or another. With this change there is a risk that some tests >>>>> are never run and always skipped. Will we somehow be tracking what >>>>> gets skipped and make sure that all tests are at least run once >>>>> with the correct GC so that it is not skipped all the time? >>>> >>>> This is a very good question! >>>> jtreg now doesn't report skipped tests, hopefully it will do soon, >>>> after getting fix of: >>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>> >>>> And yes, tracking tests which are not run is important thing. >>>> @requires - is not the only to exclude test from execution. >>>> >>>> Other examples: >>>> >>>> /* >>>> *@ignore >>>> *@test >>>> */ >>>> ... >>>> >>>> /*@bug 4445555 >>>> *@test >>>> */ >>>> ... >>>> Such tests will never be run, because jtreg treats as test only >>>> files with @test on the first place... >>>> >>>> So, making sure that tests do not disappear is important SQE task, >>>> we know about that, we're thinking on solution (may be very >>>> actively). But this subject for another discussion, not within RFR :) >>> >>> Right. Glad to hear that you are actively working on this! >> >> I was going to say "not very actively", but never mind, we know about >> this problem. With introducing @requires mechanism it will become >> more important! >> >> >> Thanks for your comments! >> >> -- Dima >> >> >>> >>> Bengt >>> >>>> >>>> Thanks, >>>> Dima >>>> >>>> >>>> >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zoltan.majo at oracle.com Tue Nov 4 13:04:18 2014 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Tue, 04 Nov 2014 14:04:18 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <5458908E.1050909@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> Message-ID: <5458CED2.6030203@oracle.com> Hi Roland, Vladimir, and Albert! thank you for your feedback and for your suggestions! Please see comments on specific issues inline. On 11/04/2014 09:38 AM, Albert Noll wrote: > Hi, > > please see comments inline. > On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >> I agree with additional check tak != TypeKlassPtr::OBJECT (solution 2). >> >> Instead of creating new LoadKlassNode::make() method you can set >> control in parseHelper.cpp since it is the only place which set it: >> >> Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), >> p2, tak); >> if (always_see_exact_class) { >> a_e_klass->init_req(MemNode::Control, control()); >> } >> a_e_klass = _gvn.transform(a_e_klass); >> > An alternative solution is to make a default parameter for control. > This way we can set the control in make() without having to add a new > make() function. I took Albert's alternative solution. Now we also set the default value for parameter TypeKlassPtr* tk in parseHelper.cpp at line 230. >> Also an other reason to have control edge in this place is the >> address of this klass load is based on ConP constant. In a normal >> case after class check you should have CheckCastPP attached to >> control and used as address's base. I see, Vladimir. Just a related question: If the address of the klass load were *not* based on a ConP constant, would it be better (or required) to check with a CheckCastPP the value that was loaded? >> >> And agree with Roland's suggestion about Op_LoadKlass check. >> > What you are suggesting is to make a super class (LoadNode) aware of > its subclasses (LoadPNode and LoadKlassNode). From a software > engineering point of view, we should try to avoid doing that. Another > solution would be to add a method (e.g., can_remove_control()) to > LoadNode, LoadPNode, and LoadKlassNode. The function returns 'false' > for LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What > do you think? I think Albert's suggestion has the advantage that if we will ever decide to inherit from LoadKlass, we won't have to worry about modifying its superclass. So I think it will be good if we go with it. Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.02/ JPRT and also the failing test cases pass. Thank you and best regards, Zoltan > > Best, > Albert > >> Thanks, >> Vladimir >> >> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>> Hi, >>> >>> >>> please review the following patch. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>> >>> >>> Problem: >>> >>> >>> We have five tests that fail with SIGSEGV in our nightlies: >>> >>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>> >>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>> >>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>> >>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>> >>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>> >>> All tests fail when executing the C2-compiled version of >>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>> >>> >>> 489: private void inflateSpine() { >>> 490: if (spine == null) { >>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>> 493: spine[0] = curChunk; >>> 494: } >>> 495: } >>> >>> >>> The failure is due to the 'aastore' at line 493; the failure is caused >>> by the monomorphic array check optimization >>> (-XX:+MonomorphicArrayCheck, enabled by default). >>> >>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>> profiling information) that inflateSpine() has two hot receiver types, >>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>> >>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>> >>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>> double[][]. >>> >>> Please consider the following pseudo Java code that illustrates how C2 >>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on >>> the two hot receiver types SpinedBuffer.OfInt and >>> SpinedBuffer.OfDouble: >>> >>> >>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>> spine[0] = curChunk; >>> } >>> >>> >>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>> of 'spine' *at runtime*. The check looks something like: >>> >>> >>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>> throw new ArrayStoreException(); >>> } >>> >>> >>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>> check like this: >>> >>> >>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>> throw new ArrayStoreException(); >>> } >>> >>> >>> where TYPE is the type of the 'spine' variable, as it is determined by >>> C2 *at compile time*. The optimization treats TYPE as a constant and >>> thus saves a load from memory + an offset calculation (I think). >>> >>> The problem is that due to 'spine' being either of type int[][] or of >>> type double[][], C2 determines that TYPE==java/lang/Object. >>> >>> As a result, when the inflateSpine() method is executed, it first >>> loads the >>> Klass* corresponding to java/lang/Object (TYPE). Let us call this >>> Klass* >>> array_klass. Then, the method obtains a Klass* from >>> array_klass->_element_klass. >>> Then, the method reads from offset 20 from >>> array_klass->_element_klass, which >>> results in a SIGSEGV. >>> >>> The reason for the SIGSEGV is that the Klass* array_klass is of type >>> java/lang/Object and is therefore represented as an 'InstanceKlass'. >>> But _element_klass is defined only in objects of type 'ObjArrayKlass'. >>> The compiler reads array_klass->_element_klass because it expects the >>> destination of an 'aastore' to be an array, which is not true if >>> TYPE==java/lang/Object. >>> >>> >>> Solution: >>> >>> >>> The compiler already checks if the compile-time type of the array >>> (TYPE) is >>> the same as the array's type at runtime. If that check fails, the >>> method >>> branches to an uncommon trap. In our case the SIGSEGV happens because >>> the load of array_klass->_element_klass "floats" above the check. >>> >>> I propose two solutions: >>> >>> Solution 1. Add a control edge between the IfTrue branch of the >>> check and >>> the load of _array_klass->_element_klass. That prohibits the load of >>> _array_klass->element_klass floating above the check. >>> >>> Solution 2. In addition to the changes in Solution 1, the compiler >>> checks if >>> the compile-time type of the array (TYPE) is java/lang/Object. If >>> TYPE==java/lang/Object, the compiler should not assume array_type to be >>> TYPE, but should determine array type at runtime instead. >>> >>> The reason is for Solution 2 is: We can't do an array store into a >>> java/lang/Object. >>> So, if we see TYPE==java/lang/Object at compile time, it is pretty >>> sure that the >>> method will deoptimize at runtime. Instead of assuming TYPE to be >>> java/lang/Object, >>> we read at runtime the type of the array store's destination. That >>> still gives us some >>> chance that the compiled version of the method will successfully >>> execute. The >>> additional check is in parseHelper.cpp in Parse::array_store_check() >>> on line 171. >>> >>> >>> Webrev: >>> >>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>> >>> >>> Testing (for both solutions): >>> >>> JPRT, manual testing with failing test cases. >>> >>> >>> Please note that both solutions are somewhat "hacky", because that >>> allowed me to >>> produce a solution within reasonable time and with reasonable >>> effort. Any suggestions >>> or feedback is welcome. >>> >>> >>> Thank you and best regards, >>> >>> >>> Zoltan >>> > From peter.levart at gmail.com Mon Nov 3 20:09:29 2014 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 03 Nov 2014 21:09:29 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> Message-ID: <5457E0F9.8090004@gmail.com> Hi David, I was thinking about the fact that java.lang.invoke code is leaking into java.lang.Class. Perhaps, If you don't mind rewriting the code, a better code structure would be, if j.l.Class changes only consisted of adding a simple: + // A reference to canonicalizing cache of java.lang.invoke.MemberName(s) + // for members declared by class represented by this Class object + private transient volatile Object memberNameData; ...and nothing else. All the logic could live in MemberName itself (together with Unsafe machinery for accessing/cas-ing Class.memberNameData). Now to an idea about implementation. Since VM code is not doing any binary-search and only linearly scans the array when it has to update MemberNames, the code could be changed to scan a linked-list of MemberName(s) instead. You could add a field to MemberName: class MemberName { ... // next MemberName in chain of interned MemberNames for particular declaring class private MemberName next; Have a volatile field in MemberNameData (or ClassData - whatever you call it): class MemberNameData { ... // a chain of interned MemberName(s) for particular declaring class // accessed by VM when it has to modify them in-place private volatile MemberName memberNames; MemberName add(Class klass, int index, MemberName mn, int redefined_count) { mn.next = memberNames; memberNames = mn; if (jla.getClassRedefinedCount(klass) == redefined_count) { // no changes to class ... ... code to update array of sorted MemberName(s) with new 'mn' ... return mn; } // lost race, undo insertion memberNames = mn.next; return null; } This way all the worries about ordering of writes into array and/or size are gone. The array is still used to quickly search for an element, but VM only scans the linked-list. What do you think of this? Regards, Peter On 11/03/2014 05:36 PM, David Chase wrote: >>> My main concern is that the compiler is inhibited from any peculiar code motion; I assume that taking a safe point has a bit of barrier built into it anyway, especially given that the worry case is safepoint + JVMTI. >>> >>> Given the worry, what?s the best way to spell ?barrier? here? >>> I could synchronize on classData (it would be a recursive lock in the current version of the code) >>> synchronized (this) { size++; } >>> or I could synchronize on elementData (no longer used for a lock elsewhere, so always uncontended) >>> synchronized (elementData) { size++; } >>> or is there some Unsafe thing that would be better? >> You're worried that writes moving array elements up for one slot would bubble up before write of size = size+1, right? If that happens, VM could skip an existing (last) element and not update it. > exactly, with the restriction that it would be compiler-induced bubbling, not architectural. > Which is both better, and worse ? I don?t have to worry about crazy hardware, but the rules > of java/jvm "memory model" are not as thoroughly defined as those for java itself. > > I added a method to Atomic (.storeFence() ). New webrev to come after I rebuild and retest. > > Thanks much, > > David > >> It seems that Unsafe.storeFence() between size++ and moving of elements could do, as the javadoc for it says: >> >> /** >> * Ensures lack of reordering of stores before the fence >> * with loads or stores after the fence. >> * @since 1.8 >> */ >> public native void storeFence(); -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.levart at gmail.com Tue Nov 4 10:07:56 2014 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 04 Nov 2014 11:07:56 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> Message-ID: <5458A57C.4060208@gmail.com> On 11/03/2014 09:41 PM, David Chase wrote: > On 2014-11-03, at 3:09 PM, Peter Levart wrote: > >> Hi David, >> >> I was thinking about the fact that java.lang.invoke code is leaking into java.lang.Class. Perhaps, If you don't mind rewriting the code, a better code structure would be, if j.l.Class changes only consisted of adding a simple: >> ? >> This way all the worries about ordering of writes into array and/or size are gone. The array is still used to quickly search for an element, but VM only scans the linked-list. >> >> What do you think of this? > I?m not sure. I know Coleen Ph would like to see that happen. > > A couple of people have vague plans to move more of the MemberName resolution into core libs. > (Years ago I worked on a VM where *all* of this occurred in Java, but some of it was ahead of time.) Hi David, > > I heard mention of ?we want to put more stuff in there? but I got the impression that already happened > (there?s reflection data, for example) so I?m not sure that makes sense. Reflection is an API that is rooted in j.l.Class. If the plans are to move some of the java.lang.invoke public API to java.lang package (into the j.l.Class, ...), then this is understandable. > > There?s also a proposal from people in the runtime to just use a jmethodid, take the hit of an extra indirection, > and no need to for this worrisome jvm/java concurrency. The linked list of MemberName(s) is also worry-less and doesn't need an extra indirection via jmethodid. Does the hit of extra indirection occur when invoking a MethodHandle? > > And if we instead wrote a hash table that only grew, and never relocated elements, we could > (I think) allow non-synchronized O(1) probes of the table from the Java side, synchronized > O(1) insertions from the Java side, and because nothing moves, a smaller dance with the > VM. I?m rather tempted to look into this ? given the amount of work it would take to do the > benchmarking to see if (a) jmethodid would have acceptable performance or (b) the existing > costs are too high, I could instead just write fast code and be done. Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? > > And another way to view this is that we?re now quibbling about performance, when we still > have an existing correctness problem that this patch solves, so maybe we should just get this > done and then file an RFE. Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. Regards, Peter > > David > From david.r.chase at oracle.com Tue Nov 4 15:19:24 2014 From: david.r.chase at oracle.com (David Chase) Date: Tue, 4 Nov 2014 10:19:24 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5458A57C.4060208@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> Message-ID: <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> On 2014-11-04, at 5:07 AM, Peter Levart wrote: > Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? It can?t be an identityHashMap, because we are interning member names. In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. One possibility would be to use two data structures, one for interning, the other for communication with the VM. Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, and the synchronization dance is much simpler. For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: mn = resolve(args) // deal with any errors mn? = chm.get(mn) if (mn? != null) return mn? // hoped-for-common-case synchronized (something) { mn? = chm.get(mn) if (mn? != null) return mn? txn_class = mn.getDeclaringClass() while (true) { redef_count = txn_class.redefCount() mn = resolve(args) shared_array.add(mn); // barrier, because we are a paranoid if (redef_count = redef_count.redefCount()) { chm.add(mn); // safe to publish to other Java threads. return mn; } shared_array.drop_last(); // Try again } } (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). David >> >> And another way to view this is that we?re now quibbling about performance, when we still >> have an existing correctness problem that this patch solves, so maybe we should just get this >> done and then file an RFE. > > Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. > > Regards, Peter > >> >> David -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From peter.levart at gmail.com Tue Nov 4 16:48:14 2014 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 04 Nov 2014 17:48:14 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> Message-ID: <5459034E.8070809@gmail.com> On 11/04/2014 04:19 PM, David Chase wrote: > On 2014-11-04, at 5:07 AM, Peter Levart wrote: >> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? > It can?t be an identityHashMap, because we are interning member names. I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. > In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. > One possibility would be to use two data structures, one for interning, the other for communication with the VM. > Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, > and the synchronization dance is much simpler. > > For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: > > mn = resolve(args) > // deal with any errors > mn? = chm.get(mn) > if (mn? != null) return mn? // hoped-for-common-case > > synchronized (something) { > mn? = chm.get(mn) > if (mn? != null) return mn? > > txn_class = mn.getDeclaringClass() > > while (true) { > redef_count = txn_class.redefCount() > mn = resolve(args) > > shared_array.add(mn); > // barrier, because we are a paranoid > if (redef_count = redef_count.redefCount()) { > chm.add(mn); // safe to publish to other Java threads. > return mn; > } > shared_array.drop_last(); // Try again > } > } > > (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. A combination of the two approaches is also possible: - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) - use ConcurrentHashMap for interning. Regards, Peter > > David > >>> And another way to view this is that we?re now quibbling about performance, when we still >>> have an existing correctness problem that this patch solves, so maybe we should just get this >>> done and then file an RFE. >> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >> >> Regards, Peter >> >>> David From vladimir.kozlov at oracle.com Tue Nov 4 18:30:03 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 04 Nov 2014 10:30:03 -0800 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <5458908E.1050909@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> Message-ID: <54591B2B.80603@oracle.com> On 11/4/14 12:38 AM, Albert Noll wrote: > Hi, > > please see comments inline. > On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >> I agree with additional check tak != TypeKlassPtr::OBJECT (solution 2). >> >> Instead of creating new LoadKlassNode::make() method you can set control in parseHelper.cpp since it is the only place >> which set it: >> >> Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), p2, tak); >> if (always_see_exact_class) { >> a_e_klass->init_req(MemNode::Control, control()); >> } >> a_e_klass = _gvn.transform(a_e_klass); >> > An alternative solution is to make a default parameter for control. This way we can set the control in make() without > having to add a new make() function. My objection was against having 2 make() methods. I agree to have control parameter for make() but think it should be explicit so we know where we pass NULL and I don't what it to be last parameter. > >> Also an other reason to have control edge in this place is the address of this klass load is based on ConP constant. >> In a normal case after class check you should have CheckCastPP attached to control and used as address's base. >> >> And agree with Roland's suggestion about Op_LoadKlass check. >> > What you are suggesting is to make a super class (LoadNode) aware of its subclasses (LoadPNode and LoadKlassNode). From > a software engineering point of view, we should try to avoid doing that. Another solution would be to add a method > (e.g., can_remove_control()) to LoadNode, LoadPNode, and LoadKlassNode. The function returns 'false' for LoadPNode and > LoadKlassNode; and returns 'true' for LoadNode. What do you think? To have can_remove_control() is good idea. It could be used in other cases too when we need - it could check some conditions instead of simple false/true. Thanks, Vladimir > > Best, > Albert > >> Thanks, >> Vladimir >> >> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>> Hi, >>> >>> >>> please review the following patch. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>> >>> >>> Problem: >>> >>> >>> We have five tests that fail with SIGSEGV in our nightlies: >>> >>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>> >>> All tests fail when executing the C2-compiled version of >>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>> >>> >>> 489: private void inflateSpine() { >>> 490: if (spine == null) { >>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>> 493: spine[0] = curChunk; >>> 494: } >>> 495: } >>> >>> >>> The failure is due to the 'aastore' at line 493; the failure is caused >>> by the monomorphic array check optimization >>> (-XX:+MonomorphicArrayCheck, enabled by default). >>> >>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>> profiling information) that inflateSpine() has two hot receiver types, >>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>> >>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>> >>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>> double[][]. >>> >>> Please consider the following pseudo Java code that illustrates how C2 >>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on >>> the two hot receiver types SpinedBuffer.OfInt and >>> SpinedBuffer.OfDouble: >>> >>> >>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>> spine[0] = curChunk; >>> } >>> >>> >>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>> of 'spine' *at runtime*. The check looks something like: >>> >>> >>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>> throw new ArrayStoreException(); >>> } >>> >>> >>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>> check like this: >>> >>> >>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>> throw new ArrayStoreException(); >>> } >>> >>> >>> where TYPE is the type of the 'spine' variable, as it is determined by >>> C2 *at compile time*. The optimization treats TYPE as a constant and >>> thus saves a load from memory + an offset calculation (I think). >>> >>> The problem is that due to 'spine' being either of type int[][] or of >>> type double[][], C2 determines that TYPE==java/lang/Object. >>> >>> As a result, when the inflateSpine() method is executed, it first loads the >>> Klass* corresponding to java/lang/Object (TYPE). Let us call this Klass* >>> array_klass. Then, the method obtains a Klass* from array_klass->_element_klass. >>> Then, the method reads from offset 20 from array_klass->_element_klass, which >>> results in a SIGSEGV. >>> >>> The reason for the SIGSEGV is that the Klass* array_klass is of type >>> java/lang/Object and is therefore represented as an 'InstanceKlass'. >>> But _element_klass is defined only in objects of type 'ObjArrayKlass'. >>> The compiler reads array_klass->_element_klass because it expects the >>> destination of an 'aastore' to be an array, which is not true if >>> TYPE==java/lang/Object. >>> >>> >>> Solution: >>> >>> >>> The compiler already checks if the compile-time type of the array (TYPE) is >>> the same as the array's type at runtime. If that check fails, the method >>> branches to an uncommon trap. In our case the SIGSEGV happens because >>> the load of array_klass->_element_klass "floats" above the check. >>> >>> I propose two solutions: >>> >>> Solution 1. Add a control edge between the IfTrue branch of the check and >>> the load of _array_klass->_element_klass. That prohibits the load of >>> _array_klass->element_klass floating above the check. >>> >>> Solution 2. In addition to the changes in Solution 1, the compiler checks if >>> the compile-time type of the array (TYPE) is java/lang/Object. If >>> TYPE==java/lang/Object, the compiler should not assume array_type to be >>> TYPE, but should determine array type at runtime instead. >>> >>> The reason is for Solution 2 is: We can't do an array store into a java/lang/Object. >>> So, if we see TYPE==java/lang/Object at compile time, it is pretty sure that the >>> method will deoptimize at runtime. Instead of assuming TYPE to be java/lang/Object, >>> we read at runtime the type of the array store's destination. That still gives us some >>> chance that the compiled version of the method will successfully execute. The >>> additional check is in parseHelper.cpp in Parse::array_store_check() on line 171. >>> >>> >>> Webrev: >>> >>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>> >>> >>> Testing (for both solutions): >>> >>> JPRT, manual testing with failing test cases. >>> >>> >>> Please note that both solutions are somewhat "hacky", because that allowed me to >>> produce a solution within reasonable time and with reasonable effort. Any suggestions >>> or feedback is welcome. >>> >>> >>> Thank you and best regards, >>> >>> >>> Zoltan >>> > From vladimir.kozlov at oracle.com Tue Nov 4 18:40:04 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 04 Nov 2014 10:40:04 -0800 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <5458CED2.6030203@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> <5458CED2.6030203@oracle.com> Message-ID: <54591D84.30709@oracle.com> On 11/4/14 5:04 AM, Zolt?n Maj? wrote: > Hi Roland, Vladimir, and Albert! > > > thank you for your feedback and for your suggestions! Please see comments on specific issues inline. > > On 11/04/2014 09:38 AM, Albert Noll wrote: >> Hi, >> >> please see comments inline. >> On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >>> I agree with additional check tak != TypeKlassPtr::OBJECT (solution 2). >>> >>> Instead of creating new LoadKlassNode::make() method you can set control in parseHelper.cpp since it is the only >>> place which set it: >>> >>> Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), p2, tak); >>> if (always_see_exact_class) { >>> a_e_klass->init_req(MemNode::Control, control()); >>> } >>> a_e_klass = _gvn.transform(a_e_klass); >>> >> An alternative solution is to make a default parameter for control. This way we can set the control in make() without >> having to add a new make() function. > > I took Albert's alternative solution. Now we also set the default value for parameter TypeKlassPtr* tk in > parseHelper.cpp at line 230. As I replied to Albert, I think it should be explicit parameter and not default. Yes, you would need to modify more files but it is better to be explicit I think. > >>> Also an other reason to have control edge in this place is the address of this klass load is based on ConP constant. >>> In a normal case after class check you should have CheckCastPP attached to control and used as address's base. > > I see, Vladimir. Just a related question: If the address of the klass load were *not* based on a ConP constant, would it > be better (or required) to check with a CheckCastPP the value that was loaded? Yes, you would use CheckCastPP to cast type to exact class you checked for and pin it to 'true' projection of the check. As we do for in gen_checkcast() or other places (speculative types). if (a.class == exact_class) (exact_class)a In this case you don't need control on following load. > >>> >>> And agree with Roland's suggestion about Op_LoadKlass check. >>> >> What you are suggesting is to make a super class (LoadNode) aware of its subclasses (LoadPNode and LoadKlassNode). >> From a software engineering point of view, we should try to avoid doing that. Another solution would be to add a >> method (e.g., can_remove_control()) to LoadNode, LoadPNode, and LoadKlassNode. The function returns 'false' for >> LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What do you think? > > I think Albert's suggestion has the advantage that if we will ever decide to inherit from LoadKlass, we won't have to > worry about modifying its superclass. So I think it will be good if we go with it. I prefer Albert's name can_remove_control() Thanks, Vladimir > > Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.02/ > > JPRT and also the failing test cases pass. > > > Thank you and best regards, > > > Zoltan > >> >> Best, >> Albert >> >>> Thanks, >>> Vladimir >>> >>> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>>> Hi, >>>> >>>> >>>> please review the following patch. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>>> >>>> >>>> Problem: >>>> >>>> >>>> We have five tests that fail with SIGSEGV in our nightlies: >>>> >>>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>>> >>>> All tests fail when executing the C2-compiled version of >>>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>>> >>>> >>>> 489: private void inflateSpine() { >>>> 490: if (spine == null) { >>>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>>> 493: spine[0] = curChunk; >>>> 494: } >>>> 495: } >>>> >>>> >>>> The failure is due to the 'aastore' at line 493; the failure is caused >>>> by the monomorphic array check optimization >>>> (-XX:+MonomorphicArrayCheck, enabled by default). >>>> >>>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>>> profiling information) that inflateSpine() has two hot receiver types, >>>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>>> >>>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>>> >>>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>>> double[][]. >>>> >>>> Please consider the following pseudo Java code that illustrates how C2 >>>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on >>>> the two hot receiver types SpinedBuffer.OfInt and >>>> SpinedBuffer.OfDouble: >>>> >>>> >>>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>>> spine[0] = curChunk; >>>> } >>>> >>>> >>>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>>> of 'spine' *at runtime*. The check looks something like: >>>> >>>> >>>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>>> throw new ArrayStoreException(); >>>> } >>>> >>>> >>>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>>> check like this: >>>> >>>> >>>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>>> throw new ArrayStoreException(); >>>> } >>>> >>>> >>>> where TYPE is the type of the 'spine' variable, as it is determined by >>>> C2 *at compile time*. The optimization treats TYPE as a constant and >>>> thus saves a load from memory + an offset calculation (I think). >>>> >>>> The problem is that due to 'spine' being either of type int[][] or of >>>> type double[][], C2 determines that TYPE==java/lang/Object. >>>> >>>> As a result, when the inflateSpine() method is executed, it first loads the >>>> Klass* corresponding to java/lang/Object (TYPE). Let us call this Klass* >>>> array_klass. Then, the method obtains a Klass* from array_klass->_element_klass. >>>> Then, the method reads from offset 20 from array_klass->_element_klass, which >>>> results in a SIGSEGV. >>>> >>>> The reason for the SIGSEGV is that the Klass* array_klass is of type >>>> java/lang/Object and is therefore represented as an 'InstanceKlass'. >>>> But _element_klass is defined only in objects of type 'ObjArrayKlass'. >>>> The compiler reads array_klass->_element_klass because it expects the >>>> destination of an 'aastore' to be an array, which is not true if >>>> TYPE==java/lang/Object. >>>> >>>> >>>> Solution: >>>> >>>> >>>> The compiler already checks if the compile-time type of the array (TYPE) is >>>> the same as the array's type at runtime. If that check fails, the method >>>> branches to an uncommon trap. In our case the SIGSEGV happens because >>>> the load of array_klass->_element_klass "floats" above the check. >>>> >>>> I propose two solutions: >>>> >>>> Solution 1. Add a control edge between the IfTrue branch of the check and >>>> the load of _array_klass->_element_klass. That prohibits the load of >>>> _array_klass->element_klass floating above the check. >>>> >>>> Solution 2. In addition to the changes in Solution 1, the compiler checks if >>>> the compile-time type of the array (TYPE) is java/lang/Object. If >>>> TYPE==java/lang/Object, the compiler should not assume array_type to be >>>> TYPE, but should determine array type at runtime instead. >>>> >>>> The reason is for Solution 2 is: We can't do an array store into a java/lang/Object. >>>> So, if we see TYPE==java/lang/Object at compile time, it is pretty sure that the >>>> method will deoptimize at runtime. Instead of assuming TYPE to be java/lang/Object, >>>> we read at runtime the type of the array store's destination. That still gives us some >>>> chance that the compiled version of the method will successfully execute. The >>>> additional check is in parseHelper.cpp in Parse::array_store_check() on line 171. >>>> >>>> >>>> Webrev: >>>> >>>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>>> >>>> >>>> Testing (for both solutions): >>>> >>>> JPRT, manual testing with failing test cases. >>>> >>>> >>>> Please note that both solutions are somewhat "hacky", because that allowed me to >>>> produce a solution within reasonable time and with reasonable effort. Any suggestions >>>> or feedback is welcome. >>>> >>>> >>>> Thank you and best regards, >>>> >>>> >>>> Zoltan >>>> >> > From roland.westrelin at oracle.com Tue Nov 4 20:03:25 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Tue, 4 Nov 2014 21:03:25 +0100 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <54581D5E.2090706@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> Message-ID: > Sorry, it took so much time. I was confused by number of files you changed. In general it seems good to me. Thanks Vladimir for looking at this. > Do you know why GCM scheduled the load into previous block? Usually loads are placed near their uses in: > > 0c7 B13: # B17 B14 <- B11 Freq: 0.768468 > 0c7 movzwl R8, [RCX + #24 + R10 << #1] # ushort/char << SEGV > 0cd addl RDI, #-2 # int > 0d0 movl R9, #-2 # int > 0d6 testl R10, R10 > 0d9 jle,s B17 P=0.000001 C=-1.000000 > 0d9 > 0db B14: # B21 B15 <- B13 Freq: 0.768467 > 0db testl R8, R8 > 0de je,s B21 P=0.100000 C=-1.000000 There are 2 uses for the value that is loaded. I don?t have the same block numbering/register allocation when I run it so it looks like this for me: 04f B2: # B23 B3 <- B15 Freq: 5848.51 04f movzwl R10, [RDX + #16 + R8 << #1] # ushort/char 055 addl RCX, #-2 # int 058 testl R8, R8 05b jle B23 P=0.000001 C=-1.000000 05b 061 B3: # B22 B4 <- B2 Freq: 5848.5 061 testl R10, R10 064 je B22 P=0.000001 C=-1.000000 ... 149 B23: # B5 B24 <- B2 Freq: 0.00592617 149 cmpl R8, #-1 14d jle B5 P=0.100000 C=-1.000000 14d 153 B24: # B22 B25 <- B23 Freq: 0.00533355 153 testl R10, R10 156 je,s B22 P=0.000001 C=-1.000000 B2 (in my case) is the LCA of both uses (B3 & B24). > castnode.hpp: > > Fields names: _required --> _carry_dependency. Better naming indeed. > I am not convinced that you need _inexact. Based on your code _inexact can be converted from 'true' to 'false' only when phase->type(cmp->in(2))->is_int()->is_con() and all conditions at the beginning of CastIINode::Ideal() are true. But if input's type is narrowed later we can still improve CastII's type. What do you mean by later? That the input becomes a constant and then the constant changes? > Do you think we will get worse code if we don't do Identity for _carry_dependency CastII nodes? If _carry_dependency is set we can't replace it with an other node, I think, (unless all inputs are matching - hash() and cmp()). To be safe I should define hash() and cmp() for CastIINode to account for _carry_dependency (and _inexact if we keep it), right? Do I think we should have Identity skip _carry_dependency CastII nodes entirely? I think we might miss some optimization opportunities if we do. For instance something like this maybe: for (int i = j; i < 100; i++) { value += some_array[i]; // some test that make the backbranch dead in the main loop } value += some_array[j+1]; Assuming a 1 iteration pre loop, the load after the loop should common with the load in the main loop but wouldn?t with the CastII because they wouldn't have the same address input. Anyway I couldn?t make that in an actual test case. Also, I think we might use the modified CastII elsewhere in the future. Currently, we only add CastII during parsing when we know we compare against a constant. The modified CastII could be added even if we don?t compare against a constant and if the constant is discovered later, the CastII would help optimization. If we sprinkle CastII nodes in the graph, we would want the useless ones to go away. > castnode.cpp > > The type change belongs to Value() not Ideal(). Especially if you don't need to set _inexact field and create new CastII node for that. That makes sense. > Use fatal with message which includes BoolTest::mask name instead of ShouldNotReachHere(). Ok. > loopTransform.cpp > > insert_castii_before_loop() --> cast_incr_before_loop() > You missed registration of new CastII nodes - register_new_node(). Ok. Thanks for spotting that. > opaquenode.cpp > > What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. > There is loop_limit_check predicate before a counted loop which has opaqued limit. It . Those opaque nodes are removed by cleanup_loop_predicates(). Why Phi is not put on worklist at that time? So that code: Node *Opaque1Node::Identity(PhaseTransform *phase) { return phase->C->major_progress() ? this : in(1); } is not used to remove the opaque nodes? I thought I observed they were removed there. Roland. > > Thanks, > Vladimir > > On 10/6/14 2:49 AM, Roland Westrelin wrote: >> >> Anyone? >> >> Roland. >> >> On Sep 17, 2014, at 7:03 PM, Roland Westrelin wrote: >> >>> http://cr.openjdk.java.net/~roland/8054478/webrev.00/ >>> >>> The IR after parsing is this in pseudo code: >>> >>> i_main = 0; >>> while(true) { >>> if (i_main >= 1000000) return; >>> i_main++; >>> char[] array = new char[1]; >>> if (pattern1 != null) { >>> i = 0; >>> while(true) { >>> if (i >= 0) { >>> do { >>> if (pattern0 == null) uncommon_trap; >>> if (pattern0.length u< i) uncommon_trap; >>> if (pattern0[i] != 0) { >>> if (pattern1.length u< i) uncommon_trap; >>> if (pattern1[i] != 0) { >>> goto out1; >>> } >>> } >>> i?; >>> pos?; >>> if (pos != -1) { >>> goto out2; >>> } >>> } while (i >= 0); >>> goto out1; >>> } >>> out2: >>> c = array[pos]; >>> } >>> } >>> out1: >>> } >>> >>> The do {} while loop is a CountedLoop. The null check & range check are moved out of the loop. Then a pre and a post loops are created and the body is unrolled. The pattern0[i] LoadNode nodes in the unrolled body have a control that is set to the range check before the pre loop. In the unrolled loop, because of the if (pos != -1) test in the first copy of the loop, the compiler finds that in the second copy of the loop body pos is != -1 and so the loop is exited before reaching the end of the unrolled loop. The back branch of the unrolled loop is dead. The compiler optimizes the CountedLoopNode of the unrolled loop out because it doesn?t have a back branch anymore, PhiNodes are removed and the LoadNode for pattern0[i] in the unroll loop body is now independent of the input control of the CountedLoop. Its control is still set to the range check before the pre loop. In the generated code, the pre loop is followed by the pattern1[i] access which is incorrect because it happens before > the if (i >= 0) that dominated the unrolled loop before it was removed. >>> >>> The graph is good as long as the CountedLoop is not removed because the LoadNode depends on a PhiNode of the CountedLoop. But as soon as the CountedLoop is removed and the PhiNode is disconnected, the LoadNode no longer depends on the check that guarded the CountedLoop. >>> >>> Vladimir suggested (in a private conversation) that I use a CastII (control dependent on the if that guards the main loop with as input the input value to the induction variable in the loop) as a way to guarantee correct dependencies. There are 2 problems with this: >>> >>> 1) the CastII nodes are removed after CCP. I added a flag to the CastII nodes so we can mark CastII nodes that can?t be removed and should be left as they are post-CCP. >>> 2) we don?t know the range of values that are possible on the if branch that enters the main loop right away (it can be adjusted as loop opts are applied). We may actually never know it if it depends on non constants. If we use a type for a range that is too wide in the CastII the CastII may be optimized out incorrectly. I added another flag to the CastII to mark nodes that cannot be optimized. In CastII:Ideal I added code that tries to figure out a better type for the CastII so we may be able to optimize the CastII as the compiler optimizes the IR. >>> >>> Another independent issue here is that the main loop is actually never executed because the loop of the test is a single iteration loop so the compiler should be able to optimize out the main loop entirely but it doesn?t. PhiNode::Value() has this code: >>> >>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>> if( l && l->can_be_counted_loop(phase) && >>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>> >>> ... >>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>> return TypeInt::make(lo->_lo,hi->_hi,3); >>> } >>> } >>> } >>> >>> that returns an accurate value for the Phi node of a CountedLoop. In this case we want the pre loop?s Phi to return an accurate type that would then make the guard of the main loop optimize. The pre loop is guarded by a test on an opaque node which is later removed. If PhiNode::Value() is called before the Opaque1 node is removed then, the limit type is int and it cannot return anything accurate. If it?s called after, then limit type is a constant and PhiNode::Value() also returns a constant for the pre-loop. Then the main loop is optimized out. To fix this: I added code to Opaque1Node::Ideal so it reenqueues the Phi when the opaque node is removed. >>> >>> Roland. >> From poonam.bajaj at oracle.com Tue Nov 4 20:09:34 2014 From: poonam.bajaj at oracle.com (Poonam Bajaj) Date: Tue, 04 Nov 2014 12:09:34 -0800 Subject: Review request(7u): JDK-8046275: Fastdebug build failing on jdk9/hs/ control jobs after pulling some hs-comp changes Message-ID: <5459327E.2050409@oracle.com> Hi, Please review these backport changes to 7u-dev: JDK-8046275 : Fastdebug build failing on jdk9/hs/ control jobs after pulling some hs-comp changes webrev: http://cr.openjdk.java.net/~poonam/8046275/webrev.00/ 8u changeset: http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/rev/ad51f24671c2 Testing: JPRT build and testing Thanks, Poonam -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kozlov at oracle.com Tue Nov 4 20:28:58 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 04 Nov 2014 12:28:58 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> Message-ID: <5459370A.1060305@oracle.com> On 11/4/14 12:03 PM, Roland Westrelin wrote: >> Sorry, it took so much time. I was confused by number of files you changed. In general it seems good to me. > > Thanks Vladimir for looking at this. > >> Do you know why GCM scheduled the load into previous block? Usually loads are placed near their uses in: >> >> 0c7 B13: # B17 B14 <- B11 Freq: 0.768468 >> 0c7 movzwl R8, [RCX + #24 + R10 << #1] # ushort/char << SEGV >> 0cd addl RDI, #-2 # int >> 0d0 movl R9, #-2 # int >> 0d6 testl R10, R10 >> 0d9 jle,s B17 P=0.000001 C=-1.000000 >> 0d9 >> 0db B14: # B21 B15 <- B13 Freq: 0.768467 >> 0db testl R8, R8 >> 0de je,s B21 P=0.100000 C=-1.000000 > > There are 2 uses for the value that is loaded. I don?t have the same block numbering/register allocation when I run it so it looks like this for me: Ahh. Second check may coming from post-loop so we merged two loads into one which dominates both. Then this can't be fixed in GCM :( > > 04f B2: # B23 B3 <- B15 Freq: 5848.51 > 04f movzwl R10, [RDX + #16 + R8 << #1] # ushort/char > 055 addl RCX, #-2 # int > 058 testl R8, R8 > 05b jle B23 P=0.000001 C=-1.000000 > 05b > 061 B3: # B22 B4 <- B2 Freq: 5848.5 > 061 testl R10, R10 > 064 je B22 P=0.000001 C=-1.000000 > > ... > > 149 B23: # B5 B24 <- B2 Freq: 0.00592617 > 149 cmpl R8, #-1 > 14d jle B5 P=0.100000 C=-1.000000 > 14d > 153 B24: # B22 B25 <- B23 Freq: 0.00533355 > 153 testl R10, R10 > 156 je,s B22 P=0.000001 C=-1.000000 > > B2 (in my case) is the LCA of both uses (B3 & B24). > >> castnode.hpp: >> >> Fields names: _required --> _carry_dependency. > > Better naming indeed. > >> I am not convinced that you need _inexact. Based on your code _inexact can be converted from 'true' to 'false' only when phase->type(cmp->in(2))->is_int()->is_con() and all conditions at the beginning of CastIINode::Ideal() are true. But if input's type is narrowed later we can still improve CastII's type. > > What do you mean by later? That the input becomes a constant and then the constant changes? If you have, for example, 'i < con' condition the LOW bound type(i)->_lo can improve later so _lo of CastII type can be also improved. > >> Do you think we will get worse code if we don't do Identity for _carry_dependency CastII nodes? If _carry_dependency is set we can't replace it with an other node, I think, (unless all inputs are matching - hash() and cmp()). > > To be safe I should define hash() and cmp() for CastIINode to account for _carry_dependency (and _inexact if we keep it), right? I looked and CastII inherits these methods from TypeNode and Node classes. They only match if all inputs are matching (please, verify this). Which means control input should be matched too - in this case it is safe to use pre-existing node. > > Do I think we should have Identity skip _carry_dependency CastII nodes entirely? I think we might miss some optimization opportunities if we do. For instance something like this maybe: > > for (int i = j; i < 100; i++) { > value += some_array[i]; > // some test that make the backbranch dead in the main loop > } > > value += some_array[j+1]; > > Assuming a 1 iteration pre loop, the load after the loop should common with the load in the main loop but wouldn?t with the CastII because they wouldn't have the same address input. > > Anyway I couldn?t make that in an actual test case. Yes, please create test and see if we can skip Identity. If not, then we may indeed need this additional field or complex checks. > > Also, I think we might use the modified CastII elsewhere in the future. Currently, we only add CastII during parsing when we know we compare against a constant. The modified CastII could be added even if we don?t compare against a constant and if the constant is discovered later, the CastII would help optimization. If we sprinkle CastII nodes in the graph, we would want the useless ones to go away. Please, write tests for such cases and see how it will work. But it should be separate RFE. We need to fix this bug first. > >> castnode.cpp >> >> The type change belongs to Value() not Ideal(). Especially if you don't need to set _inexact field and create new CastII node for that. > > That makes sense. > >> Use fatal with message which includes BoolTest::mask name instead of ShouldNotReachHere(). > > Ok. > >> loopTransform.cpp >> >> insert_castii_before_loop() --> cast_incr_before_loop() >> You missed registration of new CastII nodes - register_new_node(). > > Ok. Thanks for spotting that. > >> opaquenode.cpp >> >> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. > > Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. > >> There is loop_limit_check predicate before a counted loop which has opaqued limit. It . Those opaque nodes are removed by cleanup_loop_predicates(). Why Phi is not put on worklist at that time? > > So that code: > > Node *Opaque1Node::Identity(PhaseTransform *phase) { > return phase->C->major_progress() ? this : in(1); > } > > is not used to remove the opaque nodes? > I thought I observed they were removed there. cleanup_loop_predicates() only removes Opaque1 from predicate checks and not from main- post-loop guards. Vladimir > > Roland. > >> >> Thanks, >> Vladimir >> >> On 10/6/14 2:49 AM, Roland Westrelin wrote: >>> >>> Anyone? >>> >>> Roland. >>> >>> On Sep 17, 2014, at 7:03 PM, Roland Westrelin wrote: >>> >>>> http://cr.openjdk.java.net/~roland/8054478/webrev.00/ >>>> >>>> The IR after parsing is this in pseudo code: >>>> >>>> i_main = 0; >>>> while(true) { >>>> if (i_main >= 1000000) return; >>>> i_main++; >>>> char[] array = new char[1]; >>>> if (pattern1 != null) { >>>> i = 0; >>>> while(true) { >>>> if (i >= 0) { >>>> do { >>>> if (pattern0 == null) uncommon_trap; >>>> if (pattern0.length u< i) uncommon_trap; >>>> if (pattern0[i] != 0) { >>>> if (pattern1.length u< i) uncommon_trap; >>>> if (pattern1[i] != 0) { >>>> goto out1; >>>> } >>>> } >>>> i?; >>>> pos?; >>>> if (pos != -1) { >>>> goto out2; >>>> } >>>> } while (i >= 0); >>>> goto out1; >>>> } >>>> out2: >>>> c = array[pos]; >>>> } >>>> } >>>> out1: >>>> } >>>> >>>> The do {} while loop is a CountedLoop. The null check & range check are moved out of the loop. Then a pre and a post loops are created and the body is unrolled. The pattern0[i] LoadNode nodes in the unrolled body have a control that is set to the range check before the pre loop. In the unrolled loop, because of the if (pos != -1) test in the first copy of the loop, the compiler finds that in the second copy of the loop body pos is != -1 and so the loop is exited before reaching the end of the unrolled loop. The back branch of the unrolled loop is dead. The compiler optimizes the CountedLoopNode of the unrolled loop out because it doesn?t have a back branch anymore, PhiNodes are removed and the LoadNode for pattern0[i] in the unroll loop body is now independent of the input control of the CountedLoop. Its control is still set to the range check before the pre loop. In the generated code, the pre loop is followed by the pattern1[i] access which is incorrect because it happens befor e >> the if (i >= 0) that dominated the unrolled loop before it was removed. >>>> >>>> The graph is good as long as the CountedLoop is not removed because the LoadNode depends on a PhiNode of the CountedLoop. But as soon as the CountedLoop is removed and the PhiNode is disconnected, the LoadNode no longer depends on the check that guarded the CountedLoop. >>>> >>>> Vladimir suggested (in a private conversation) that I use a CastII (control dependent on the if that guards the main loop with as input the input value to the induction variable in the loop) as a way to guarantee correct dependencies. There are 2 problems with this: >>>> >>>> 1) the CastII nodes are removed after CCP. I added a flag to the CastII nodes so we can mark CastII nodes that can?t be removed and should be left as they are post-CCP. >>>> 2) we don?t know the range of values that are possible on the if branch that enters the main loop right away (it can be adjusted as loop opts are applied). We may actually never know it if it depends on non constants. If we use a type for a range that is too wide in the CastII the CastII may be optimized out incorrectly. I added another flag to the CastII to mark nodes that cannot be optimized. In CastII:Ideal I added code that tries to figure out a better type for the CastII so we may be able to optimize the CastII as the compiler optimizes the IR. >>>> >>>> Another independent issue here is that the main loop is actually never executed because the loop of the test is a single iteration loop so the compiler should be able to optimize out the main loop entirely but it doesn?t. PhiNode::Value() has this code: >>>> >>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>> if( l && l->can_be_counted_loop(phase) && >>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>> >>>> ... >>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>> } >>>> } >>>> } >>>> >>>> that returns an accurate value for the Phi node of a CountedLoop. In this case we want the pre loop?s Phi to return an accurate type that would then make the guard of the main loop optimize. The pre loop is guarded by a test on an opaque node which is later removed. If PhiNode::Value() is called before the Opaque1 node is removed then, the limit type is int and it cannot return anything accurate. If it?s called after, then limit type is a constant and PhiNode::Value() also returns a constant for the pre-loop. Then the main loop is optimized out. To fix this: I added code to Opaque1Node::Ideal so it reenqueues the Phi when the opaque node is removed. >>>> >>>> Roland. >>> > From david.r.chase at oracle.com Tue Nov 4 20:54:03 2014 From: david.r.chase at oracle.com (David Chase) Date: Tue, 4 Nov 2014 15:54:03 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <5459034E.8070809@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> Message-ID: I?m working on the initial benchmarking, and so far this arrangement (with synchronization and binary search for lookup, lots of barriers and linear cost insertion) has not yet been any slower. I am nonetheless tempted by the 2-tables solution, because I think the simpler JVM-side interface that it allows is desirable. David On 2014-11-04, at 11:48 AM, Peter Levart wrote: > On 11/04/2014 04:19 PM, David Chase wrote: >> On 2014-11-04, at 5:07 AM, Peter Levart wrote: >>> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? >> It can?t be an identityHashMap, because we are interning member names. > > I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. > >> In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. >> One possibility would be to use two data structures, one for interning, the other for communication with the VM. >> Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, >> and the synchronization dance is much simpler. >> >> For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: >> >> mn = resolve(args) >> // deal with any errors >> mn? = chm.get(mn) >> if (mn? != null) return mn? // hoped-for-common-case >> >> synchronized (something) { >> mn? = chm.get(mn) >> if (mn? != null) return mn? >> txn_class = mn.getDeclaringClass() >> >> while (true) { >> redef_count = txn_class.redefCount() >> mn = resolve(args) >> >> shared_array.add(mn); >> // barrier, because we are a paranoid >> if (redef_count = redef_count.redefCount()) { >> chm.add(mn); // safe to publish to other Java threads. >> return mn; >> } >> shared_array.drop_last(); // Try again >> } >> } >> >> (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). > > Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. > > A combination of the two approaches is also possible: > > - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) > - use ConcurrentHashMap for interning. > > Regards, Peter > >> >> David >> >>>> And another way to view this is that we?re now quibbling about performance, when we still >>>> have an existing correctness problem that this patch solves, so maybe we should just get this >>>> done and then file an RFE. >>> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >>> >>> Regards, Peter >>> >>>> David -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From vladimir.kozlov at oracle.com Tue Nov 4 21:18:37 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 04 Nov 2014 13:18:37 -0800 Subject: Review request(7u): JDK-8046275: Fastdebug build failing on jdk9/hs/ control jobs after pulling some hs-comp changes In-Reply-To: <5459327E.2050409@oracle.com> References: <5459327E.2050409@oracle.com> Message-ID: <545942AD.1020000@oracle.com> Looks good. I think you need to send request for backport approval to jdk7u-dev at openjdk.java.net Thanks, Vladimir On 11/4/14 12:09 PM, Poonam Bajaj wrote: > Hi, > > Please review these backport changes to 7u-dev: > > JDK-8046275 : Fastdebug build failing on jdk9/hs/ control jobs after > pulling some hs-comp changes > webrev: http://cr.openjdk.java.net/~poonam/8046275/webrev.00/ > 8u changeset: http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/rev/ad51f24671c2 > > Testing: JPRT build and testing > > Thanks, > Poonam > > From poonam.bajaj at oracle.com Wed Nov 5 00:16:44 2014 From: poonam.bajaj at oracle.com (Poonam Bajaj) Date: Tue, 04 Nov 2014 16:16:44 -0800 Subject: Review request(7u): JDK-8046275: Fastdebug build failing on jdk9/hs/ control jobs after pulling some hs-comp changes In-Reply-To: <545942AD.1020000@oracle.com> References: <5459327E.2050409@oracle.com> <545942AD.1020000@oracle.com> Message-ID: <54596C6C.1090400@oracle.com> Thanks Vladimir. Yes, I will send the approval request to jdk7u-dev alias. regards, Poonam On 11/4/2014 1:18 PM, Vladimir Kozlov wrote: > Looks good. > I think you need to send request for backport approval to > jdk7u-dev at openjdk.java.net > > Thanks, > Vladimir > > On 11/4/14 12:09 PM, Poonam Bajaj wrote: >> Hi, >> >> Please review these backport changes to 7u-dev: >> >> JDK-8046275 : >> Fastdebug build failing on jdk9/hs/ control jobs after >> pulling some hs-comp changes >> webrev: http://cr.openjdk.java.net/~poonam/8046275/webrev.00/ >> 8u changeset: >> http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/rev/ad51f24671c2 >> >> Testing: JPRT build and testing >> >> Thanks, >> Poonam >> >> From igor.ignatyev at oracle.com Wed Nov 5 10:57:31 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 05 Nov 2014 13:57:31 +0300 Subject: RFR(S) : 8043125 : compiler/types/correctness/CorrectnessTest.java: assert(layout->tag() == DataLayout::speculative_trap_data_tag) failed: wrong type In-Reply-To: <0FBF83F9-4D3C-4D1A-B8D0-3A81F9B9600C@oracle.com> References: <5451A9F5.2060207@oracle.com> <0FBF83F9-4D3C-4D1A-B8D0-3A81F9B9600C@oracle.com> Message-ID: <545A029B.7020702@oracle.com> Hi Roland, actually, I've already pushed this. However before that, I ran modified 'types/correctness/CorrectnessTest.java' to check the case similar to the one which you described : w/ the extra thread which calls WB.ClearMethodState in the loop. And I didn't get any failures, so it looks like it's safe. if we bump into problem w/ that, I'll return VM operation. Igor On 11/03/2014 04:08 PM, Roland Westrelin wrote: >> http://cr.openjdk.java.net/~iignatyev/8043125/webrev.00/ > > Wouldn?t it be better to keep the VM operation? If there?s java code running concurrently with the ClearMethodState call, profile will be updated and cleared concurrently. I?m not sure that would be safe. Or are we sure that ClearMethodState will never be run concurrently with other code? > > Roland. > From tobias.hartmann at oracle.com Wed Nov 5 12:12:53 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 05 Nov 2014 13:12:53 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' Message-ID: <545A1445.2050200@oracle.com> Hi, please review the following patch. Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ Problem: The test compiles and deoptimizes a method m multiple times. After each deoptimization the test checks if m was compiled with C2 and if so increments a counter. The test assumes that m is compilable until the counter reaches the 'PerMethodRecompilationCutoff'. This does not hold in the following case: - The first compilation request for m is added to the compile queue. Although we profiled m and the MDO suggests that m is trivial, we do not use this information because due to deoptimization there is no corresponding nmethod [1]. We compile at level 4 and execution continues in the interpreter [2]. - A second compilation request for m is issued and while it is being processed the first compilation request finishes [3]. Because a nmethod now exists, we use the MDO and decide to compile at level 1 since m is trivial. - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. After the following deoptimization the counter is not incremented because the nmethod was compiled at level 1. As a result m is set to not compilable before the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. Problems with the current implementation: (1) We should always use the MDO if it is valid, even if there is no corresponding nmethod. Otherwise we compile trivial methods at level 4 or start profiling again. (2) If there is a nmethod the MDO may still be invalid: We can compile a method immediately at C2, deoptimize, and the MDO is uninitialized. In this case we wrongly assume that the method is trivial and re-compile it with C1. (3) To avoid a level 2 or 3 compilation and profiling we should mark simple constant getter methods, like those used by the test, as trivial. (4) We should add verification code to avoid/catch such bad tiered level transitions in the future. Solution: To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we always compile the (trivial) methods at level 1 and the test fails. To fix (3) I added the method 'Method::is_constant_getter()' that determines if a method consists of a constant push and a return statement only. If so, we treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. I filed JDK-8062913 for (4) because the verification code I added still fails with the current implementation due to 2->2 and 3->2 transitions. Testing: - JPRT including failing whitebox test Thanks, Tobias [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' [2] First compilation request: InterpreterRuntime::frequency_counter_overflow InterpreterRuntime::frequency_counter_overflow_inner SimpleThresholdPolicy::event AdvancedThresholdPolicy::method_invocation_event AdvancedThresholdPolicy::call_event AdvancedThresholdPolicy::common [is_trivial() returns false because no nmethod available] [next_level = CompLevel_Full_Optimization] SimpleThresholdPolicy::compile SimpleThresholdPolicy::submit_compile CompileBroker::compile_method CompileBroker::compile_method_base CompileBroker::create_compile_task [continue execution in interpreter] [3] Second compilation request: InterpreterRuntime::frequency_counter_overflow InterpreterRuntime::frequency_counter_overflow_inner SimpleThresholdPolicy::event AdvancedThresholdPolicy::method_invocation_event [First compilation finishes here -> !CompileBroker::compilation_is_in_queue()] AdvancedThresholdPolicy::call_event AdvancedThresholdPolicy::common [is_trivial() returns true because nmethod/mdo now available] [next_level = CompLevel_Simple] [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log From roland.westrelin at oracle.com Wed Nov 5 12:52:33 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Wed, 5 Nov 2014 13:52:33 +0100 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <5459370A.1060305@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> Message-ID: <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> Vladimir, See inlined. >>> Do you know why GCM scheduled the load into previous block? Usually loads are placed near their uses in: >>> >>> 0c7 B13: # B17 B14 <- B11 Freq: 0.768468 >>> 0c7 movzwl R8, [RCX + #24 + R10 << #1] # ushort/char << SEGV >>> 0cd addl RDI, #-2 # int >>> 0d0 movl R9, #-2 # int >>> 0d6 testl R10, R10 >>> 0d9 jle,s B17 P=0.000001 C=-1.000000 >>> 0d9 >>> 0db B14: # B21 B15 <- B13 Freq: 0.768467 >>> 0db testl R8, R8 >>> 0de je,s B21 P=0.100000 C=-1.000000 >> >> There are 2 uses for the value that is loaded. I don?t have the same block numbering/register allocation when I run it so it looks like this for me: > > Ahh. Second check may coming from post-loop so we merged two loads into one which dominates both. Then this can't be fixed in GCM :( > >> >> 04f B2: # B23 B3 <- B15 Freq: 5848.51 >> 04f movzwl R10, [RDX + #16 + R8 << #1] # ushort/char >> 055 addl RCX, #-2 # int >> 058 testl R8, R8 >> 05b jle B23 P=0.000001 C=-1.000000 >> 05b >> 061 B3: # B22 B4 <- B2 Freq: 5848.5 >> 061 testl R10, R10 >> 064 je B22 P=0.000001 C=-1.000000 >> >> ... >> >> 149 B23: # B5 B24 <- B2 Freq: 0.00592617 >> 149 cmpl R8, #-1 >> 14d jle B5 P=0.100000 C=-1.000000 >> 14d >> 153 B24: # B22 B25 <- B23 Freq: 0.00533355 >> 153 testl R10, R10 >> 156 je,s B22 P=0.000001 C=-1.000000 >> >> B2 (in my case) is the LCA of both uses (B3 & B24). >> >>> castnode.hpp: >>> >>> Fields names: _required --> _carry_dependency. >> >> Better naming indeed. >> >>> I am not convinced that you need _inexact. Based on your code _inexact can be converted from 'true' to 'false' only when phase->type(cmp->in(2))->is_int()->is_con() and all conditions at the beginning of CastIINode::Ideal() are true. But if input's type is narrowed later we can still improve CastII's type. >> >> What do you mean by later? That the input becomes a constant and then the constant changes? > > If you have, for example, 'i < con' condition the LOW bound type(i)->_lo can improve later so _lo of CastII type can be also improved. But then i is data input to the CastII and that would be handled by the CastII::Value() and call to set_type from the (i)GVN? >>> Do you think we will get worse code if we don't do Identity for _carry_dependency CastII nodes? If _carry_dependency is set we can't replace it with an other node, I think, (unless all inputs are matching - hash() and cmp()). >> >> To be safe I should define hash() and cmp() for CastIINode to account for _carry_dependency (and _inexact if we keep it), right? > > I looked and CastII inherits these methods from TypeNode and Node classes. > They only match if all inputs are matching (please, verify this). I verified it. > Which means control input should be matched too - in this case it is safe to use pre-existing node. It still feels safer to me that CastII::cmp checks _carry_dependency (and _inexact) to make sure we don?t hit a case where we have 2 identical casts but one of them doesn?t have the _carry_dependency set and we lose the _carry_dependency during GVN. >> Do I think we should have Identity skip _carry_dependency CastII nodes entirely? I think we might miss some optimization opportunities if we do. For instance something like this maybe: >> >> for (int i = j; i < 100; i++) { >> value += some_array[i]; >> // some test that make the backbranch dead in the main loop >> } >> >> value += some_array[j+1]; >> >> Assuming a 1 iteration pre loop, the load after the loop should common with the load in the main loop but wouldn?t with the CastII because they wouldn't have the same address input. >> >> Anyway I couldn?t make that in an actual test case. > > Yes, please create test and see if we can skip Identity. If not, then we may indeed need this additional field or complex checks. I didn?t succeed in writing that test case. >> Also, I think we might use the modified CastII elsewhere in the future. Currently, we only add CastII during parsing when we know we compare against a constant. The modified CastII could be added even if we don?t compare against a constant and if the constant is discovered later, the CastII would help optimization. If we sprinkle CastII nodes in the graph, we would want the useless ones to go away. > > Please, write tests for such cases and see how it will work. But it should be separate RFE. We need to fix this bug first. How about I drop _inexact for now, keep _carry_dependency and investigate other uses of CastII as part of other work? >>> castnode.cpp >>> >>> The type change belongs to Value() not Ideal(). Especially if you don't need to set _inexact field and create new CastII node for that. >> >> That makes sense. >> >>> Use fatal with message which includes BoolTest::mask name instead of ShouldNotReachHere(). >> >> Ok. >> >>> loopTransform.cpp >>> >>> insert_castii_before_loop() --> cast_incr_before_loop() >>> You missed registration of new CastII nodes - register_new_node(). >> >> Ok. Thanks for spotting that. >> >>> opaquenode.cpp >>> >>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >> >> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. > > That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). Roland. > >> >>> There is loop_limit_check predicate before a counted loop which has opaqued limit. It . Those opaque nodes are removed by cleanup_loop_predicates(). Why Phi is not put on worklist at that time? >> >> So that code: >> >> Node *Opaque1Node::Identity(PhaseTransform *phase) { >> return phase->C->major_progress() ? this : in(1); >> } >> >> is not used to remove the opaque nodes? >> I thought I observed they were removed there. > > cleanup_loop_predicates() only removes Opaque1 from predicate checks and not from main- post-loop guards. > > Vladimir > >> >> Roland. >> >>> >>> Thanks, >>> Vladimir >>> >>> On 10/6/14 2:49 AM, Roland Westrelin wrote: >>>> >>>> Anyone? >>>> >>>> Roland. >>>> >>>> On Sep 17, 2014, at 7:03 PM, Roland Westrelin wrote: >>>> >>>>> http://cr.openjdk.java.net/~roland/8054478/webrev.00/ >>>>> >>>>> The IR after parsing is this in pseudo code: >>>>> >>>>> i_main = 0; >>>>> while(true) { >>>>> if (i_main >= 1000000) return; >>>>> i_main++; >>>>> char[] array = new char[1]; >>>>> if (pattern1 != null) { >>>>> i = 0; >>>>> while(true) { >>>>> if (i >= 0) { >>>>> do { >>>>> if (pattern0 == null) uncommon_trap; >>>>> if (pattern0.length u< i) uncommon_trap; >>>>> if (pattern0[i] != 0) { >>>>> if (pattern1.length u< i) uncommon_trap; >>>>> if (pattern1[i] != 0) { >>>>> goto out1; >>>>> } >>>>> } >>>>> i?; >>>>> pos?; >>>>> if (pos != -1) { >>>>> goto out2; >>>>> } >>>>> } while (i >= 0); >>>>> goto out1; >>>>> } >>>>> out2: >>>>> c = array[pos]; >>>>> } >>>>> } >>>>> out1: >>>>> } >>>>> >>>>> The do {} while loop is a CountedLoop. The null check & range check are moved out of the loop. Then a pre and a post loops are created and the body is unrolled. The pattern0[i] LoadNode nodes in the unrolled body have a control that is set to the range check before the pre loop. In the unrolled loop, because of the if (pos != -1) test in the first copy of the loop, the compiler finds that in the second copy of the loop body pos is != -1 and so the loop is exited before reaching the end of the unrolled loop. The back branch of the unrolled loop is dead. The compiler optimizes the CountedLoopNode of the unrolled loop out because it doesn?t have a back branch anymore, PhiNodes are removed and the LoadNode for pattern0[i] in the unroll loop body is now independent of the input control of the CountedLoop. Its control is still set to the range check before the pre loop. In the generated code, the pre loop is followed by the pattern1[i] access which is incorrect because it happens befor > e >>> the if (i >= 0) that dominated the unrolled loop before it was removed. >>>>> >>>>> The graph is good as long as the CountedLoop is not removed because the LoadNode depends on a PhiNode of the CountedLoop. But as soon as the CountedLoop is removed and the PhiNode is disconnected, the LoadNode no longer depends on the check that guarded the CountedLoop. >>>>> >>>>> Vladimir suggested (in a private conversation) that I use a CastII (control dependent on the if that guards the main loop with as input the input value to the induction variable in the loop) as a way to guarantee correct dependencies. There are 2 problems with this: >>>>> >>>>> 1) the CastII nodes are removed after CCP. I added a flag to the CastII nodes so we can mark CastII nodes that can?t be removed and should be left as they are post-CCP. >>>>> 2) we don?t know the range of values that are possible on the if branch that enters the main loop right away (it can be adjusted as loop opts are applied). We may actually never know it if it depends on non constants. If we use a type for a range that is too wide in the CastII the CastII may be optimized out incorrectly. I added another flag to the CastII to mark nodes that cannot be optimized. In CastII:Ideal I added code that tries to figure out a better type for the CastII so we may be able to optimize the CastII as the compiler optimizes the IR. >>>>> >>>>> Another independent issue here is that the main loop is actually never executed because the loop of the test is a single iteration loop so the compiler should be able to optimize out the main loop entirely but it doesn?t. PhiNode::Value() has this code: >>>>> >>>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>>> if( l && l->can_be_counted_loop(phase) && >>>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>>> >>>>> ... >>>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>>> } >>>>> } >>>>> } >>>>> >>>>> that returns an accurate value for the Phi node of a CountedLoop. In this case we want the pre loop?s Phi to return an accurate type that would then make the guard of the main loop optimize. The pre loop is guarded by a test on an opaque node which is later removed. If PhiNode::Value() is called before the Opaque1 node is removed then, the limit type is int and it cannot return anything accurate. If it?s called after, then limit type is a constant and PhiNode::Value() also returns a constant for the pre-loop. Then the main loop is optimized out. To fix this: I added code to Opaque1Node::Ideal so it reenqueues the Phi when the opaque node is removed. >>>>> >>>>> Roland. From roland.westrelin at oracle.com Wed Nov 5 13:00:19 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Wed, 5 Nov 2014 14:00:19 +0100 Subject: RFR(S) : 8043125 : compiler/types/correctness/CorrectnessTest.java: assert(layout->tag() == DataLayout::speculative_trap_data_tag) failed: wrong type In-Reply-To: <545A029B.7020702@oracle.com> References: <5451A9F5.2060207@oracle.com> <0FBF83F9-4D3C-4D1A-B8D0-3A81F9B9600C@oracle.com> <545A029B.7020702@oracle.com> Message-ID: <9DCA06AE-7220-476F-BE4A-E79ACF476C49@oracle.com> > actually, I've already pushed this. However before that, I ran modified 'types/correctness/CorrectnessTest.java' to check the case similar to the one which you described : w/ the extra thread which calls WB.ClearMethodState in the loop. And I didn't get any failures, so it looks like it's safe. > > if we bump into problem w/ that, I'll return VM operation. FWIW, I find this use of clean_method_data() misleading. clean_method_data only removes the klass/method references and it doesn?t reset counters or remove recorded traps so WB_ClearMethodState() doesn?t really clear the method state. It would be better, I think, if you had your own method in MethodData to clean everything. Roland. From igor.ignatyev at oracle.com Wed Nov 5 13:07:14 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 05 Nov 2014 16:07:14 +0300 Subject: RFR(S) : 8043125 : compiler/types/correctness/CorrectnessTest.java: assert(layout->tag() == DataLayout::speculative_trap_data_tag) failed: wrong type In-Reply-To: <9DCA06AE-7220-476F-BE4A-E79ACF476C49@oracle.com> References: <5451A9F5.2060207@oracle.com> <0FBF83F9-4D3C-4D1A-B8D0-3A81F9B9600C@oracle.com> <545A029B.7020702@oracle.com> <9DCA06AE-7220-476F-BE4A-E79ACF476C49@oracle.com> Message-ID: <545A2102.2040208@oracle.com> On 11/05/2014 04:00 PM, Roland Westrelin wrote: >> actually, I've already pushed this. However before that, I ran modified 'types/correctness/CorrectnessTest.java' to check the case similar to the one which you described : w/ the extra thread which calls WB.ClearMethodState in the loop. And I didn't get any failures, so it looks like it's safe. >> >> if we bump into problem w/ that, I'll return VM operation. > > FWIW, I find this use of clean_method_data() misleading. clean_method_data only removes the klass/method references and it doesn?t reset counters or remove recorded traps so WB_ClearMethodState() doesn?t really clear the method state. It would be better, I think, if you had your own method in MethodData to clean everything. agree, filed JDK-8062929 for this. thank you Roland. > > Roland. > From zoltan.majo at oracle.com Wed Nov 5 13:34:34 2014 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Wed, 05 Nov 2014 14:34:34 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <54591D84.30709@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> <5458CED2.6030203@oracle.com> <54591D84.30709@oracle.com> Message-ID: <545A276A.9020509@oracle.com> Hi Vladimir, thank you for the feedback. On 11/04/2014 07:40 PM, Vladimir Kozlov wrote: > On 11/4/14 5:04 AM, Zolt?n Maj? wrote: >> Hi Roland, Vladimir, and Albert! >> >> >> thank you for your feedback and for your suggestions! Please see >> comments on specific issues inline. >> >> On 11/04/2014 09:38 AM, Albert Noll wrote: >>> Hi, >>> >>> please see comments inline. >>> On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >>>> I agree with additional check tak != TypeKlassPtr::OBJECT (solution >>>> 2). >>>> >>>> Instead of creating new LoadKlassNode::make() method you can set >>>> control in parseHelper.cpp since it is the only >>>> place which set it: >>>> >>>> Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), >>>> p2, tak); >>>> if (always_see_exact_class) { >>>> a_e_klass->init_req(MemNode::Control, control()); >>>> } >>>> a_e_klass = _gvn.transform(a_e_klass); >>>> >>> An alternative solution is to make a default parameter for control. >>> This way we can set the control in make() without >>> having to add a new make() function. >> >> I took Albert's alternative solution. Now we also set the default >> value for parameter TypeKlassPtr* tk in >> parseHelper.cpp at line 230. > > As I replied to Albert, I think it should be explicit parameter and > not default. > Yes, you would need to modify more files but it is better to be > explicit I think. OK, I see. I updated the code so that we have one LoadKlassNode::make() method, with the control node as second parameter. In the new version the control parameter is explicit and we pass in NULL explicitly at all call sites where control is not needed. > >> >>>> Also an other reason to have control edge in this place is the >>>> address of this klass load is based on ConP constant. >>>> In a normal case after class check you should have CheckCastPP >>>> attached to control and used as address's base. >> >> I see, Vladimir. Just a related question: If the address of the klass >> load were *not* based on a ConP constant, would it >> be better (or required) to check with a CheckCastPP the value that >> was loaded? > > Yes, you would use CheckCastPP to cast type to exact class you checked > for and pin it to 'true' projection of the check. As we do for in > gen_checkcast() or other places (speculative types). > > if (a.class == exact_class) > (exact_class)a Thank you for the explanation. > > In this case you don't need control on following load. > >> >>>> >>>> And agree with Roland's suggestion about Op_LoadKlass check. >>>> >>> What you are suggesting is to make a super class (LoadNode) aware of >>> its subclasses (LoadPNode and LoadKlassNode). >>> From a software engineering point of view, we should try to avoid >>> doing that. Another solution would be to add a >>> method (e.g., can_remove_control()) to LoadNode, LoadPNode, and >>> LoadKlassNode. The function returns 'false' for >>> LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What >>> do you think? >> >> I think Albert's suggestion has the advantage that if we will ever >> decide to inherit from LoadKlass, we won't have to >> worry about modifying its superclass. So I think it will be good if >> we go with it. > > I prefer Albert's name can_remove_control() OK, I changed the method's name. Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.03/ All JPRT tests and also the failing test cases pass with the new version. Thank you and best regards, Zoltan > > Thanks, > Vladimir > >> >> Here is the new webrev: >> http://cr.openjdk.java.net/~zmajo/8057622/webrev.02/ >> >> JPRT and also the failing test cases pass. >> >> >> Thank you and best regards, >> >> >> Zoltan >> >>> >>> Best, >>> Albert >>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>>>> Hi, >>>>> >>>>> >>>>> please review the following patch. >>>>> >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>>>> >>>>> >>>>> Problem: >>>>> >>>>> >>>>> We have five tests that fail with SIGSEGV in our nightlies: >>>>> >>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>>>> >>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>>>> >>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>>>> >>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>>>> >>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>>>> >>>>> All tests fail when executing the C2-compiled version of >>>>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>>>> >>>>> >>>>> 489: private void inflateSpine() { >>>>> 490: if (spine == null) { >>>>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>>>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>>>> 493: spine[0] = curChunk; >>>>> 494: } >>>>> 495: } >>>>> >>>>> >>>>> The failure is due to the 'aastore' at line 493; the failure is >>>>> caused >>>>> by the monomorphic array check optimization >>>>> (-XX:+MonomorphicArrayCheck, enabled by default). >>>>> >>>>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>>>> profiling information) that inflateSpine() has two hot receiver >>>>> types, >>>>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>>>> >>>>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>>>> >>>>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>>>> double[][]. >>>>> >>>>> Please consider the following pseudo Java code that illustrates >>>>> how C2 >>>>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on >>>>> the two hot receiver types SpinedBuffer.OfInt and >>>>> SpinedBuffer.OfDouble: >>>>> >>>>> >>>>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>>>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>>>> spine[0] = curChunk; >>>>> } >>>>> >>>>> >>>>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>>>> of 'spine' *at runtime*. The check looks something like: >>>>> >>>>> >>>>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>>>> throw new ArrayStoreException(); >>>>> } >>>>> >>>>> >>>>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>>>> check like this: >>>>> >>>>> >>>>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>>>> throw new ArrayStoreException(); >>>>> } >>>>> >>>>> >>>>> where TYPE is the type of the 'spine' variable, as it is >>>>> determined by >>>>> C2 *at compile time*. The optimization treats TYPE as a constant and >>>>> thus saves a load from memory + an offset calculation (I think). >>>>> >>>>> The problem is that due to 'spine' being either of type int[][] or of >>>>> type double[][], C2 determines that TYPE==java/lang/Object. >>>>> >>>>> As a result, when the inflateSpine() method is executed, it first >>>>> loads the >>>>> Klass* corresponding to java/lang/Object (TYPE). Let us call this >>>>> Klass* >>>>> array_klass. Then, the method obtains a Klass* from >>>>> array_klass->_element_klass. >>>>> Then, the method reads from offset 20 from >>>>> array_klass->_element_klass, which >>>>> results in a SIGSEGV. >>>>> >>>>> The reason for the SIGSEGV is that the Klass* array_klass is of type >>>>> java/lang/Object and is therefore represented as an 'InstanceKlass'. >>>>> But _element_klass is defined only in objects of type >>>>> 'ObjArrayKlass'. >>>>> The compiler reads array_klass->_element_klass because it expects the >>>>> destination of an 'aastore' to be an array, which is not true if >>>>> TYPE==java/lang/Object. >>>>> >>>>> >>>>> Solution: >>>>> >>>>> >>>>> The compiler already checks if the compile-time type of the array >>>>> (TYPE) is >>>>> the same as the array's type at runtime. If that check fails, the >>>>> method >>>>> branches to an uncommon trap. In our case the SIGSEGV happens because >>>>> the load of array_klass->_element_klass "floats" above the check. >>>>> >>>>> I propose two solutions: >>>>> >>>>> Solution 1. Add a control edge between the IfTrue branch of the >>>>> check and >>>>> the load of _array_klass->_element_klass. That prohibits the load of >>>>> _array_klass->element_klass floating above the check. >>>>> >>>>> Solution 2. In addition to the changes in Solution 1, the compiler >>>>> checks if >>>>> the compile-time type of the array (TYPE) is java/lang/Object. If >>>>> TYPE==java/lang/Object, the compiler should not assume array_type >>>>> to be >>>>> TYPE, but should determine array type at runtime instead. >>>>> >>>>> The reason is for Solution 2 is: We can't do an array store into a >>>>> java/lang/Object. >>>>> So, if we see TYPE==java/lang/Object at compile time, it is pretty >>>>> sure that the >>>>> method will deoptimize at runtime. Instead of assuming TYPE to be >>>>> java/lang/Object, >>>>> we read at runtime the type of the array store's destination. That >>>>> still gives us some >>>>> chance that the compiled version of the method will successfully >>>>> execute. The >>>>> additional check is in parseHelper.cpp in >>>>> Parse::array_store_check() on line 171. >>>>> >>>>> >>>>> Webrev: >>>>> >>>>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>>>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>>>> >>>>> >>>>> Testing (for both solutions): >>>>> >>>>> JPRT, manual testing with failing test cases. >>>>> >>>>> >>>>> Please note that both solutions are somewhat "hacky", because that >>>>> allowed me to >>>>> produce a solution within reasonable time and with reasonable >>>>> effort. Any suggestions >>>>> or feedback is welcome. >>>>> >>>>> >>>>> Thank you and best regards, >>>>> >>>>> >>>>> Zoltan >>>>> >>> >> From nils.eliasson at oracle.com Wed Nov 5 14:21:54 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Wed, 05 Nov 2014 15:21:54 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545A1445.2050200@oracle.com> References: <545A1445.2050200@oracle.com> Message-ID: <545A3282.6060308@oracle.com> Hi Tobias, Looks good, Thanks for fixing this, Nils (not a reviewer) On 2014-11-05 13:12, Tobias Hartmann wrote: > Hi, > > please review the following patch. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 > Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ > > Problem: > The test compiles and deoptimizes a method m multiple times. After each > deoptimization the test checks if m was compiled with C2 and if so increments a > counter. The test assumes that m is compilable until the counter reaches the > 'PerMethodRecompilationCutoff'. This does not hold in the following case: > - The first compilation request for m is added to the compile queue. Although we > profiled m and the MDO suggests that m is trivial, we do not use this > information because due to deoptimization there is no corresponding nmethod [1]. > We compile at level 4 and execution continues in the interpreter [2]. > - A second compilation request for m is issued and while it is being processed > the first compilation request finishes [3]. Because a nmethod now exists, we use > the MDO and decide to compile at level 1 since m is trivial. > - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. > > After the following deoptimization the counter is not incremented because the > nmethod was compiled at level 1. As a result m is set to not compilable before > the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. > > Problems with the current implementation: > (1) We should always use the MDO if it is valid, even if there is no > corresponding nmethod. Otherwise we compile trivial methods at level 4 or start > profiling again. > (2) If there is a nmethod the MDO may still be invalid: We can compile a method > immediately at C2, deoptimize, and the MDO is uninitialized. In this case we > wrongly assume that the method is trivial and re-compile it with C1. > (3) To avoid a level 2 or 3 compilation and profiling we should mark simple > constant getter methods, like those used by the test, as trivial. > (4) We should add verification code to avoid/catch such bad tiered level > transitions in the future. > > Solution: > To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines > if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 > compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the > MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we > always compile the (trivial) methods at level 1 and the test fails. > > To fix (3) I added the method 'Method::is_constant_getter()' that determines if > a method consists of a constant push and a return statement only. If so, we > treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. > > I filed JDK-8062913 for (4) because the verification code I added still fails > with the current implementation due to 2->2 and 3->2 transitions. > > Testing: > - JPRT including failing whitebox test > > Thanks, > Tobias > > > [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' > [2] First compilation request: > InterpreterRuntime::frequency_counter_overflow > InterpreterRuntime::frequency_counter_overflow_inner > SimpleThresholdPolicy::event > AdvancedThresholdPolicy::method_invocation_event > AdvancedThresholdPolicy::call_event > AdvancedThresholdPolicy::common > [is_trivial() returns false because no nmethod available] > [next_level = CompLevel_Full_Optimization] > SimpleThresholdPolicy::compile > SimpleThresholdPolicy::submit_compile > CompileBroker::compile_method > CompileBroker::compile_method_base > CompileBroker::create_compile_task > [continue execution in interpreter] > > [3] Second compilation request: > InterpreterRuntime::frequency_counter_overflow > InterpreterRuntime::frequency_counter_overflow_inner > SimpleThresholdPolicy::event > AdvancedThresholdPolicy::method_invocation_event > [First compilation finishes here -> !CompileBroker::compilation_is_in_queue()] > AdvancedThresholdPolicy::call_event > AdvancedThresholdPolicy::common > [is_trivial() returns true because nmethod/mdo now available] > [next_level = CompLevel_Simple] > [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] > > [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log > [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log > [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log From tobias.hartmann at oracle.com Wed Nov 5 14:39:04 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 05 Nov 2014 15:39:04 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545A3282.6060308@oracle.com> References: <545A1445.2050200@oracle.com> <545A3282.6060308@oracle.com> Message-ID: <545A3688.7020701@oracle.com> Thanks, Nils. Best, Tobias On 05.11.2014 15:21, Nils Eliasson wrote: > Hi Tobias, > > Looks good, > > Thanks for fixing this, > Nils > (not a reviewer) > > On 2014-11-05 13:12, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >> >> Problem: >> The test compiles and deoptimizes a method m multiple times. After each >> deoptimization the test checks if m was compiled with C2 and if so increments a >> counter. The test assumes that m is compilable until the counter reaches the >> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >> - The first compilation request for m is added to the compile queue. Although we >> profiled m and the MDO suggests that m is trivial, we do not use this >> information because due to deoptimization there is no corresponding nmethod [1]. >> We compile at level 4 and execution continues in the interpreter [2]. >> - A second compilation request for m is issued and while it is being processed >> the first compilation request finishes [3]. Because a nmethod now exists, we use >> the MDO and decide to compile at level 1 since m is trivial. >> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >> >> After the following deoptimization the counter is not incremented because the >> nmethod was compiled at level 1. As a result m is set to not compilable before >> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >> >> Problems with the current implementation: >> (1) We should always use the MDO if it is valid, even if there is no >> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >> profiling again. >> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >> wrongly assume that the method is trivial and re-compile it with C1. >> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >> constant getter methods, like those used by the test, as trivial. >> (4) We should add verification code to avoid/catch such bad tiered level >> transitions in the future. >> >> Solution: >> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >> always compile the (trivial) methods at level 1 and the test fails. >> >> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >> a method consists of a constant push and a return statement only. If so, we >> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >> >> I filed JDK-8062913 for (4) because the verification code I added still fails >> with the current implementation due to 2->2 and 3->2 transitions. >> >> Testing: >> - JPRT including failing whitebox test >> >> Thanks, >> Tobias >> >> >> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >> [2] First compilation request: >> InterpreterRuntime::frequency_counter_overflow >> InterpreterRuntime::frequency_counter_overflow_inner >> SimpleThresholdPolicy::event >> AdvancedThresholdPolicy::method_invocation_event >> AdvancedThresholdPolicy::call_event >> AdvancedThresholdPolicy::common >> [is_trivial() returns false because no nmethod available] >> [next_level = CompLevel_Full_Optimization] >> SimpleThresholdPolicy::compile >> SimpleThresholdPolicy::submit_compile >> CompileBroker::compile_method >> CompileBroker::compile_method_base >> CompileBroker::create_compile_task >> [continue execution in interpreter] >> >> [3] Second compilation request: >> InterpreterRuntime::frequency_counter_overflow >> InterpreterRuntime::frequency_counter_overflow_inner >> SimpleThresholdPolicy::event >> AdvancedThresholdPolicy::method_invocation_event >> [First compilation finishes here -> >> !CompileBroker::compilation_is_in_queue()] >> AdvancedThresholdPolicy::call_event >> AdvancedThresholdPolicy::common >> [is_trivial() returns true because nmethod/mdo now available] >> [next_level = CompLevel_Simple] >> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >> >> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log > From martin.doerr at sap.com Wed Nov 5 15:38:17 2014 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 5 Nov 2014 15:38:17 +0000 Subject: RFR (XS): 8062950: Bug in locking code when UseOptoBiasInlining is disabled: assert(dmw->is_neutral()) failed: invariant Message-ID: <7C9B87B351A4BA4AA9EC95BB418116566ACE597E@DEWDFEMB19C.global.corp.sap> Hi, we found a bug in MacroAssembler::fast_lock on x86 which shows up when UseOptoBiasInlining is switched off. The problem is that biased_locking_enter is used with swap_reg_contains_mark==true, which is no longer correct after biased_locking_enter was put in front of check for IsInflated. Please review http://cr.openjdk.java.net/~goetz/webrevs/8062950-lockBug/webrev.00/ Best regards, Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Wed Nov 5 16:25:07 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 05 Nov 2014 19:25:07 +0300 Subject: RFR(M) : 8059624 : Test task: WhiteBox API for testing segmented codecache feature Message-ID: <545A4F63.6010008@oracle.com> http://cr.openjdk.java.net/~iignatyev/8059624/webrev.00/ 660 lines changed: 624 ins; 2 del; 34 mod; Hi all, please review the patch which adds new WhiteBox methods needed for better testing SegmentedCodeCache: - perform allocation in code cache - force code cache sweep - lock/unlock compilation - get a segment id for nmethod. besides these methods, the patch also adds a method to get all entries in code heap. changes in product code: - monitor 'Compilation_lock' was added to implement compilation locking/unlocking - align_code_offset function was made a static member of CodeBlob - WhiteBox was made a friend of several classes testing: jprt, new added tests jbs: https://bugs.openjdk.java.net/browse/JDK-8059624 -- Thanks, Igor From vladimir.kozlov at oracle.com Wed Nov 5 16:59:43 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 05 Nov 2014 08:59:43 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> Message-ID: <545A577F.5080805@oracle.com> On 11/5/14 4:52 AM, Roland Westrelin wrote: > Vladimir, > > See inlined. > >>> >>>> I am not convinced that you need _inexact. Based on your code _inexact can be converted from 'true' to 'false' only when phase->type(cmp->in(2))->is_int()->is_con() and all conditions at the beginning of CastIINode::Ideal() are true. But if input's type is narrowed later we can still improve CastII's type. >>> >>> What do you mean by later? That the input becomes a constant and then the constant changes? >> >> If you have, for example, 'i < con' condition the LOW bound type(i)->_lo can improve later so _lo of CastII type can be also improved. > > But then i is data input to the CastII and that would be handled by the CastII::Value() and call to set_type from the (i)GVN? I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. > >>>> Do you think we will get worse code if we don't do Identity for _carry_dependency CastII nodes? If _carry_dependency is set we can't replace it with an other node, I think, (unless all inputs are matching - hash() and cmp()). >>> >>> To be safe I should define hash() and cmp() for CastIINode to account for _carry_dependency (and _inexact if we keep it), right? >> >> I looked and CastII inherits these methods from TypeNode and Node classes. >> They only match if all inputs are matching (please, verify this). > > I verified it. > >> Which means control input should be matched too - in this case it is safe to use pre-existing node. > > It still feels safer to me that CastII::cmp checks _carry_dependency (and _inexact) to make sure we don?t hit a case where we have 2 identical casts but one of them doesn?t have the _carry_dependency set and we lose the _carry_dependency during GVN. Yes, we can play safe for now. > >>> Do I think we should have Identity skip _carry_dependency CastII nodes entirely? I think we might miss some optimization opportunities if we do. For instance something like this maybe: >>> >>> for (int i = j; i < 100; i++) { >>> value += some_array[i]; >>> // some test that make the backbranch dead in the main loop >>> } >>> >>> value += some_array[j+1]; >>> >>> Assuming a 1 iteration pre loop, the load after the loop should common with the load in the main loop but wouldn?t with the CastII because they wouldn't have the same address input. >>> >>> Anyway I couldn?t make that in an actual test case. >> >> Yes, please create test and see if we can skip Identity. If not, then we may indeed need this additional field or complex checks. > > I didn?t succeed in writing that test case. Loads in pre-loop will not have CastII. We add them only for main- and post-loop. One iteration loops are converted before we split them to pre-main-post. I think it would be very rare case where we can lost opportunity to common loads. I think Identity() should behave the same as hash(). And I really don't want _inexact field. You can put code under UseNewCode and run benchmarks in our performance infrastructure. > >>> Also, I think we might use the modified CastII elsewhere in the future. Currently, we only add CastII during parsing when we know we compare against a constant. The modified CastII could be added even if we don?t compare against a constant and if the constant is discovered later, the CastII would help optimization. If we sprinkle CastII nodes in the graph, we would want the useless ones to go away. >> >> Please, write tests for such cases and see how it will work. But it should be separate RFE. We need to fix this bug first. > > How about I drop _inexact for now, keep _carry_dependency and investigate other uses of CastII as part of other work? Agree. > >>>> castnode.cpp >>>> >>>> The type change belongs to Value() not Ideal(). Especially if you don't need to set _inexact field and create new CastII node for that. >>> >>> That makes sense. >>> >>>> Use fatal with message which includes BoolTest::mask name instead of ShouldNotReachHere(). >>> >>> Ok. >>> >>>> loopTransform.cpp >>>> >>>> insert_castii_before_loop() --> cast_incr_before_loop() >>>> You missed registration of new CastII nodes - register_new_node(). >>> >>> Ok. Thanks for spotting that. >>> >>>> opaquenode.cpp >>>> >>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>> >>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >> >> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. > > The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. Thanks, Vladimir > > Roland. > >> >>> >>>> There is loop_limit_check predicate before a counted loop which has opaqued limit. It . Those opaque nodes are removed by cleanup_loop_predicates(). Why Phi is not put on worklist at that time? >>> >>> So that code: >>> >>> Node *Opaque1Node::Identity(PhaseTransform *phase) { >>> return phase->C->major_progress() ? this : in(1); >>> } >>> >>> is not used to remove the opaque nodes? >>> I thought I observed they were removed there. >> >> cleanup_loop_predicates() only removes Opaque1 from predicate checks and not from main- post-loop guards. >> >> Vladimir >> >>> >>> Roland. >>> >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 10/6/14 2:49 AM, Roland Westrelin wrote: >>>>> >>>>> Anyone? >>>>> >>>>> Roland. >>>>> >>>>> On Sep 17, 2014, at 7:03 PM, Roland Westrelin wrote: >>>>> >>>>>> http://cr.openjdk.java.net/~roland/8054478/webrev.00/ >>>>>> >>>>>> The IR after parsing is this in pseudo code: >>>>>> >>>>>> i_main = 0; >>>>>> while(true) { >>>>>> if (i_main >= 1000000) return; >>>>>> i_main++; >>>>>> char[] array = new char[1]; >>>>>> if (pattern1 != null) { >>>>>> i = 0; >>>>>> while(true) { >>>>>> if (i >= 0) { >>>>>> do { >>>>>> if (pattern0 == null) uncommon_trap; >>>>>> if (pattern0.length u< i) uncommon_trap; >>>>>> if (pattern0[i] != 0) { >>>>>> if (pattern1.length u< i) uncommon_trap; >>>>>> if (pattern1[i] != 0) { >>>>>> goto out1; >>>>>> } >>>>>> } >>>>>> i?; >>>>>> pos?; >>>>>> if (pos != -1) { >>>>>> goto out2; >>>>>> } >>>>>> } while (i >= 0); >>>>>> goto out1; >>>>>> } >>>>>> out2: >>>>>> c = array[pos]; >>>>>> } >>>>>> } >>>>>> out1: >>>>>> } >>>>>> >>>>>> The do {} while loop is a CountedLoop. The null check & range check are moved out of the loop. Then a pre and a post loops are created and the body is unrolled. The pattern0[i] LoadNode nodes in the unrolled body have a control that is set to the range check before the pre loop. In the unrolled loop, because of the if (pos != -1) test in the first copy of the loop, the compiler finds that in the second copy of the loop body pos is != -1 and so the loop is exited before reaching the end of the unrolled loop. The back branch of the unrolled loop is dead. The compiler optimizes the CountedLoopNode of the unrolled loop out because it doesn?t have a back branch anymore, PhiNodes are removed and the LoadNode for pattern0[i] in the unroll loop body is now independent of the input control of the CountedLoop. Its control is still set to the range check before the pre loop. In the generated code, the pre loop is followed by the pattern1[i] access which is incorrect because it happens bef or >> e >>>> the if (i >= 0) that dominated the unrolled loop before it was removed. >>>>>> >>>>>> The graph is good as long as the CountedLoop is not removed because the LoadNode depends on a PhiNode of the CountedLoop. But as soon as the CountedLoop is removed and the PhiNode is disconnected, the LoadNode no longer depends on the check that guarded the CountedLoop. >>>>>> >>>>>> Vladimir suggested (in a private conversation) that I use a CastII (control dependent on the if that guards the main loop with as input the input value to the induction variable in the loop) as a way to guarantee correct dependencies. There are 2 problems with this: >>>>>> >>>>>> 1) the CastII nodes are removed after CCP. I added a flag to the CastII nodes so we can mark CastII nodes that can?t be removed and should be left as they are post-CCP. >>>>>> 2) we don?t know the range of values that are possible on the if branch that enters the main loop right away (it can be adjusted as loop opts are applied). We may actually never know it if it depends on non constants. If we use a type for a range that is too wide in the CastII the CastII may be optimized out incorrectly. I added another flag to the CastII to mark nodes that cannot be optimized. In CastII:Ideal I added code that tries to figure out a better type for the CastII so we may be able to optimize the CastII as the compiler optimizes the IR. >>>>>> >>>>>> Another independent issue here is that the main loop is actually never executed because the loop of the test is a single iteration loop so the compiler should be able to optimize out the main loop entirely but it doesn?t. PhiNode::Value() has this code: >>>>>> >>>>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>>>> if( l && l->can_be_counted_loop(phase) && >>>>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>>>> >>>>>> ... >>>>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>>>> } >>>>>> } >>>>>> } >>>>>> >>>>>> that returns an accurate value for the Phi node of a CountedLoop. In this case we want the pre loop?s Phi to return an accurate type that would then make the guard of the main loop optimize. The pre loop is guarded by a test on an opaque node which is later removed. If PhiNode::Value() is called before the Opaque1 node is removed then, the limit type is int and it cannot return anything accurate. If it?s called after, then limit type is a constant and PhiNode::Value() also returns a constant for the pre-loop. Then the main loop is optimized out. To fix this: I added code to Opaque1Node::Ideal so it reenqueues the Phi when the opaque node is removed. >>>>>> >>>>>> Roland. > From vladimir.kozlov at oracle.com Wed Nov 5 17:18:13 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 05 Nov 2014 09:18:13 -0800 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <545A276A.9020509@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> <5458CED2.6030203@oracle.com> <54591D84.30709@oracle.com> <545A276A.9020509@oracle.com> Message-ID: <545A5BD5.60000@oracle.com> This looks good. Don't forget to add label noreg-sqe since we have tests already. Thanks, Vladimir On 11/5/14 5:34 AM, Zolt?n Maj? wrote: > Hi Vladimir, > > > thank you for the feedback. > > On 11/04/2014 07:40 PM, Vladimir Kozlov wrote: >> On 11/4/14 5:04 AM, Zolt?n Maj? wrote: >>> Hi Roland, Vladimir, and Albert! >>> >>> >>> thank you for your feedback and for your suggestions! Please see comments on specific issues inline. >>> >>> On 11/04/2014 09:38 AM, Albert Noll wrote: >>>> Hi, >>>> >>>> please see comments inline. >>>> On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >>>>> I agree with additional check tak != TypeKlassPtr::OBJECT (solution 2). >>>>> >>>>> Instead of creating new LoadKlassNode::make() method you can set control in parseHelper.cpp since it is the only >>>>> place which set it: >>>>> >>>>> Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), p2, tak); >>>>> if (always_see_exact_class) { >>>>> a_e_klass->init_req(MemNode::Control, control()); >>>>> } >>>>> a_e_klass = _gvn.transform(a_e_klass); >>>>> >>>> An alternative solution is to make a default parameter for control. This way we can set the control in make() without >>>> having to add a new make() function. >>> >>> I took Albert's alternative solution. Now we also set the default value for parameter TypeKlassPtr* tk in >>> parseHelper.cpp at line 230. >> >> As I replied to Albert, I think it should be explicit parameter and not default. >> Yes, you would need to modify more files but it is better to be explicit I think. > > OK, I see. I updated the code so that we have one LoadKlassNode::make() method, with the control node as second > parameter. In the new version the control parameter is explicit and we pass in NULL explicitly at all call sites where > control is not needed. > >> >>> >>>>> Also an other reason to have control edge in this place is the address of this klass load is based on ConP constant. >>>>> In a normal case after class check you should have CheckCastPP attached to control and used as address's base. >>> >>> I see, Vladimir. Just a related question: If the address of the klass load were *not* based on a ConP constant, would it >>> be better (or required) to check with a CheckCastPP the value that was loaded? >> >> Yes, you would use CheckCastPP to cast type to exact class you checked for and pin it to 'true' projection of the >> check. As we do for in gen_checkcast() or other places (speculative types). >> >> if (a.class == exact_class) >> (exact_class)a > > Thank you for the explanation. > >> >> In this case you don't need control on following load. >> >>> >>>>> >>>>> And agree with Roland's suggestion about Op_LoadKlass check. >>>>> >>>> What you are suggesting is to make a super class (LoadNode) aware of its subclasses (LoadPNode and LoadKlassNode). >>>> From a software engineering point of view, we should try to avoid doing that. Another solution would be to add a >>>> method (e.g., can_remove_control()) to LoadNode, LoadPNode, and LoadKlassNode. The function returns 'false' for >>>> LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What do you think? >>> >>> I think Albert's suggestion has the advantage that if we will ever decide to inherit from LoadKlass, we won't have to >>> worry about modifying its superclass. So I think it will be good if we go with it. >> >> I prefer Albert's name can_remove_control() > > OK, I changed the method's name. > > Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.03/ > > All JPRT tests and also the failing test cases pass with the new version. > > Thank you and best regards, > > > Zoltan > >> >> Thanks, >> Vladimir >> >>> >>> Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.02/ >>> >>> JPRT and also the failing test cases pass. >>> >>> >>> Thank you and best regards, >>> >>> >>> Zoltan >>> >>>> >>>> Best, >>>> Albert >>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>>>>> Hi, >>>>>> >>>>>> >>>>>> please review the following patch. >>>>>> >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>>>>> >>>>>> >>>>>> Problem: >>>>>> >>>>>> >>>>>> We have five tests that fail with SIGSEGV in our nightlies: >>>>>> >>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>>>>> >>>>>> All tests fail when executing the C2-compiled version of >>>>>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>>>>> >>>>>> >>>>>> 489: private void inflateSpine() { >>>>>> 490: if (spine == null) { >>>>>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>>>>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>>>>> 493: spine[0] = curChunk; >>>>>> 494: } >>>>>> 495: } >>>>>> >>>>>> >>>>>> The failure is due to the 'aastore' at line 493; the failure is caused >>>>>> by the monomorphic array check optimization >>>>>> (-XX:+MonomorphicArrayCheck, enabled by default). >>>>>> >>>>>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>>>>> profiling information) that inflateSpine() has two hot receiver types, >>>>>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>>>>> >>>>>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>>>>> >>>>>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>>>>> double[][]. >>>>>> >>>>>> Please consider the following pseudo Java code that illustrates how C2 >>>>>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on >>>>>> the two hot receiver types SpinedBuffer.OfInt and >>>>>> SpinedBuffer.OfDouble: >>>>>> >>>>>> >>>>>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>>>>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>>>>> spine[0] = curChunk; >>>>>> } >>>>>> >>>>>> >>>>>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>>>>> of 'spine' *at runtime*. The check looks something like: >>>>>> >>>>>> >>>>>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>>>>> throw new ArrayStoreException(); >>>>>> } >>>>>> >>>>>> >>>>>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>>>>> check like this: >>>>>> >>>>>> >>>>>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>>>>> throw new ArrayStoreException(); >>>>>> } >>>>>> >>>>>> >>>>>> where TYPE is the type of the 'spine' variable, as it is determined by >>>>>> C2 *at compile time*. The optimization treats TYPE as a constant and >>>>>> thus saves a load from memory + an offset calculation (I think). >>>>>> >>>>>> The problem is that due to 'spine' being either of type int[][] or of >>>>>> type double[][], C2 determines that TYPE==java/lang/Object. >>>>>> >>>>>> As a result, when the inflateSpine() method is executed, it first loads the >>>>>> Klass* corresponding to java/lang/Object (TYPE). Let us call this Klass* >>>>>> array_klass. Then, the method obtains a Klass* from array_klass->_element_klass. >>>>>> Then, the method reads from offset 20 from array_klass->_element_klass, which >>>>>> results in a SIGSEGV. >>>>>> >>>>>> The reason for the SIGSEGV is that the Klass* array_klass is of type >>>>>> java/lang/Object and is therefore represented as an 'InstanceKlass'. >>>>>> But _element_klass is defined only in objects of type 'ObjArrayKlass'. >>>>>> The compiler reads array_klass->_element_klass because it expects the >>>>>> destination of an 'aastore' to be an array, which is not true if >>>>>> TYPE==java/lang/Object. >>>>>> >>>>>> >>>>>> Solution: >>>>>> >>>>>> >>>>>> The compiler already checks if the compile-time type of the array (TYPE) is >>>>>> the same as the array's type at runtime. If that check fails, the method >>>>>> branches to an uncommon trap. In our case the SIGSEGV happens because >>>>>> the load of array_klass->_element_klass "floats" above the check. >>>>>> >>>>>> I propose two solutions: >>>>>> >>>>>> Solution 1. Add a control edge between the IfTrue branch of the check and >>>>>> the load of _array_klass->_element_klass. That prohibits the load of >>>>>> _array_klass->element_klass floating above the check. >>>>>> >>>>>> Solution 2. In addition to the changes in Solution 1, the compiler checks if >>>>>> the compile-time type of the array (TYPE) is java/lang/Object. If >>>>>> TYPE==java/lang/Object, the compiler should not assume array_type to be >>>>>> TYPE, but should determine array type at runtime instead. >>>>>> >>>>>> The reason is for Solution 2 is: We can't do an array store into a java/lang/Object. >>>>>> So, if we see TYPE==java/lang/Object at compile time, it is pretty sure that the >>>>>> method will deoptimize at runtime. Instead of assuming TYPE to be java/lang/Object, >>>>>> we read at runtime the type of the array store's destination. That still gives us some >>>>>> chance that the compiled version of the method will successfully execute. The >>>>>> additional check is in parseHelper.cpp in Parse::array_store_check() on line 171. >>>>>> >>>>>> >>>>>> Webrev: >>>>>> >>>>>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>>>>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>>>>> >>>>>> >>>>>> Testing (for both solutions): >>>>>> >>>>>> JPRT, manual testing with failing test cases. >>>>>> >>>>>> >>>>>> Please note that both solutions are somewhat "hacky", because that allowed me to >>>>>> produce a solution within reasonable time and with reasonable effort. Any suggestions >>>>>> or feedback is welcome. >>>>>> >>>>>> >>>>>> Thank you and best regards, >>>>>> >>>>>> >>>>>> Zoltan >>>>>> >>>> >>> > From igor.veresov at oracle.com Wed Nov 5 20:19:17 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Wed, 5 Nov 2014 10:19:17 -1000 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545A1445.2050200@oracle.com> References: <545A1445.2050200@oracle.com> Message-ID: Looks good to me. igor On Nov 5, 2014, at 2:12 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 > Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ > > Problem: > The test compiles and deoptimizes a method m multiple times. After each > deoptimization the test checks if m was compiled with C2 and if so increments a > counter. The test assumes that m is compilable until the counter reaches the > 'PerMethodRecompilationCutoff'. This does not hold in the following case: > - The first compilation request for m is added to the compile queue. Although we > profiled m and the MDO suggests that m is trivial, we do not use this > information because due to deoptimization there is no corresponding nmethod [1]. > We compile at level 4 and execution continues in the interpreter [2]. > - A second compilation request for m is issued and while it is being processed > the first compilation request finishes [3]. Because a nmethod now exists, we use > the MDO and decide to compile at level 1 since m is trivial. > - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. > > After the following deoptimization the counter is not incremented because the > nmethod was compiled at level 1. As a result m is set to not compilable before > the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. > > Problems with the current implementation: > (1) We should always use the MDO if it is valid, even if there is no > corresponding nmethod. Otherwise we compile trivial methods at level 4 or start > profiling again. > (2) If there is a nmethod the MDO may still be invalid: We can compile a method > immediately at C2, deoptimize, and the MDO is uninitialized. In this case we > wrongly assume that the method is trivial and re-compile it with C1. > (3) To avoid a level 2 or 3 compilation and profiling we should mark simple > constant getter methods, like those used by the test, as trivial. > (4) We should add verification code to avoid/catch such bad tiered level > transitions in the future. > > Solution: > To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines > if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 > compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the > MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we > always compile the (trivial) methods at level 1 and the test fails. > > To fix (3) I added the method 'Method::is_constant_getter()' that determines if > a method consists of a constant push and a return statement only. If so, we > treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. > > I filed JDK-8062913 for (4) because the verification code I added still fails > with the current implementation due to 2->2 and 3->2 transitions. > > Testing: > - JPRT including failing whitebox test > > Thanks, > Tobias > > > [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' > [2] First compilation request: > InterpreterRuntime::frequency_counter_overflow > InterpreterRuntime::frequency_counter_overflow_inner > SimpleThresholdPolicy::event > AdvancedThresholdPolicy::method_invocation_event > AdvancedThresholdPolicy::call_event > AdvancedThresholdPolicy::common > [is_trivial() returns false because no nmethod available] > [next_level = CompLevel_Full_Optimization] > SimpleThresholdPolicy::compile > SimpleThresholdPolicy::submit_compile > CompileBroker::compile_method > CompileBroker::compile_method_base > CompileBroker::create_compile_task > [continue execution in interpreter] > > [3] Second compilation request: > InterpreterRuntime::frequency_counter_overflow > InterpreterRuntime::frequency_counter_overflow_inner > SimpleThresholdPolicy::event > AdvancedThresholdPolicy::method_invocation_event > [First compilation finishes here -> !CompileBroker::compilation_is_in_queue()] > AdvancedThresholdPolicy::call_event > AdvancedThresholdPolicy::common > [is_trivial() returns true because nmethod/mdo now available] > [next_level = CompLevel_Simple] > [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] > > [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log > [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log > [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log From aleksey.shipilev at oracle.com Wed Nov 5 21:26:36 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 06 Nov 2014 00:26:36 +0300 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) Message-ID: <545A960C.7010008@oracle.com> Hi, Long story short: current implementation of IndexSet, while smart, is too smart for its own good. Trying to be sparse, it loses locality. Trying to be smart about bit tricks, it loses the native word length. Because of that, sophisticated IndexSet does not yield a desired performance benefit on compilation-heavy workloads like Nashorn. Delegating the work to already existing BitMap both conserves the source code, and brings more performance. (C1 also uses the BitMap adapter like that for ValueMap-s). IndexSet is a major data structure for representing IFG in C2 Regalloc, and that is why improvements in IndexSet translate to faster register allocation, and faster C2 compiles. If you gut the IndexSet internals, and replace it with BitMap, the sample performance runs on Nashorn running Octane suite yield reliable improvements in compilation speed (average: 22.7 Kb/s -> 24.4 Kb/s), explained by the decrease in C2 regalloc time (average: 155s -> 132s). These improvements are in line with predicted improvements from a trial experiment of "coarsening" the IndexSet, see the relevant RFE: https://bugs.openjdk.java.net/browse/JDK-8059461 In other words, we can have a performance-improving change which also removes lots of code. Or, a cleanup change, which also improves performance. Here it is: http://cr.openjdk.java.net/~shade/8059461/webrev.02/ The patch is mostly proof-of-concept, and not ready for commit. Please let me know what you think about the approach. Code/style/idea suggestions are welcome. Brief summary of changes: - Almost all contents of IndexSet are purged, and delegated to BitMap - IndexSetIterator performance is important, and therefore the lookup table approach from IndexSetIterator was transplanted to new BitMapIterator. We might want to improve BitMap::get_next_one_offset with lookup tables as well, but that will contaminate the current experiment. - lrg_union was moved to appropriate place (why was it in IndexSet to begin with?) - some of IndexSet memory management / tracing functions were purged The testing so far was very light: - smoke tests with JPRT - Nashorn/Octane benchmarks Thanks, -Aleksey. From vladimir.kozlov at oracle.com Wed Nov 5 22:58:05 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 05 Nov 2014 14:58:05 -0800 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <545A960C.7010008@oracle.com> References: <545A960C.7010008@oracle.com> Message-ID: <545AAB7D.3010100@oracle.com> I think it is nice cleanup with performance benefits :) Aleksey, can you compare average memory consumed by IndexSet before and after? Why you need initialize_in_resource_arena()? by default BitMap() uses resource area: BitMap(idx_t size_in_bits, bool in_resource_area = true); Make lrg_union() PhaseConservativeCoalesce class's method. Thanks, Vladimir On 11/5/14 1:26 PM, Aleksey Shipilev wrote: > Hi, > > Long story short: current implementation of IndexSet, while smart, is > too smart for its own good. Trying to be sparse, it loses locality. > Trying to be smart about bit tricks, it loses the native word length. > > Because of that, sophisticated IndexSet does not yield a desired > performance benefit on compilation-heavy workloads like Nashorn. > Delegating the work to already existing BitMap both conserves the source > code, and brings more performance. (C1 also uses the BitMap adapter like > that for ValueMap-s). > > IndexSet is a major data structure for representing IFG in C2 Regalloc, > and that is why improvements in IndexSet translate to faster register > allocation, and faster C2 compiles. If you gut the IndexSet internals, > and replace it with BitMap, the sample performance runs on Nashorn > running Octane suite yield reliable improvements in compilation speed > (average: 22.7 Kb/s -> 24.4 Kb/s), explained by the decrease in C2 > regalloc time (average: 155s -> 132s). > > These improvements are in line with predicted improvements from a trial > experiment of "coarsening" the IndexSet, see the relevant RFE: > https://bugs.openjdk.java.net/browse/JDK-8059461 > > In other words, we can have a performance-improving change which also > removes lots of code. Or, a cleanup change, which also improves > performance. Here it is: > http://cr.openjdk.java.net/~shade/8059461/webrev.02/ > > The patch is mostly proof-of-concept, and not ready for commit. Please > let me know what you think about the approach. Code/style/idea > suggestions are welcome. > > Brief summary of changes: > - Almost all contents of IndexSet are purged, and delegated to BitMap > - IndexSetIterator performance is important, and therefore the lookup > table approach from IndexSetIterator was transplanted to new > BitMapIterator. We might want to improve BitMap::get_next_one_offset > with lookup tables as well, but that will contaminate the current > experiment. > - lrg_union was moved to appropriate place (why was it in IndexSet to > begin with?) > - some of IndexSet memory management / tracing functions were purged > > The testing so far was very light: > - smoke tests with JPRT > - Nashorn/Octane benchmarks > > Thanks, > -Aleksey. > From christian.thalinger at oracle.com Wed Nov 5 23:13:12 2014 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 5 Nov 2014 15:13:12 -0800 Subject: RFR (XS): 8062950: Bug in locking code when UseOptoBiasInlining is disabled: assert(dmw->is_neutral()) failed: invariant In-Reply-To: <7C9B87B351A4BA4AA9EC95BB418116566ACE597E@DEWDFEMB19C.global.corp.sap> References: <7C9B87B351A4BA4AA9EC95BB418116566ACE597E@DEWDFEMB19C.global.corp.sap> Message-ID: I?m not exactly sure who is our biased locking expert these days but I guess it?s someone from the runtime team. CC?ing them. > On Nov 5, 2014, at 7:38 AM, Doerr, Martin wrote: > > Hi, > > we found a bug in MacroAssembler::fast_lock on x86 which shows up when UseOptoBiasInlining is switched off. > The problem is that biased_locking_enter is used with swap_reg_contains_mark==true, which is no longer correct after biased_locking_enter was put in front of check for IsInflated. > > Please review > http://cr.openjdk.java.net/~goetz/webrevs/8062950-lockBug/webrev.00/ > > Best regards, > Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kozlov at oracle.com Wed Nov 5 23:30:54 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 05 Nov 2014 15:30:54 -0800 Subject: RFR (XS): 8062950: Bug in locking code when UseOptoBiasInlining is disabled: assert(dmw->is_neutral()) failed: invariant In-Reply-To: References: <7C9B87B351A4BA4AA9EC95BB418116566ACE597E@DEWDFEMB19C.global.corp.sap> Message-ID: <545AB32E.8070402@oracle.com> It is our (Compiler group) code. This problem was introduced with my changes for RTM locking. Martin your changes are good. But you cleanup a bit this code since we now never put markword to tmpReg before this call? Thanks, Vladimir On 11/5/14 3:13 PM, Christian Thalinger wrote: > I?m not exactly sure who is our biased locking expert these days but I guess it?s someone from the runtime team. CC?ing them. > >> On Nov 5, 2014, at 7:38 AM, Doerr, Martin wrote: >> >> Hi, >> >> we found a bug in MacroAssembler::fast_lock on x86 which shows up when UseOptoBiasInlining is switched off. >> The problem is that biased_locking_enter is used with swap_reg_contains_mark==true, which is no longer correct after biased_locking_enter was put in front of check for IsInflated. >> >> Please review >> http://cr.openjdk.java.net/~goetz/webrevs/8062950-lockBug/webrev.00/ >> >> Best regards, >> Martin > From david.holmes at oracle.com Wed Nov 5 23:44:57 2014 From: david.holmes at oracle.com (David Holmes) Date: Thu, 06 Nov 2014 09:44:57 +1000 Subject: RFR (XS): 8062950: Bug in locking code when UseOptoBiasInlining is disabled: assert(dmw->is_neutral()) failed: invariant In-Reply-To: References: <7C9B87B351A4BA4AA9EC95BB418116566ACE597E@DEWDFEMB19C.global.corp.sap> Message-ID: <545AB679.4040705@oracle.com> On 6/11/2014 9:13 AM, Christian Thalinger wrote: > I?m not exactly sure who is our biased locking expert these days but I guess it?s someone from the runtime team. CC?ing them. The fact I am responding does not imply I am, or consider myself, such an expert. ;-) I think we need to hear from Vladimir and Roland concerning the original fix for: 8033805: Move Fast_Lock/Fast_Unlock code from .ad files to macroassembler Looking at that changeset: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/5292439ef895 it seems that in x86_32.ad we had: if (UseBiasedLocking) { masm.biased_locking_enter(boxReg, objReg, tmpReg, scrReg, false, DONE_LABEL, NULL, _counters); } which passes "false", but in x86_64.ad we had: if (UseBiasedLocking && !UseOptoBiasInlining) { masm.biased_locking_enter(boxReg, objReg, tmpReg, scrReg, true, DONE_LABEL, NULL, _counters); masm.movptr(tmpReg, Address(objReg, 0)) ; // [FETCH] } which passes "true" because there was a prior load of the markword into tmpReg. The new code then has the 64-bit version: if (UseBiasedLocking && !UseOptoBiasInlining) { biased_locking_enter(boxReg, objReg, tmpReg, scrReg, true, DONE_LABEL, NULL, counters); } but not the prior load and hence is incorrect. So I concur with Martin's suggested fix. Cheers, David >> On Nov 5, 2014, at 7:38 AM, Doerr, Martin wrote: >> >> Hi, >> >> we found a bug in MacroAssembler::fast_lock on x86 which shows up when UseOptoBiasInlining is switched off. >> The problem is that biased_locking_enter is used with swap_reg_contains_mark==true, which is no longer correct after biased_locking_enter was put in front of check for IsInflated. Thanks, David >> >> Please review >> http://cr.openjdk.java.net/~goetz/webrevs/8062950-lockBug/webrev.00/ >> >> Best regards, >> Martin > From vladimir.kozlov at oracle.com Wed Nov 5 23:55:19 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 05 Nov 2014 15:55:19 -0800 Subject: RFR(M) : 8059624 : Test task: WhiteBox API for testing segmented codecache feature In-Reply-To: <545A4F63.6010008@oracle.com> References: <545A4F63.6010008@oracle.com> Message-ID: <545AB8E7.40400@oracle.com> Hi Igor, codeBlob.hpp Why you need?: + // placement + inline void* operator new(size_t s, void* p) throw() { return p; } compileBroker.cpp I think MonitorLockerEx locker() should be before the loop. Then you may not need to get lock in WB_UnlockCompilation. whitebox.cpp allocateCodeHeap() - what if CodeCache::allocate() failed to allocate? In WB_GetCodeHeapEntries() checks that blobs array is not empty after the loop which add entries. You need to prevent sweeping when you processing codecache blobs (getCodeHeapEntries). Otherwise pointers from getCodeBlobs() may point to dead or overwritten space, consider -Xcomp -XX:+DeoptimizeALot Thanks, Vladimir On 11/5/14 8:25 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8059624/webrev.00/ > 660 lines changed: 624 ins; 2 del; 34 mod; > > Hi all, > > please review the patch which adds new WhiteBox methods needed for > better testing SegmentedCodeCache: > - perform allocation in code cache > - force code cache sweep > - lock/unlock compilation > - get a segment id for nmethod. > besides these methods, the patch also adds a method to get all entries > in code heap. > > changes in product code: > - monitor 'Compilation_lock' was added to implement compilation > locking/unlocking > - align_code_offset function was made a static member of CodeBlob > - WhiteBox was made a friend of several classes > > testing: jprt, new added tests > jbs: https://bugs.openjdk.java.net/browse/JDK-8059624 > From vladimir.kozlov at oracle.com Thu Nov 6 00:56:33 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 05 Nov 2014 16:56:33 -0800 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545A1445.2050200@oracle.com> References: <545A1445.2050200@oracle.com> Message-ID: <545AC741.2040803@oracle.com> So the fix is add -XX:-TieredCompilation. The rest is fix problems found due to these failure. May be we should add a new test (other RFE) which would test compilation of trivial methods in Tiered mode. Current failure shows that we don't test it. I am not comfortable to rely on data which is set only by C1. This information is accessed in non Tiered mode too. Why not use method->has_loops(), for example? Thanks, Vladimir On 11/5/14 4:12 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 > Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ > > Problem: > The test compiles and deoptimizes a method m multiple times. After each > deoptimization the test checks if m was compiled with C2 and if so increments a > counter. The test assumes that m is compilable until the counter reaches the > 'PerMethodRecompilationCutoff'. This does not hold in the following case: > - The first compilation request for m is added to the compile queue. Although we > profiled m and the MDO suggests that m is trivial, we do not use this > information because due to deoptimization there is no corresponding nmethod [1]. > We compile at level 4 and execution continues in the interpreter [2]. > - A second compilation request for m is issued and while it is being processed > the first compilation request finishes [3]. Because a nmethod now exists, we use > the MDO and decide to compile at level 1 since m is trivial. > - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. > > After the following deoptimization the counter is not incremented because the > nmethod was compiled at level 1. As a result m is set to not compilable before > the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. > > Problems with the current implementation: > (1) We should always use the MDO if it is valid, even if there is no > corresponding nmethod. Otherwise we compile trivial methods at level 4 or start > profiling again. > (2) If there is a nmethod the MDO may still be invalid: We can compile a method > immediately at C2, deoptimize, and the MDO is uninitialized. In this case we > wrongly assume that the method is trivial and re-compile it with C1. > (3) To avoid a level 2 or 3 compilation and profiling we should mark simple > constant getter methods, like those used by the test, as trivial. > (4) We should add verification code to avoid/catch such bad tiered level > transitions in the future. > > Solution: > To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines > if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 > compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the > MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we > always compile the (trivial) methods at level 1 and the test fails. > > To fix (3) I added the method 'Method::is_constant_getter()' that determines if > a method consists of a constant push and a return statement only. If so, we > treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. > > I filed JDK-8062913 for (4) because the verification code I added still fails > with the current implementation due to 2->2 and 3->2 transitions. > > Testing: > - JPRT including failing whitebox test > > Thanks, > Tobias > > > [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' > [2] First compilation request: > InterpreterRuntime::frequency_counter_overflow > InterpreterRuntime::frequency_counter_overflow_inner > SimpleThresholdPolicy::event > AdvancedThresholdPolicy::method_invocation_event > AdvancedThresholdPolicy::call_event > AdvancedThresholdPolicy::common > [is_trivial() returns false because no nmethod available] > [next_level = CompLevel_Full_Optimization] > SimpleThresholdPolicy::compile > SimpleThresholdPolicy::submit_compile > CompileBroker::compile_method > CompileBroker::compile_method_base > CompileBroker::create_compile_task > [continue execution in interpreter] > > [3] Second compilation request: > InterpreterRuntime::frequency_counter_overflow > InterpreterRuntime::frequency_counter_overflow_inner > SimpleThresholdPolicy::event > AdvancedThresholdPolicy::method_invocation_event > [First compilation finishes here -> !CompileBroker::compilation_is_in_queue()] > AdvancedThresholdPolicy::call_event > AdvancedThresholdPolicy::common > [is_trivial() returns true because nmethod/mdo now available] > [next_level = CompLevel_Simple] > [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] > > [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log > [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log > [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log > From zoltan.majo at oracle.com Thu Nov 6 08:44:51 2014 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Thu, 06 Nov 2014 09:44:51 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <545A5BD5.60000@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> <5458CED2.6030203@oracle.com> <54591D84.30709@oracle.com> <545A276A.9020509@oracle.com> <545A5BD5.60000@oracle.com> Message-ID: <545B3503.2030002@oracle.com> Thank you, Vladimir! On 11/05/2014 06:18 PM, Vladimir Kozlov wrote: > This looks good. Don't forget to add label noreg-sqe since we have > tests already. I added the label. Best regards, Zoltan > > Thanks, > Vladimir > > On 11/5/14 5:34 AM, Zolt?n Maj? wrote: >> Hi Vladimir, >> >> >> thank you for the feedback. >> >> On 11/04/2014 07:40 PM, Vladimir Kozlov wrote: >>> On 11/4/14 5:04 AM, Zolt?n Maj? wrote: >>>> Hi Roland, Vladimir, and Albert! >>>> >>>> >>>> thank you for your feedback and for your suggestions! Please see >>>> comments on specific issues inline. >>>> >>>> On 11/04/2014 09:38 AM, Albert Noll wrote: >>>>> Hi, >>>>> >>>>> please see comments inline. >>>>> On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >>>>>> I agree with additional check tak != TypeKlassPtr::OBJECT >>>>>> (solution 2). >>>>>> >>>>>> Instead of creating new LoadKlassNode::make() method you can set >>>>>> control in parseHelper.cpp since it is the only >>>>>> place which set it: >>>>>> >>>>>> Node* a_e_klass = LoadKlassNode::make(_gvn, >>>>>> immutable_memory(), p2, tak); >>>>>> if (always_see_exact_class) { >>>>>> a_e_klass->init_req(MemNode::Control, control()); >>>>>> } >>>>>> a_e_klass = _gvn.transform(a_e_klass); >>>>>> >>>>> An alternative solution is to make a default parameter for >>>>> control. This way we can set the control in make() without >>>>> having to add a new make() function. >>>> >>>> I took Albert's alternative solution. Now we also set the default >>>> value for parameter TypeKlassPtr* tk in >>>> parseHelper.cpp at line 230. >>> >>> As I replied to Albert, I think it should be explicit parameter and >>> not default. >>> Yes, you would need to modify more files but it is better to be >>> explicit I think. >> >> OK, I see. I updated the code so that we have one >> LoadKlassNode::make() method, with the control node as second >> parameter. In the new version the control parameter is explicit and >> we pass in NULL explicitly at all call sites where >> control is not needed. >> >>> >>>> >>>>>> Also an other reason to have control edge in this place is the >>>>>> address of this klass load is based on ConP constant. >>>>>> In a normal case after class check you should have CheckCastPP >>>>>> attached to control and used as address's base. >>>> >>>> I see, Vladimir. Just a related question: If the address of the >>>> klass load were *not* based on a ConP constant, would it >>>> be better (or required) to check with a CheckCastPP the value that >>>> was loaded? >>> >>> Yes, you would use CheckCastPP to cast type to exact class you >>> checked for and pin it to 'true' projection of the >>> check. As we do for in gen_checkcast() or other places (speculative >>> types). >>> >>> if (a.class == exact_class) >>> (exact_class)a >> >> Thank you for the explanation. >> >>> >>> In this case you don't need control on following load. >>> >>>> >>>>>> >>>>>> And agree with Roland's suggestion about Op_LoadKlass check. >>>>>> >>>>> What you are suggesting is to make a super class (LoadNode) aware >>>>> of its subclasses (LoadPNode and LoadKlassNode). >>>>> From a software engineering point of view, we should try to avoid >>>>> doing that. Another solution would be to add a >>>>> method (e.g., can_remove_control()) to LoadNode, LoadPNode, and >>>>> LoadKlassNode. The function returns 'false' for >>>>> LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What >>>>> do you think? >>>> >>>> I think Albert's suggestion has the advantage that if we will ever >>>> decide to inherit from LoadKlass, we won't have to >>>> worry about modifying its superclass. So I think it will be good if >>>> we go with it. >>> >>> I prefer Albert's name can_remove_control() >> >> OK, I changed the method's name. >> >> Here is the new webrev: >> http://cr.openjdk.java.net/~zmajo/8057622/webrev.03/ >> >> All JPRT tests and also the failing test cases pass with the new >> version. >> >> Thank you and best regards, >> >> >> Zoltan >> >>> >>> Thanks, >>> Vladimir >>> >>>> >>>> Here is the new webrev: >>>> http://cr.openjdk.java.net/~zmajo/8057622/webrev.02/ >>>> >>>> JPRT and also the failing test cases pass. >>>> >>>> >>>> Thank you and best regards, >>>> >>>> >>>> Zoltan >>>> >>>>> >>>>> Best, >>>>> Albert >>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> please review the following patch. >>>>>>> >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>>>>>> >>>>>>> >>>>>>> Problem: >>>>>>> >>>>>>> >>>>>>> We have five tests that fail with SIGSEGV in our nightlies: >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>>>>>> >>>>>>> All tests fail when executing the C2-compiled version of >>>>>>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>>>>>> >>>>>>> >>>>>>> 489: private void inflateSpine() { >>>>>>> 490: if (spine == null) { >>>>>>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>>>>>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>>>>>> 493: spine[0] = curChunk; >>>>>>> 494: } >>>>>>> 495: } >>>>>>> >>>>>>> >>>>>>> The failure is due to the 'aastore' at line 493; the failure is >>>>>>> caused >>>>>>> by the monomorphic array check optimization >>>>>>> (-XX:+MonomorphicArrayCheck, enabled by default). >>>>>>> >>>>>>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>>>>>> profiling information) that inflateSpine() has two hot receiver >>>>>>> types, >>>>>>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>>>>>> >>>>>>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>>>>>> >>>>>>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>>>>>> double[][]. >>>>>>> >>>>>>> Please consider the following pseudo Java code that illustrates >>>>>>> how C2 >>>>>>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining >>>>>>> based on >>>>>>> the two hot receiver types SpinedBuffer.OfInt and >>>>>>> SpinedBuffer.OfDouble: >>>>>>> >>>>>>> >>>>>>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>>>>>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>>>>>> spine[0] = curChunk; >>>>>>> } >>>>>>> >>>>>>> >>>>>>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>>>>>> of 'spine' *at runtime*. The check looks something like: >>>>>>> >>>>>>> >>>>>>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>>>>>> throw new ArrayStoreException(); >>>>>>> } >>>>>>> >>>>>>> >>>>>>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>>>>>> check like this: >>>>>>> >>>>>>> >>>>>>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>>>>>> throw new ArrayStoreException(); >>>>>>> } >>>>>>> >>>>>>> >>>>>>> where TYPE is the type of the 'spine' variable, as it is >>>>>>> determined by >>>>>>> C2 *at compile time*. The optimization treats TYPE as a constant >>>>>>> and >>>>>>> thus saves a load from memory + an offset calculation (I think). >>>>>>> >>>>>>> The problem is that due to 'spine' being either of type int[][] >>>>>>> or of >>>>>>> type double[][], C2 determines that TYPE==java/lang/Object. >>>>>>> >>>>>>> As a result, when the inflateSpine() method is executed, it >>>>>>> first loads the >>>>>>> Klass* corresponding to java/lang/Object (TYPE). Let us call >>>>>>> this Klass* >>>>>>> array_klass. Then, the method obtains a Klass* from >>>>>>> array_klass->_element_klass. >>>>>>> Then, the method reads from offset 20 from >>>>>>> array_klass->_element_klass, which >>>>>>> results in a SIGSEGV. >>>>>>> >>>>>>> The reason for the SIGSEGV is that the Klass* array_klass is of >>>>>>> type >>>>>>> java/lang/Object and is therefore represented as an >>>>>>> 'InstanceKlass'. >>>>>>> But _element_klass is defined only in objects of type >>>>>>> 'ObjArrayKlass'. >>>>>>> The compiler reads array_klass->_element_klass because it >>>>>>> expects the >>>>>>> destination of an 'aastore' to be an array, which is not true if >>>>>>> TYPE==java/lang/Object. >>>>>>> >>>>>>> >>>>>>> Solution: >>>>>>> >>>>>>> >>>>>>> The compiler already checks if the compile-time type of the >>>>>>> array (TYPE) is >>>>>>> the same as the array's type at runtime. If that check fails, >>>>>>> the method >>>>>>> branches to an uncommon trap. In our case the SIGSEGV happens >>>>>>> because >>>>>>> the load of array_klass->_element_klass "floats" above the check. >>>>>>> >>>>>>> I propose two solutions: >>>>>>> >>>>>>> Solution 1. Add a control edge between the IfTrue branch of the >>>>>>> check and >>>>>>> the load of _array_klass->_element_klass. That prohibits the >>>>>>> load of >>>>>>> _array_klass->element_klass floating above the check. >>>>>>> >>>>>>> Solution 2. In addition to the changes in Solution 1, the >>>>>>> compiler checks if >>>>>>> the compile-time type of the array (TYPE) is java/lang/Object. If >>>>>>> TYPE==java/lang/Object, the compiler should not assume >>>>>>> array_type to be >>>>>>> TYPE, but should determine array type at runtime instead. >>>>>>> >>>>>>> The reason is for Solution 2 is: We can't do an array store into >>>>>>> a java/lang/Object. >>>>>>> So, if we see TYPE==java/lang/Object at compile time, it is >>>>>>> pretty sure that the >>>>>>> method will deoptimize at runtime. Instead of assuming TYPE to >>>>>>> be java/lang/Object, >>>>>>> we read at runtime the type of the array store's destination. >>>>>>> That still gives us some >>>>>>> chance that the compiled version of the method will successfully >>>>>>> execute. The >>>>>>> additional check is in parseHelper.cpp in >>>>>>> Parse::array_store_check() on line 171. >>>>>>> >>>>>>> >>>>>>> Webrev: >>>>>>> >>>>>>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>>>>>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>>>>>> >>>>>>> >>>>>>> Testing (for both solutions): >>>>>>> >>>>>>> JPRT, manual testing with failing test cases. >>>>>>> >>>>>>> >>>>>>> Please note that both solutions are somewhat "hacky", because >>>>>>> that allowed me to >>>>>>> produce a solution within reasonable time and with reasonable >>>>>>> effort. Any suggestions >>>>>>> or feedback is welcome. >>>>>>> >>>>>>> >>>>>>> Thank you and best regards, >>>>>>> >>>>>>> >>>>>>> Zoltan >>>>>>> >>>>> >>>> >> From zoltan.majo at oracle.com Thu Nov 6 08:45:34 2014 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Thu, 06 Nov 2014 09:45:34 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <545A5BD5.60000@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> <5458CED2.6030203@oracle.com> <54591D84.30709@oracle.com> <545A276A.9020509@oracle.com> <545A5BD5.60000@oracle.com> Message-ID: <545B352E.3030403@oracle.com> Thank you, Roland, Vladimir, and Albert, for the reviews! Zoltan On 11/05/2014 06:18 PM, Vladimir Kozlov wrote: > This looks good. Don't forget to add label noreg-sqe since we have > tests already. > > Thanks, > Vladimir > > On 11/5/14 5:34 AM, Zolt?n Maj? wrote: >> Hi Vladimir, >> >> >> thank you for the feedback. >> >> On 11/04/2014 07:40 PM, Vladimir Kozlov wrote: >>> On 11/4/14 5:04 AM, Zolt?n Maj? wrote: >>>> Hi Roland, Vladimir, and Albert! >>>> >>>> >>>> thank you for your feedback and for your suggestions! Please see >>>> comments on specific issues inline. >>>> >>>> On 11/04/2014 09:38 AM, Albert Noll wrote: >>>>> Hi, >>>>> >>>>> please see comments inline. >>>>> On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >>>>>> I agree with additional check tak != TypeKlassPtr::OBJECT >>>>>> (solution 2). >>>>>> >>>>>> Instead of creating new LoadKlassNode::make() method you can set >>>>>> control in parseHelper.cpp since it is the only >>>>>> place which set it: >>>>>> >>>>>> Node* a_e_klass = LoadKlassNode::make(_gvn, >>>>>> immutable_memory(), p2, tak); >>>>>> if (always_see_exact_class) { >>>>>> a_e_klass->init_req(MemNode::Control, control()); >>>>>> } >>>>>> a_e_klass = _gvn.transform(a_e_klass); >>>>>> >>>>> An alternative solution is to make a default parameter for >>>>> control. This way we can set the control in make() without >>>>> having to add a new make() function. >>>> >>>> I took Albert's alternative solution. Now we also set the default >>>> value for parameter TypeKlassPtr* tk in >>>> parseHelper.cpp at line 230. >>> >>> As I replied to Albert, I think it should be explicit parameter and >>> not default. >>> Yes, you would need to modify more files but it is better to be >>> explicit I think. >> >> OK, I see. I updated the code so that we have one >> LoadKlassNode::make() method, with the control node as second >> parameter. In the new version the control parameter is explicit and >> we pass in NULL explicitly at all call sites where >> control is not needed. >> >>> >>>> >>>>>> Also an other reason to have control edge in this place is the >>>>>> address of this klass load is based on ConP constant. >>>>>> In a normal case after class check you should have CheckCastPP >>>>>> attached to control and used as address's base. >>>> >>>> I see, Vladimir. Just a related question: If the address of the >>>> klass load were *not* based on a ConP constant, would it >>>> be better (or required) to check with a CheckCastPP the value that >>>> was loaded? >>> >>> Yes, you would use CheckCastPP to cast type to exact class you >>> checked for and pin it to 'true' projection of the >>> check. As we do for in gen_checkcast() or other places (speculative >>> types). >>> >>> if (a.class == exact_class) >>> (exact_class)a >> >> Thank you for the explanation. >> >>> >>> In this case you don't need control on following load. >>> >>>> >>>>>> >>>>>> And agree with Roland's suggestion about Op_LoadKlass check. >>>>>> >>>>> What you are suggesting is to make a super class (LoadNode) aware >>>>> of its subclasses (LoadPNode and LoadKlassNode). >>>>> From a software engineering point of view, we should try to avoid >>>>> doing that. Another solution would be to add a >>>>> method (e.g., can_remove_control()) to LoadNode, LoadPNode, and >>>>> LoadKlassNode. The function returns 'false' for >>>>> LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What >>>>> do you think? >>>> >>>> I think Albert's suggestion has the advantage that if we will ever >>>> decide to inherit from LoadKlass, we won't have to >>>> worry about modifying its superclass. So I think it will be good if >>>> we go with it. >>> >>> I prefer Albert's name can_remove_control() >> >> OK, I changed the method's name. >> >> Here is the new webrev: >> http://cr.openjdk.java.net/~zmajo/8057622/webrev.03/ >> >> All JPRT tests and also the failing test cases pass with the new >> version. >> >> Thank you and best regards, >> >> >> Zoltan >> >>> >>> Thanks, >>> Vladimir >>> >>>> >>>> Here is the new webrev: >>>> http://cr.openjdk.java.net/~zmajo/8057622/webrev.02/ >>>> >>>> JPRT and also the failing test cases pass. >>>> >>>> >>>> Thank you and best regards, >>>> >>>> >>>> Zoltan >>>> >>>>> >>>>> Best, >>>>> Albert >>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> please review the following patch. >>>>>>> >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>>>>>> >>>>>>> >>>>>>> Problem: >>>>>>> >>>>>>> >>>>>>> We have five tests that fail with SIGSEGV in our nightlies: >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>>>>>> >>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>>>>>> >>>>>>> All tests fail when executing the C2-compiled version of >>>>>>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>>>>>> >>>>>>> >>>>>>> 489: private void inflateSpine() { >>>>>>> 490: if (spine == null) { >>>>>>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>>>>>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>>>>>> 493: spine[0] = curChunk; >>>>>>> 494: } >>>>>>> 495: } >>>>>>> >>>>>>> >>>>>>> The failure is due to the 'aastore' at line 493; the failure is >>>>>>> caused >>>>>>> by the monomorphic array check optimization >>>>>>> (-XX:+MonomorphicArrayCheck, enabled by default). >>>>>>> >>>>>>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>>>>>> profiling information) that inflateSpine() has two hot receiver >>>>>>> types, >>>>>>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>>>>>> >>>>>>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>>>>>> >>>>>>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>>>>>> double[][]. >>>>>>> >>>>>>> Please consider the following pseudo Java code that illustrates >>>>>>> how C2 >>>>>>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining >>>>>>> based on >>>>>>> the two hot receiver types SpinedBuffer.OfInt and >>>>>>> SpinedBuffer.OfDouble: >>>>>>> >>>>>>> >>>>>>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>>>>>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>>>>>> spine[0] = curChunk; >>>>>>> } >>>>>>> >>>>>>> >>>>>>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>>>>>> of 'spine' *at runtime*. The check looks something like: >>>>>>> >>>>>>> >>>>>>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>>>>>> throw new ArrayStoreException(); >>>>>>> } >>>>>>> >>>>>>> >>>>>>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>>>>>> check like this: >>>>>>> >>>>>>> >>>>>>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>>>>>> throw new ArrayStoreException(); >>>>>>> } >>>>>>> >>>>>>> >>>>>>> where TYPE is the type of the 'spine' variable, as it is >>>>>>> determined by >>>>>>> C2 *at compile time*. The optimization treats TYPE as a constant >>>>>>> and >>>>>>> thus saves a load from memory + an offset calculation (I think). >>>>>>> >>>>>>> The problem is that due to 'spine' being either of type int[][] >>>>>>> or of >>>>>>> type double[][], C2 determines that TYPE==java/lang/Object. >>>>>>> >>>>>>> As a result, when the inflateSpine() method is executed, it >>>>>>> first loads the >>>>>>> Klass* corresponding to java/lang/Object (TYPE). Let us call >>>>>>> this Klass* >>>>>>> array_klass. Then, the method obtains a Klass* from >>>>>>> array_klass->_element_klass. >>>>>>> Then, the method reads from offset 20 from >>>>>>> array_klass->_element_klass, which >>>>>>> results in a SIGSEGV. >>>>>>> >>>>>>> The reason for the SIGSEGV is that the Klass* array_klass is of >>>>>>> type >>>>>>> java/lang/Object and is therefore represented as an >>>>>>> 'InstanceKlass'. >>>>>>> But _element_klass is defined only in objects of type >>>>>>> 'ObjArrayKlass'. >>>>>>> The compiler reads array_klass->_element_klass because it >>>>>>> expects the >>>>>>> destination of an 'aastore' to be an array, which is not true if >>>>>>> TYPE==java/lang/Object. >>>>>>> >>>>>>> >>>>>>> Solution: >>>>>>> >>>>>>> >>>>>>> The compiler already checks if the compile-time type of the >>>>>>> array (TYPE) is >>>>>>> the same as the array's type at runtime. If that check fails, >>>>>>> the method >>>>>>> branches to an uncommon trap. In our case the SIGSEGV happens >>>>>>> because >>>>>>> the load of array_klass->_element_klass "floats" above the check. >>>>>>> >>>>>>> I propose two solutions: >>>>>>> >>>>>>> Solution 1. Add a control edge between the IfTrue branch of the >>>>>>> check and >>>>>>> the load of _array_klass->_element_klass. That prohibits the >>>>>>> load of >>>>>>> _array_klass->element_klass floating above the check. >>>>>>> >>>>>>> Solution 2. In addition to the changes in Solution 1, the >>>>>>> compiler checks if >>>>>>> the compile-time type of the array (TYPE) is java/lang/Object. If >>>>>>> TYPE==java/lang/Object, the compiler should not assume >>>>>>> array_type to be >>>>>>> TYPE, but should determine array type at runtime instead. >>>>>>> >>>>>>> The reason is for Solution 2 is: We can't do an array store into >>>>>>> a java/lang/Object. >>>>>>> So, if we see TYPE==java/lang/Object at compile time, it is >>>>>>> pretty sure that the >>>>>>> method will deoptimize at runtime. Instead of assuming TYPE to >>>>>>> be java/lang/Object, >>>>>>> we read at runtime the type of the array store's destination. >>>>>>> That still gives us some >>>>>>> chance that the compiled version of the method will successfully >>>>>>> execute. The >>>>>>> additional check is in parseHelper.cpp in >>>>>>> Parse::array_store_check() on line 171. >>>>>>> >>>>>>> >>>>>>> Webrev: >>>>>>> >>>>>>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>>>>>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>>>>>> >>>>>>> >>>>>>> Testing (for both solutions): >>>>>>> >>>>>>> JPRT, manual testing with failing test cases. >>>>>>> >>>>>>> >>>>>>> Please note that both solutions are somewhat "hacky", because >>>>>>> that allowed me to >>>>>>> produce a solution within reasonable time and with reasonable >>>>>>> effort. Any suggestions >>>>>>> or feedback is welcome. >>>>>>> >>>>>>> >>>>>>> Thank you and best regards, >>>>>>> >>>>>>> >>>>>>> Zoltan >>>>>>> >>>>> >>>> >> From tobias.hartmann at oracle.com Thu Nov 6 09:11:04 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 06 Nov 2014 10:11:04 +0100 Subject: RFR(M) : 8059624 : Test task: WhiteBox API for testing segmented codecache feature In-Reply-To: <545A4F63.6010008@oracle.com> References: <545A4F63.6010008@oracle.com> Message-ID: <545B3B28.5040604@oracle.com> Hi Igor, looks good (not a reviewer). In addition to Vladimir's comments: 'WhiteBox::allocateCodeHeap' should be 'allocate_code_heap'. I think the name is a bit misleading since we don't allocate a CodeHeap but a CodeBlob in a CodeHeap. Maybe use 'code_heap_allocate' or 'allocate_code_blob'. Same for 'WB_AllocateCodeHeap'. In 'BlobType::getAvailable' you always return the NonProfiledCodeHeap if CodeCacheSegmentation is enabled. But if we only use the interpreter (-Xint), we don't have any method code heaps. Also with 'TieredStopAtLevel <= CompLevel_simple' we don't need the ProfiledCodeHeap (see 'CodeCache::heap_available'). The corresponding tests will hit the "heap is null" assert in 'CodeCache::allocate'. Best, Tobias On 05.11.2014 17:25, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8059624/webrev.00/ > 660 lines changed: 624 ins; 2 del; 34 mod; > > Hi all, > > please review the patch which adds new WhiteBox methods needed for better > testing SegmentedCodeCache: > - perform allocation in code cache > - force code cache sweep > - lock/unlock compilation > - get a segment id for nmethod. > besides these methods, the patch also adds a method to get all entries in code > heap. > > changes in product code: > - monitor 'Compilation_lock' was added to implement compilation locking/unlocking > - align_code_offset function was made a static member of CodeBlob > - WhiteBox was made a friend of several classes > > testing: jprt, new added tests > jbs: https://bugs.openjdk.java.net/browse/JDK-8059624 > From tobias.hartmann at oracle.com Thu Nov 6 09:12:37 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 06 Nov 2014 10:12:37 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: References: <545A1445.2050200@oracle.com> Message-ID: <545B3B85.70309@oracle.com> Thanks, Igor. Best, Tobias On 05.11.2014 21:19, Igor Veresov wrote: > Looks good to me. > > igor > > On Nov 5, 2014, at 2:12 AM, Tobias Hartmann wrote: > >> Hi, >> >> please review the following patch. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >> >> Problem: >> The test compiles and deoptimizes a method m multiple times. After each >> deoptimization the test checks if m was compiled with C2 and if so increments a >> counter. The test assumes that m is compilable until the counter reaches the >> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >> - The first compilation request for m is added to the compile queue. Although we >> profiled m and the MDO suggests that m is trivial, we do not use this >> information because due to deoptimization there is no corresponding nmethod [1]. >> We compile at level 4 and execution continues in the interpreter [2]. >> - A second compilation request for m is issued and while it is being processed >> the first compilation request finishes [3]. Because a nmethod now exists, we use >> the MDO and decide to compile at level 1 since m is trivial. >> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >> >> After the following deoptimization the counter is not incremented because the >> nmethod was compiled at level 1. As a result m is set to not compilable before >> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >> >> Problems with the current implementation: >> (1) We should always use the MDO if it is valid, even if there is no >> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >> profiling again. >> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >> wrongly assume that the method is trivial and re-compile it with C1. >> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >> constant getter methods, like those used by the test, as trivial. >> (4) We should add verification code to avoid/catch such bad tiered level >> transitions in the future. >> >> Solution: >> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >> always compile the (trivial) methods at level 1 and the test fails. >> >> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >> a method consists of a constant push and a return statement only. If so, we >> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >> >> I filed JDK-8062913 for (4) because the verification code I added still fails >> with the current implementation due to 2->2 and 3->2 transitions. >> >> Testing: >> - JPRT including failing whitebox test >> >> Thanks, >> Tobias >> >> >> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >> [2] First compilation request: >> InterpreterRuntime::frequency_counter_overflow >> InterpreterRuntime::frequency_counter_overflow_inner >> SimpleThresholdPolicy::event >> AdvancedThresholdPolicy::method_invocation_event >> AdvancedThresholdPolicy::call_event >> AdvancedThresholdPolicy::common >> [is_trivial() returns false because no nmethod available] >> [next_level = CompLevel_Full_Optimization] >> SimpleThresholdPolicy::compile >> SimpleThresholdPolicy::submit_compile >> CompileBroker::compile_method >> CompileBroker::compile_method_base >> CompileBroker::create_compile_task >> [continue execution in interpreter] >> >> [3] Second compilation request: >> InterpreterRuntime::frequency_counter_overflow >> InterpreterRuntime::frequency_counter_overflow_inner >> SimpleThresholdPolicy::event >> AdvancedThresholdPolicy::method_invocation_event >> [First compilation finishes here -> !CompileBroker::compilation_is_in_queue()] >> AdvancedThresholdPolicy::call_event >> AdvancedThresholdPolicy::common >> [is_trivial() returns true because nmethod/mdo now available] >> [next_level = CompLevel_Simple] >> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >> >> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log > From albert.noll at oracle.com Thu Nov 6 09:36:52 2014 From: albert.noll at oracle.com (Albert Noll) Date: Thu, 06 Nov 2014 10:36:52 +0100 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <545AAB7D.3010100@oracle.com> References: <545A960C.7010008@oracle.com> <545AAB7D.3010100@oracle.com> Message-ID: <545B4134.4040103@oracle.com> It would be good to have data from different applications / architectures. Does anyone know more about the motivation for IndexSet, i.e., why it was introduced in the first place? Thanks, Albert On 11/05/2014 11:58 PM, Vladimir Kozlov wrote: > I think it is nice cleanup with performance benefits :) > > Aleksey, can you compare average memory consumed by IndexSet before > and after? > > Why you need initialize_in_resource_arena()? by default BitMap() uses > resource area: > > BitMap(idx_t size_in_bits, bool in_resource_area = true); > > Make lrg_union() PhaseConservativeCoalesce class's method. > > Thanks, > Vladimir > > On 11/5/14 1:26 PM, Aleksey Shipilev wrote: >> Hi, >> >> Long story short: current implementation of IndexSet, while smart, is >> too smart for its own good. Trying to be sparse, it loses locality. >> Trying to be smart about bit tricks, it loses the native word length. >> >> Because of that, sophisticated IndexSet does not yield a desired >> performance benefit on compilation-heavy workloads like Nashorn. >> Delegating the work to already existing BitMap both conserves the source >> code, and brings more performance. (C1 also uses the BitMap adapter like >> that for ValueMap-s). >> >> IndexSet is a major data structure for representing IFG in C2 Regalloc, >> and that is why improvements in IndexSet translate to faster register >> allocation, and faster C2 compiles. If you gut the IndexSet internals, >> and replace it with BitMap, the sample performance runs on Nashorn >> running Octane suite yield reliable improvements in compilation speed >> (average: 22.7 Kb/s -> 24.4 Kb/s), explained by the decrease in C2 >> regalloc time (average: 155s -> 132s). >> >> These improvements are in line with predicted improvements from a trial >> experiment of "coarsening" the IndexSet, see the relevant RFE: >> https://bugs.openjdk.java.net/browse/JDK-8059461 >> >> In other words, we can have a performance-improving change which also >> removes lots of code. Or, a cleanup change, which also improves >> performance. Here it is: >> http://cr.openjdk.java.net/~shade/8059461/webrev.02/ >> >> The patch is mostly proof-of-concept, and not ready for commit. Please >> let me know what you think about the approach. Code/style/idea >> suggestions are welcome. >> >> Brief summary of changes: >> - Almost all contents of IndexSet are purged, and delegated to BitMap >> - IndexSetIterator performance is important, and therefore the lookup >> table approach from IndexSetIterator was transplanted to new >> BitMapIterator. We might want to improve BitMap::get_next_one_offset >> with lookup tables as well, but that will contaminate the current >> experiment. >> - lrg_union was moved to appropriate place (why was it in IndexSet to >> begin with?) >> - some of IndexSet memory management / tracing functions were purged >> >> The testing so far was very light: >> - smoke tests with JPRT >> - Nashorn/Octane benchmarks >> >> Thanks, >> -Aleksey. >> From tatiana.pivovarova at oracle.com Thu Nov 6 09:41:01 2014 From: tatiana.pivovarova at oracle.com (Tatiana Pivovarova) Date: Thu, 06 Nov 2014 12:41:01 +0300 Subject: [9] RFR(M): 8062011: JT_HS/compiler/7068051 uses jre/lib/javaws.jar Message-ID: <545B422D.6070207@oracle.com> Hi all, please review the patch Bug: https://bugs.openjdk.java.net/browse/JDK-8062011 Webrev: http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062011/webrev.00/ Problem: test/compiler/7068051 uses jre/lib/javaws.jar which is removed in jigsaw M2 Solution: We create .jar file by ourselves. Testing: Manual Thanks, Tatiana From roland.westrelin at oracle.com Thu Nov 6 10:05:06 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Thu, 6 Nov 2014 11:05:06 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <545B352E.3030403@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> <5458CED2.6030203@oracle.com> <54591D84.30709@oracle.com> <545A276A.9020509@oracle.com> <545A5BD5.60000@oracle.com> <545B352E.3030403@oracle.com> Message-ID: <16C2529F-2925-423C-96FD-8C25EBF7347B@oracle.com> For the record, final webrev looks good to me. Roland. > On Nov 6, 2014, at 9:45 AM, Zolt?n Maj? wrote: > > Thank you, Roland, Vladimir, and Albert, for the reviews! > > > Zoltan > > On 11/05/2014 06:18 PM, Vladimir Kozlov wrote: >> This looks good. Don't forget to add label noreg-sqe since we have tests already. >> >> Thanks, >> Vladimir >> >> On 11/5/14 5:34 AM, Zolt?n Maj? wrote: >>> Hi Vladimir, >>> >>> >>> thank you for the feedback. >>> >>> On 11/04/2014 07:40 PM, Vladimir Kozlov wrote: >>>> On 11/4/14 5:04 AM, Zolt?n Maj? wrote: >>>>> Hi Roland, Vladimir, and Albert! >>>>> >>>>> >>>>> thank you for your feedback and for your suggestions! Please see comments on specific issues inline. >>>>> >>>>> On 11/04/2014 09:38 AM, Albert Noll wrote: >>>>>> Hi, >>>>>> >>>>>> please see comments inline. >>>>>> On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >>>>>>> I agree with additional check tak != TypeKlassPtr::OBJECT (solution 2). >>>>>>> >>>>>>> Instead of creating new LoadKlassNode::make() method you can set control in parseHelper.cpp since it is the only >>>>>>> place which set it: >>>>>>> >>>>>>> Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), p2, tak); >>>>>>> if (always_see_exact_class) { >>>>>>> a_e_klass->init_req(MemNode::Control, control()); >>>>>>> } >>>>>>> a_e_klass = _gvn.transform(a_e_klass); >>>>>>> >>>>>> An alternative solution is to make a default parameter for control. This way we can set the control in make() without >>>>>> having to add a new make() function. >>>>> >>>>> I took Albert's alternative solution. Now we also set the default value for parameter TypeKlassPtr* tk in >>>>> parseHelper.cpp at line 230. >>>> >>>> As I replied to Albert, I think it should be explicit parameter and not default. >>>> Yes, you would need to modify more files but it is better to be explicit I think. >>> >>> OK, I see. I updated the code so that we have one LoadKlassNode::make() method, with the control node as second >>> parameter. In the new version the control parameter is explicit and we pass in NULL explicitly at all call sites where >>> control is not needed. >>> >>>> >>>>> >>>>>>> Also an other reason to have control edge in this place is the address of this klass load is based on ConP constant. >>>>>>> In a normal case after class check you should have CheckCastPP attached to control and used as address's base. >>>>> >>>>> I see, Vladimir. Just a related question: If the address of the klass load were *not* based on a ConP constant, would it >>>>> be better (or required) to check with a CheckCastPP the value that was loaded? >>>> >>>> Yes, you would use CheckCastPP to cast type to exact class you checked for and pin it to 'true' projection of the >>>> check. As we do for in gen_checkcast() or other places (speculative types). >>>> >>>> if (a.class == exact_class) >>>> (exact_class)a >>> >>> Thank you for the explanation. >>> >>>> >>>> In this case you don't need control on following load. >>>> >>>>> >>>>>>> >>>>>>> And agree with Roland's suggestion about Op_LoadKlass check. >>>>>>> >>>>>> What you are suggesting is to make a super class (LoadNode) aware of its subclasses (LoadPNode and LoadKlassNode). >>>>>> From a software engineering point of view, we should try to avoid doing that. Another solution would be to add a >>>>>> method (e.g., can_remove_control()) to LoadNode, LoadPNode, and LoadKlassNode. The function returns 'false' for >>>>>> LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What do you think? >>>>> >>>>> I think Albert's suggestion has the advantage that if we will ever decide to inherit from LoadKlass, we won't have to >>>>> worry about modifying its superclass. So I think it will be good if we go with it. >>>> >>>> I prefer Albert's name can_remove_control() >>> >>> OK, I changed the method's name. >>> >>> Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.03/ >>> >>> All JPRT tests and also the failing test cases pass with the new version. >>> >>> Thank you and best regards, >>> >>> >>> Zoltan >>> >>>> >>>> Thanks, >>>> Vladimir >>>> >>>>> >>>>> Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.02/ >>>>> >>>>> JPRT and also the failing test cases pass. >>>>> >>>>> >>>>> Thank you and best regards, >>>>> >>>>> >>>>> Zoltan >>>>> >>>>>> >>>>>> Best, >>>>>> Albert >>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> >>>>>>>> please review the following patch. >>>>>>>> >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>>>>>>> >>>>>>>> >>>>>>>> Problem: >>>>>>>> >>>>>>>> >>>>>>>> We have five tests that fail with SIGSEGV in our nightlies: >>>>>>>> >>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>>>>>>> >>>>>>>> All tests fail when executing the C2-compiled version of >>>>>>>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>>>>>>> >>>>>>>> >>>>>>>> 489: private void inflateSpine() { >>>>>>>> 490: if (spine == null) { >>>>>>>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>>>>>>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>>>>>>> 493: spine[0] = curChunk; >>>>>>>> 494: } >>>>>>>> 495: } >>>>>>>> >>>>>>>> >>>>>>>> The failure is due to the 'aastore' at line 493; the failure is caused >>>>>>>> by the monomorphic array check optimization >>>>>>>> (-XX:+MonomorphicArrayCheck, enabled by default). >>>>>>>> >>>>>>>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>>>>>>> profiling information) that inflateSpine() has two hot receiver types, >>>>>>>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>>>>>>> >>>>>>>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>>>>>>> >>>>>>>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>>>>>>> double[][]. >>>>>>>> >>>>>>>> Please consider the following pseudo Java code that illustrates how C2 >>>>>>>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on >>>>>>>> the two hot receiver types SpinedBuffer.OfInt and >>>>>>>> SpinedBuffer.OfDouble: >>>>>>>> >>>>>>>> >>>>>>>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>>>>>>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>>>>>>> spine[0] = curChunk; >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>>>>>>> of 'spine' *at runtime*. The check looks something like: >>>>>>>> >>>>>>>> >>>>>>>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>>>>>>> throw new ArrayStoreException(); >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>>>>>>> check like this: >>>>>>>> >>>>>>>> >>>>>>>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>>>>>>> throw new ArrayStoreException(); >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> where TYPE is the type of the 'spine' variable, as it is determined by >>>>>>>> C2 *at compile time*. The optimization treats TYPE as a constant and >>>>>>>> thus saves a load from memory + an offset calculation (I think). >>>>>>>> >>>>>>>> The problem is that due to 'spine' being either of type int[][] or of >>>>>>>> type double[][], C2 determines that TYPE==java/lang/Object. >>>>>>>> >>>>>>>> As a result, when the inflateSpine() method is executed, it first loads the >>>>>>>> Klass* corresponding to java/lang/Object (TYPE). Let us call this Klass* >>>>>>>> array_klass. Then, the method obtains a Klass* from array_klass->_element_klass. >>>>>>>> Then, the method reads from offset 20 from array_klass->_element_klass, which >>>>>>>> results in a SIGSEGV. >>>>>>>> >>>>>>>> The reason for the SIGSEGV is that the Klass* array_klass is of type >>>>>>>> java/lang/Object and is therefore represented as an 'InstanceKlass'. >>>>>>>> But _element_klass is defined only in objects of type 'ObjArrayKlass'. >>>>>>>> The compiler reads array_klass->_element_klass because it expects the >>>>>>>> destination of an 'aastore' to be an array, which is not true if >>>>>>>> TYPE==java/lang/Object. >>>>>>>> >>>>>>>> >>>>>>>> Solution: >>>>>>>> >>>>>>>> >>>>>>>> The compiler already checks if the compile-time type of the array (TYPE) is >>>>>>>> the same as the array's type at runtime. If that check fails, the method >>>>>>>> branches to an uncommon trap. In our case the SIGSEGV happens because >>>>>>>> the load of array_klass->_element_klass "floats" above the check. >>>>>>>> >>>>>>>> I propose two solutions: >>>>>>>> >>>>>>>> Solution 1. Add a control edge between the IfTrue branch of the check and >>>>>>>> the load of _array_klass->_element_klass. That prohibits the load of >>>>>>>> _array_klass->element_klass floating above the check. >>>>>>>> >>>>>>>> Solution 2. In addition to the changes in Solution 1, the compiler checks if >>>>>>>> the compile-time type of the array (TYPE) is java/lang/Object. If >>>>>>>> TYPE==java/lang/Object, the compiler should not assume array_type to be >>>>>>>> TYPE, but should determine array type at runtime instead. >>>>>>>> >>>>>>>> The reason is for Solution 2 is: We can't do an array store into a java/lang/Object. >>>>>>>> So, if we see TYPE==java/lang/Object at compile time, it is pretty sure that the >>>>>>>> method will deoptimize at runtime. Instead of assuming TYPE to be java/lang/Object, >>>>>>>> we read at runtime the type of the array store's destination. That still gives us some >>>>>>>> chance that the compiled version of the method will successfully execute. The >>>>>>>> additional check is in parseHelper.cpp in Parse::array_store_check() on line 171. >>>>>>>> >>>>>>>> >>>>>>>> Webrev: >>>>>>>> >>>>>>>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>>>>>>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>>>>>>> >>>>>>>> >>>>>>>> Testing (for both solutions): >>>>>>>> >>>>>>>> JPRT, manual testing with failing test cases. >>>>>>>> >>>>>>>> >>>>>>>> Please note that both solutions are somewhat "hacky", because that allowed me to >>>>>>>> produce a solution within reasonable time and with reasonable effort. Any suggestions >>>>>>>> or feedback is welcome. >>>>>>>> >>>>>>>> >>>>>>>> Thank you and best regards, >>>>>>>> >>>>>>>> >>>>>>>> Zoltan >>>>>>>> >>>>>> >>>>> >>> > From zoltan.majo at oracle.com Thu Nov 6 10:22:00 2014 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Thu, 06 Nov 2014 11:22:00 +0100 Subject: [9] RFR(S): 8057622: java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest: SEGV inside compiled code (sparc) In-Reply-To: <16C2529F-2925-423C-96FD-8C25EBF7347B@oracle.com> References: <54578A15.7060605@oracle.com> <5457C2D5.9080401@oracle.com> <5458908E.1050909@oracle.com> <5458CED2.6030203@oracle.com> <54591D84.30709@oracle.com> <545A276A.9020509@oracle.com> <545A5BD5.60000@oracle.com> <545B352E.3030403@oracle.com> <16C2529F-2925-423C-96FD-8C25EBF7347B@oracle.com> Message-ID: <545B4BC8.4080900@oracle.com> Thank you, Roland! Zoltan On 11/06/2014 11:05 AM, Roland Westrelin wrote: > For the record, final webrev looks good to me. > > Roland. > >> On Nov 6, 2014, at 9:45 AM, Zolt?n Maj? wrote: >> >> Thank you, Roland, Vladimir, and Albert, for the reviews! >> >> >> Zoltan >> >> On 11/05/2014 06:18 PM, Vladimir Kozlov wrote: >>> This looks good. Don't forget to add label noreg-sqe since we have tests already. >>> >>> Thanks, >>> Vladimir >>> >>> On 11/5/14 5:34 AM, Zolt?n Maj? wrote: >>>> Hi Vladimir, >>>> >>>> >>>> thank you for the feedback. >>>> >>>> On 11/04/2014 07:40 PM, Vladimir Kozlov wrote: >>>>> On 11/4/14 5:04 AM, Zolt?n Maj? wrote: >>>>>> Hi Roland, Vladimir, and Albert! >>>>>> >>>>>> >>>>>> thank you for your feedback and for your suggestions! Please see comments on specific issues inline. >>>>>> >>>>>> On 11/04/2014 09:38 AM, Albert Noll wrote: >>>>>>> Hi, >>>>>>> >>>>>>> please see comments inline. >>>>>>> On 11/03/2014 07:00 PM, Vladimir Kozlov wrote: >>>>>>>> I agree with additional check tak != TypeKlassPtr::OBJECT (solution 2). >>>>>>>> >>>>>>>> Instead of creating new LoadKlassNode::make() method you can set control in parseHelper.cpp since it is the only >>>>>>>> place which set it: >>>>>>>> >>>>>>>> Node* a_e_klass = LoadKlassNode::make(_gvn, immutable_memory(), p2, tak); >>>>>>>> if (always_see_exact_class) { >>>>>>>> a_e_klass->init_req(MemNode::Control, control()); >>>>>>>> } >>>>>>>> a_e_klass = _gvn.transform(a_e_klass); >>>>>>>> >>>>>>> An alternative solution is to make a default parameter for control. This way we can set the control in make() without >>>>>>> having to add a new make() function. >>>>>> I took Albert's alternative solution. Now we also set the default value for parameter TypeKlassPtr* tk in >>>>>> parseHelper.cpp at line 230. >>>>> As I replied to Albert, I think it should be explicit parameter and not default. >>>>> Yes, you would need to modify more files but it is better to be explicit I think. >>>> OK, I see. I updated the code so that we have one LoadKlassNode::make() method, with the control node as second >>>> parameter. In the new version the control parameter is explicit and we pass in NULL explicitly at all call sites where >>>> control is not needed. >>>> >>>>>>>> Also an other reason to have control edge in this place is the address of this klass load is based on ConP constant. >>>>>>>> In a normal case after class check you should have CheckCastPP attached to control and used as address's base. >>>>>> I see, Vladimir. Just a related question: If the address of the klass load were *not* based on a ConP constant, would it >>>>>> be better (or required) to check with a CheckCastPP the value that was loaded? >>>>> Yes, you would use CheckCastPP to cast type to exact class you checked for and pin it to 'true' projection of the >>>>> check. As we do for in gen_checkcast() or other places (speculative types). >>>>> >>>>> if (a.class == exact_class) >>>>> (exact_class)a >>>> Thank you for the explanation. >>>> >>>>> In this case you don't need control on following load. >>>>> >>>>>>>> And agree with Roland's suggestion about Op_LoadKlass check. >>>>>>>> >>>>>>> What you are suggesting is to make a super class (LoadNode) aware of its subclasses (LoadPNode and LoadKlassNode). >>>>>>> From a software engineering point of view, we should try to avoid doing that. Another solution would be to add a >>>>>>> method (e.g., can_remove_control()) to LoadNode, LoadPNode, and LoadKlassNode. The function returns 'false' for >>>>>>> LoadPNode and LoadKlassNode; and returns 'true' for LoadNode. What do you think? >>>>>> I think Albert's suggestion has the advantage that if we will ever decide to inherit from LoadKlass, we won't have to >>>>>> worry about modifying its superclass. So I think it will be good if we go with it. >>>>> I prefer Albert's name can_remove_control() >>>> OK, I changed the method's name. >>>> >>>> Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.03/ >>>> >>>> All JPRT tests and also the failing test cases pass with the new version. >>>> >>>> Thank you and best regards, >>>> >>>> >>>> Zoltan >>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>>> Here is the new webrev: http://cr.openjdk.java.net/~zmajo/8057622/webrev.02/ >>>>>> >>>>>> JPRT and also the failing test cases pass. >>>>>> >>>>>> >>>>>> Thank you and best regards, >>>>>> >>>>>> >>>>>> Zoltan >>>>>> >>>>>>> Best, >>>>>>> Albert >>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimir >>>>>>>> >>>>>>>> On 11/3/14 5:58 AM, Zolt?n Maj? wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> >>>>>>>>> please review the following patch. >>>>>>>>> >>>>>>>>> >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8057622 >>>>>>>>> >>>>>>>>> >>>>>>>>> Problem: >>>>>>>>> >>>>>>>>> >>>>>>>>> We have five tests that fail with SIGSEGV in our nightlies: >>>>>>>>> >>>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/InfiniteStreamWithLimitOpTest.java >>>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java >>>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamSpliteratorTest.java >>>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/StreamBuilderTest.java >>>>>>>>> java/util/stream/test/org/openjdk/tests/java/util/stream/MapOp.java >>>>>>>>> >>>>>>>>> All tests fail when executing the C2-compiled version of >>>>>>>>> java/util/stream/SpinedBuffer at OfPrimitive.inflateSpine(): >>>>>>>>> >>>>>>>>> >>>>>>>>> 489: private void inflateSpine() { >>>>>>>>> 490: if (spine == null) { >>>>>>>>> 491: spine = newArrayArray(MIN_SPINE_SIZE); >>>>>>>>> 492: priorElementCount = new long[MIN_SPINE_SIZE]; >>>>>>>>> 493: spine[0] = curChunk; >>>>>>>>> 494: } >>>>>>>>> 495: } >>>>>>>>> >>>>>>>>> >>>>>>>>> The failure is due to the 'aastore' at line 493; the failure is caused >>>>>>>>> by the monomorphic array check optimization >>>>>>>>> (-XX:+MonomorphicArrayCheck, enabled by default). >>>>>>>>> >>>>>>>>> In InfiniteStreamWithLimitOpTest.java, C2 determines (based on >>>>>>>>> profiling information) that inflateSpine() has two hot receiver types, >>>>>>>>> SpinedBuffer.OfInt and SpinedBuffer.OfDouble: >>>>>>>>> >>>>>>>>> - in SpinedBuffer.OfInt, the variable 'spine' is of type int[][] >>>>>>>>> >>>>>>>>> - in SpinedBuffer.ofDouble, the variable 'spine' is of type >>>>>>>>> double[][]. >>>>>>>>> >>>>>>>>> Please consider the following pseudo Java code that illustrates how C2 >>>>>>>>> sees SpinedBuffer.OfPrimitive.inflateSpine() after inlining based on >>>>>>>>> the two hot receiver types SpinedBuffer.OfInt and >>>>>>>>> SpinedBuffer.OfDouble: >>>>>>>>> >>>>>>>>> >>>>>>>>> static void inflateSpine(boolean isOfDouble, Object curChunk) { >>>>>>>>> Object[] spine = isOfDouble ? new double[8][] : new int[8][]; >>>>>>>>> spine[0] = curChunk; >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> If MonomorphicArrayCheck is disabled, C2 checks the element type >>>>>>>>> of 'spine' *at runtime*. The check looks something like: >>>>>>>>> >>>>>>>>> >>>>>>>>> if (!spine.getClass().getComponentType().isInstance(curChunk)) { >>>>>>>>> throw new ArrayStoreException(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> If MonomorphicArrayCheck is enabled (our case), C2 creates an array >>>>>>>>> check like this: >>>>>>>>> >>>>>>>>> >>>>>>>>> if (!TYPE.getClass().getComponentType().isInstance(curChunk)) { >>>>>>>>> throw new ArrayStoreException(); >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> where TYPE is the type of the 'spine' variable, as it is determined by >>>>>>>>> C2 *at compile time*. The optimization treats TYPE as a constant and >>>>>>>>> thus saves a load from memory + an offset calculation (I think). >>>>>>>>> >>>>>>>>> The problem is that due to 'spine' being either of type int[][] or of >>>>>>>>> type double[][], C2 determines that TYPE==java/lang/Object. >>>>>>>>> >>>>>>>>> As a result, when the inflateSpine() method is executed, it first loads the >>>>>>>>> Klass* corresponding to java/lang/Object (TYPE). Let us call this Klass* >>>>>>>>> array_klass. Then, the method obtains a Klass* from array_klass->_element_klass. >>>>>>>>> Then, the method reads from offset 20 from array_klass->_element_klass, which >>>>>>>>> results in a SIGSEGV. >>>>>>>>> >>>>>>>>> The reason for the SIGSEGV is that the Klass* array_klass is of type >>>>>>>>> java/lang/Object and is therefore represented as an 'InstanceKlass'. >>>>>>>>> But _element_klass is defined only in objects of type 'ObjArrayKlass'. >>>>>>>>> The compiler reads array_klass->_element_klass because it expects the >>>>>>>>> destination of an 'aastore' to be an array, which is not true if >>>>>>>>> TYPE==java/lang/Object. >>>>>>>>> >>>>>>>>> >>>>>>>>> Solution: >>>>>>>>> >>>>>>>>> >>>>>>>>> The compiler already checks if the compile-time type of the array (TYPE) is >>>>>>>>> the same as the array's type at runtime. If that check fails, the method >>>>>>>>> branches to an uncommon trap. In our case the SIGSEGV happens because >>>>>>>>> the load of array_klass->_element_klass "floats" above the check. >>>>>>>>> >>>>>>>>> I propose two solutions: >>>>>>>>> >>>>>>>>> Solution 1. Add a control edge between the IfTrue branch of the check and >>>>>>>>> the load of _array_klass->_element_klass. That prohibits the load of >>>>>>>>> _array_klass->element_klass floating above the check. >>>>>>>>> >>>>>>>>> Solution 2. In addition to the changes in Solution 1, the compiler checks if >>>>>>>>> the compile-time type of the array (TYPE) is java/lang/Object. If >>>>>>>>> TYPE==java/lang/Object, the compiler should not assume array_type to be >>>>>>>>> TYPE, but should determine array type at runtime instead. >>>>>>>>> >>>>>>>>> The reason is for Solution 2 is: We can't do an array store into a java/lang/Object. >>>>>>>>> So, if we see TYPE==java/lang/Object at compile time, it is pretty sure that the >>>>>>>>> method will deoptimize at runtime. Instead of assuming TYPE to be java/lang/Object, >>>>>>>>> we read at runtime the type of the array store's destination. That still gives us some >>>>>>>>> chance that the compiled version of the method will successfully execute. The >>>>>>>>> additional check is in parseHelper.cpp in Parse::array_store_check() on line 171. >>>>>>>>> >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> >>>>>>>>> Solution 1: http://cr.openjdk.java.net/~zmajo/8057622/webrev.00/ >>>>>>>>> Solution 2: http://cr.openjdk.java.net/~zmajo/8057622/webrev.01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Testing (for both solutions): >>>>>>>>> >>>>>>>>> JPRT, manual testing with failing test cases. >>>>>>>>> >>>>>>>>> >>>>>>>>> Please note that both solutions are somewhat "hacky", because that allowed me to >>>>>>>>> produce a solution within reasonable time and with reasonable effort. Any suggestions >>>>>>>>> or feedback is welcome. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thank you and best regards, >>>>>>>>> >>>>>>>>> >>>>>>>>> Zoltan >>>>>>>>> From tobias.hartmann at oracle.com Thu Nov 6 10:25:09 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 06 Nov 2014 11:25:09 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545AC741.2040803@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> Message-ID: <545B4C85.7000109@oracle.com> Hi Vladimir, thanks for the review. On 06.11.2014 01:56, Vladimir Kozlov wrote: > So the fix is add -XX:-TieredCompilation. > The rest is fix problems found due to these failure. Yes, exactly. > May be we should add a new test (other RFE) which would test compilation of > trivial methods in Tiered mode. Current failure shows that we don't test it. I agree. I filed RFE JDK-8056071 and will take care of it. > I am not comfortable to rely on data which is set only by C1. This information > is accessed in non Tiered mode too. 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if TieredCompilation is enabled. Where do you think it is used in non-tiered mode? > Why not use method->has_loops(), for example? We also decide based on the number of blocks and 'MethodData::would_profile' which is only available if we have a C1 compiled version. Thanks, Tobias > > Thanks, > Vladimir > > On 11/5/14 4:12 AM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >> >> Problem: >> The test compiles and deoptimizes a method m multiple times. After each >> deoptimization the test checks if m was compiled with C2 and if so increments a >> counter. The test assumes that m is compilable until the counter reaches the >> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >> - The first compilation request for m is added to the compile queue. Although we >> profiled m and the MDO suggests that m is trivial, we do not use this >> information because due to deoptimization there is no corresponding nmethod [1]. >> We compile at level 4 and execution continues in the interpreter [2]. >> - A second compilation request for m is issued and while it is being processed >> the first compilation request finishes [3]. Because a nmethod now exists, we use >> the MDO and decide to compile at level 1 since m is trivial. >> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >> >> After the following deoptimization the counter is not incremented because the >> nmethod was compiled at level 1. As a result m is set to not compilable before >> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >> >> Problems with the current implementation: >> (1) We should always use the MDO if it is valid, even if there is no >> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >> profiling again. >> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >> wrongly assume that the method is trivial and re-compile it with C1. >> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >> constant getter methods, like those used by the test, as trivial. >> (4) We should add verification code to avoid/catch such bad tiered level >> transitions in the future. >> >> Solution: >> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >> always compile the (trivial) methods at level 1 and the test fails. >> >> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >> a method consists of a constant push and a return statement only. If so, we >> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >> >> I filed JDK-8062913 for (4) because the verification code I added still fails >> with the current implementation due to 2->2 and 3->2 transitions. >> >> Testing: >> - JPRT including failing whitebox test >> >> Thanks, >> Tobias >> >> >> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >> [2] First compilation request: >> InterpreterRuntime::frequency_counter_overflow >> InterpreterRuntime::frequency_counter_overflow_inner >> SimpleThresholdPolicy::event >> AdvancedThresholdPolicy::method_invocation_event >> AdvancedThresholdPolicy::call_event >> AdvancedThresholdPolicy::common >> [is_trivial() returns false because no nmethod available] >> [next_level = CompLevel_Full_Optimization] >> SimpleThresholdPolicy::compile >> SimpleThresholdPolicy::submit_compile >> CompileBroker::compile_method >> CompileBroker::compile_method_base >> CompileBroker::create_compile_task >> [continue execution in interpreter] >> >> [3] Second compilation request: >> InterpreterRuntime::frequency_counter_overflow >> InterpreterRuntime::frequency_counter_overflow_inner >> SimpleThresholdPolicy::event >> AdvancedThresholdPolicy::method_invocation_event >> [First compilation finishes here -> >> !CompileBroker::compilation_is_in_queue()] >> AdvancedThresholdPolicy::call_event >> AdvancedThresholdPolicy::common >> [is_trivial() returns true because nmethod/mdo now available] >> [next_level = CompLevel_Simple] >> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >> >> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >> From martin.doerr at sap.com Thu Nov 6 10:40:24 2014 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 6 Nov 2014 10:40:24 +0000 Subject: RFR (XS): 8062950: Bug in locking code when UseOptoBiasInlining is disabled: assert(dmw->is_neutral()) failed: invariant In-Reply-To: <545AB32E.8070402@oracle.com> References: <7C9B87B351A4BA4AA9EC95BB418116566ACE597E@DEWDFEMB19C.global.corp.sap> <545AB32E.8070402@oracle.com> Message-ID: <7C9B87B351A4BA4AA9EC95BB418116566ACE6BEC@DEWDFEMB19C.global.corp.sap> Hi Vladimir, thanks for replying quickly. Are you sure you want the swap_reg_contains_mark flag to get removed? There's a TODO in front of the changed line: // TODO: optimize away redundant LDs of obj->mark and improve the markword triage // order to reduce the number of conditional branches in the most common cases. // Beware -- there's a subtle invariant that fetch of the markword // at [FETCH], below, will never observe a biased encoding (*101b). // If this invariant is not held we risk exclusion (safety) failure. So I'm not sure if the flag may be useful again when somebody works on this TODO. Best regards, Martin -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Donnerstag, 6. November 2014 00:31 To: Christian Thalinger; Doerr, Martin Cc: hotspot-compiler-dev at openjdk.java.net; Hotspot dev runtime Subject: Re: RFR (XS): 8062950: Bug in locking code when UseOptoBiasInlining is disabled: assert(dmw->is_neutral()) failed: invariant It is our (Compiler group) code. This problem was introduced with my changes for RTM locking. Martin your changes are good. But you cleanup a bit this code since we now never put markword to tmpReg before this call? Thanks, Vladimir On 11/5/14 3:13 PM, Christian Thalinger wrote: > I?m not exactly sure who is our biased locking expert these days but I guess it?s someone from the runtime team. CC?ing them. > >> On Nov 5, 2014, at 7:38 AM, Doerr, Martin wrote: >> >> Hi, >> >> we found a bug in MacroAssembler::fast_lock on x86 which shows up when UseOptoBiasInlining is switched off. >> The problem is that biased_locking_enter is used with swap_reg_contains_mark==true, which is no longer correct after biased_locking_enter was put in front of check for IsInflated. >> >> Please review >> http://cr.openjdk.java.net/~goetz/webrevs/8062950-lockBug/webrev.00/ >> >> Best regards, >> Martin > From tatiana.pivovarova at oracle.com Thu Nov 6 12:27:18 2014 From: tatiana.pivovarova at oracle.com (Tatiana Pivovarova) Date: Thu, 06 Nov 2014 15:27:18 +0300 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm Message-ID: <545B6926.8070906@oracle.com> Hi, please review the following patch Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 Webrev: http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ Problem: Flag "-XX:+EliminateAutoBox" works on server VM only. The test should work with Client VM too. Solution: Add flag "-XX:+IgnoreUnrecognizedVMOptions." Testing: Manual Thanks, Tatiana From vladimir.x.ivanov at oracle.com Thu Nov 6 11:41:52 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 06 Nov 2014 15:41:52 +0400 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm In-Reply-To: <545B6926.8070906@oracle.com> References: <545B6926.8070906@oracle.com> Message-ID: <545B5E80.3060003@oracle.com> Reviewed. Best regards, Vladimir Ivanov On 11/6/14, 4:27 PM, Tatiana Pivovarova wrote: > Hi, > > please review the following patch > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 > Webrev: > http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ > > Problem: > Flag "-XX:+EliminateAutoBox" works on server VM only. The test should > work with Client VM too. > > Solution: > Add flag "-XX:+IgnoreUnrecognizedVMOptions." > > Testing: > Manual > > Thanks, > Tatiana > From albert.noll at oracle.com Thu Nov 6 13:30:54 2014 From: albert.noll at oracle.com (Albert Noll) Date: Thu, 06 Nov 2014 14:30:54 +0100 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm In-Reply-To: <545B6926.8070906@oracle.com> References: <545B6926.8070906@oracle.com> Message-ID: <545B780E.3020403@oracle.com> Looks good to me (not a reviewer). Best, Albert On 11/06/2014 01:27 PM, Tatiana Pivovarova wrote: > Hi, > > please review the following patch > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 > Webrev: > http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ > > Problem: > Flag "-XX:+EliminateAutoBox" works on server VM only. The test should > work with Client VM too. > > Solution: > Add flag "-XX:+IgnoreUnrecognizedVMOptions." > > Testing: > Manual > > Thanks, > Tatiana > From nils.eliasson at oracle.com Thu Nov 6 13:22:35 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Thu, 06 Nov 2014 14:22:35 +0100 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint Message-ID: <545B761B.2000505@oracle.com> Hi, Please review this small change. The first implementation of compiler diagnostic commands was asserting on safepoint or lock. The safepoint was always taken since the dcmd by default block for a safepoint. That is not correct, the lock must be taken, and a whether we are at a safepoint doesn't matter. Tested with jtreg tests in Hotspot and JDK that exercise this code. Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ Thanks, Nils From rickard.backman at oracle.com Thu Nov 6 13:48:39 2014 From: rickard.backman at oracle.com (Rickard =?iso-8859-1?Q?B=E4ckman?=) Date: Thu, 6 Nov 2014 14:48:39 +0100 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm In-Reply-To: <545B6926.8070906@oracle.com> References: <545B6926.8070906@oracle.com> Message-ID: <20141106134839.GD27319@rbackman> Reviewed. /R On 11/06, Tatiana Pivovarova wrote: > Hi, > > please review the following patch > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 > Webrev: http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ > > Problem: > Flag "-XX:+EliminateAutoBox" works on server VM only. The test > should work with Client VM too. > > Solution: > Add flag "-XX:+IgnoreUnrecognizedVMOptions." > > Testing: > Manual > > Thanks, > Tatiana > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From roland.westrelin at oracle.com Thu Nov 6 15:39:21 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Thu, 6 Nov 2014 16:39:21 +0100 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <545A577F.5080805@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> Message-ID: <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> > I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. I?ll send an updated webrev. >>>>> opaquenode.cpp >>>>> >>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>> >>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>> >>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >> >> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). > > So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. (I know you don?t like when I remove the text from previous emails but that was really getting confusing) PhiNode::Value() has this code: // Check for trip-counted loop. If so, be smarter. CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; if( l && l->can_be_counted_loop(phase) && ((const Node*)l->phi() == this) ) { // Trip counted loop! // protect against init_trip() or limit() returning NULL const Node *init = l->init_trip(); const Node *limit = l->limit(); if( init != NULL && limit != NULL && l->stride_is_con() ) { const TypeInt *lo = init ->bottom_type()->isa_int(); const TypeInt *hi = limit->bottom_type()->isa_int(); if( lo && hi ) { // Dying loops might have TOP here int stride = l->stride_con(); if( stride < 0 ) { // Down-counter loop const TypeInt *tmp = lo; lo = hi; hi = tmp; stride = -stride; } if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( return TypeInt::make(lo->_lo,hi->_hi,3); } } } That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. Roland. From igor.ignatyev at oracle.com Thu Nov 6 18:02:56 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 06 Nov 2014 21:02:56 +0300 Subject: RFR(M) : 8059624 : Test task: WhiteBox API for testing segmented codecache feature In-Reply-To: <545AB8E7.40400@oracle.com> References: <545A4F63.6010008@oracle.com> <545AB8E7.40400@oracle.com> Message-ID: <545BB7D0.5080601@oracle.com> updated webrev: http://cr.openjdk.java.net/~iignatyev/8059624/webrev.01/ 679 lines changed: 643 ins; 2 del; 34 mod; Hi Vladimir/Tobias, thanks for review. please see my answers inline. On 11/06/2014 02:55 AM, Vladimir Kozlov wrote: > Hi Igor, > > codeBlob.hpp > > Why you need?: > > + // placement > + inline void* operator new(size_t s, void* p) throw() { return p; } I need this to execute ctor in WhiteBox::allocateCodeHeap: > 887 new (blob) BufferBlob("WB::DummyBlob", full_size); it's impossible to use default placement new, since BufferBlob overloads operator new. > compileBroker.cpp > > I think MonitorLockerEx locker() should be before the loop. Then you may > not need to get lock in WB_UnlockCompilation. I moved MonitorLockerEx before the loop, but I still need a lock in UnlockCompilation to call notify_all(); > whitebox.cpp > > allocateCodeHeap() - what if CodeCache::allocate() failed to allocate? it will return null, see AllocationCodeHeapTest.java: > 93 while ((addr = WHITE_BOX.allocateCodeHeap(size, type.id)) != 0) { > 94 blobs.add(addr); > 95 } > In WB_GetCodeHeapEntries() checks that blobs array is not empty after > the loop which add entries. should I? I thought that blobs.length() will be 0 and blobs.begin() will equal blobs.end() in this case, won't it? > You need to prevent sweeping when you processing codecache blobs > (getCodeHeapEntries). Otherwise pointers from getCodeBlobs() may point > to dead or overwritten space, consider -Xcomp -XX:+DeoptimizeALot sweeper does its work after acquiring CodeCache_lock and there is lock CodeCache before 1st loop, so I introduced struct CodeBlobStub and store it in GrowableArray instead of CodeBlob. > > Thanks, > Vladimir On 11/06/2014 12:11 PM, Tobias Hartmann wrote: > Hi Igor, > > looks good (not a reviewer). In addition to Vladimir's comments: > > 'WhiteBox::allocateCodeHeap' should be 'allocate_code_heap'. I think the name is > a bit misleading since we don't allocate a CodeHeap but a CodeBlob in a > CodeHeap. Maybe use 'code_heap_allocate' or 'allocate_code_blob'. Same for > 'WB_AllocateCodeHeap'. agree, changed it to AllocateCodeBlob/FreeCodeBlob // I just tried to mimic WB_AllocateMetaspace > > In 'BlobType::getAvailable' you always return the NonProfiledCodeHeap if > CodeCacheSegmentation is enabled. But if we only use the interpreter (-Xint), we > don't have any method code heaps. Also with 'TieredStopAtLevel <= > CompLevel_simple' we don't need the ProfiledCodeHeap (see > 'CodeCache::heap_available'). The corresponding tests will hit the "heap is > null" assert in 'CodeCache::allocate'. I updated 'BlobType::getAvailable' to handle such the situations. > On 11/5/14 8:25 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8059624/webrev.00/ >> 660 lines changed: 624 ins; 2 del; 34 mod; >> >> Hi all, >> >> please review the patch which adds new WhiteBox methods needed for >> better testing SegmentedCodeCache: >> - perform allocation in code cache >> - force code cache sweep >> - lock/unlock compilation >> - get a segment id for nmethod. >> besides these methods, the patch also adds a method to get all entries >> in code heap. >> >> changes in product code: >> - monitor 'Compilation_lock' was added to implement compilation >> locking/unlocking >> - align_code_offset function was made a static member of CodeBlob >> - WhiteBox was made a friend of several classes >> >> testing: jprt, new added tests >> jbs: https://bugs.openjdk.java.net/browse/JDK-8059624 >> From roland.westrelin at oracle.com Thu Nov 6 19:14:56 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Thu, 6 Nov 2014 20:14:56 +0100 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <54581D5E.2090706@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> Message-ID: > loopTransform.cpp > > insert_castii_before_loop() --> cast_incr_before_loop() > You missed registration of new CastII nodes - register_new_node(). Actually I don?t understand why there?s a need for the register_new_node() call on the CastII nodes since I already _igvn.transform() them. Roland. From igor.ignatyev at oracle.com Thu Nov 6 19:47:44 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 06 Nov 2014 22:47:44 +0300 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545B4C85.7000109@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> Message-ID: <545BD060.9010705@oracle.com> Hi Tobias, On 11/06/2014 01:25 PM, Tobias Hartmann wrote: > Hi Vladimir, > > thanks for the review. > > On 06.11.2014 01:56, Vladimir Kozlov wrote: >> So the fix is add -XX:-TieredCompilation. >> The rest is fix problems found due to these failure. > > Yes, exactly. > >> May be we should add a new test (other RFE) which would test compilation of >> trivial methods in Tiered mode. Current failure shows that we don't test it. > > I agree. I filed RFE JDK-8056071 and will take care of it. you posted wrong cr. actually, PavelP is working on such the tests, it's tracked by 8059575 and/or 8028590. > >> I am not comfortable to rely on data which is set only by C1. This information >> is accessed in non Tiered mode too. > > 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if > TieredCompilation is enabled. Where do you think it is used in non-tiered mode? > >> Why not use method->has_loops(), for example? > > We also decide based on the number of blocks and 'MethodData::would_profile' > which is only available if we have a C1 compiled version. > > Thanks, > Tobias > >> >> Thanks, >> Vladimir >> >> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>> Hi, >>> >>> please review the following patch. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>> >>> Problem: >>> The test compiles and deoptimizes a method m multiple times. After each >>> deoptimization the test checks if m was compiled with C2 and if so increments a >>> counter. The test assumes that m is compilable until the counter reaches the >>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>> - The first compilation request for m is added to the compile queue. Although we >>> profiled m and the MDO suggests that m is trivial, we do not use this >>> information because due to deoptimization there is no corresponding nmethod [1]. >>> We compile at level 4 and execution continues in the interpreter [2]. >>> - A second compilation request for m is issued and while it is being processed >>> the first compilation request finishes [3]. Because a nmethod now exists, we use >>> the MDO and decide to compile at level 1 since m is trivial. >>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>> >>> After the following deoptimization the counter is not incremented because the >>> nmethod was compiled at level 1. As a result m is set to not compilable before >>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>> >>> Problems with the current implementation: >>> (1) We should always use the MDO if it is valid, even if there is no >>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >>> profiling again. >>> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>> wrongly assume that the method is trivial and re-compile it with C1. >>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>> constant getter methods, like those used by the test, as trivial. >>> (4) We should add verification code to avoid/catch such bad tiered level >>> transitions in the future. >>> >>> Solution: >>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>> always compile the (trivial) methods at level 1 and the test fails. >>> >>> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >>> a method consists of a constant push and a return statement only. If so, we >>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>> >>> I filed JDK-8062913 for (4) because the verification code I added still fails >>> with the current implementation due to 2->2 and 3->2 transitions. >>> >>> Testing: >>> - JPRT including failing whitebox test >>> >>> Thanks, >>> Tobias >>> >>> >>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>> [2] First compilation request: >>> InterpreterRuntime::frequency_counter_overflow >>> InterpreterRuntime::frequency_counter_overflow_inner >>> SimpleThresholdPolicy::event >>> AdvancedThresholdPolicy::method_invocation_event >>> AdvancedThresholdPolicy::call_event >>> AdvancedThresholdPolicy::common >>> [is_trivial() returns false because no nmethod available] >>> [next_level = CompLevel_Full_Optimization] >>> SimpleThresholdPolicy::compile >>> SimpleThresholdPolicy::submit_compile >>> CompileBroker::compile_method >>> CompileBroker::compile_method_base >>> CompileBroker::create_compile_task >>> [continue execution in interpreter] >>> >>> [3] Second compilation request: >>> InterpreterRuntime::frequency_counter_overflow >>> InterpreterRuntime::frequency_counter_overflow_inner >>> SimpleThresholdPolicy::event >>> AdvancedThresholdPolicy::method_invocation_event >>> [First compilation finishes here -> >>> !CompileBroker::compilation_is_in_queue()] >>> AdvancedThresholdPolicy::call_event >>> AdvancedThresholdPolicy::common >>> [is_trivial() returns true because nmethod/mdo now available] >>> [next_level = CompLevel_Simple] >>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>> >>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >>> From vladimir.kozlov at oracle.com Thu Nov 6 20:30:30 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 12:30:30 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> Message-ID: <545BDA66.3060503@oracle.com> register_new_node() place/map a node into corresponding loop body. That is what is missing. Also calling _igvn.transform() here could be dangerous during this graph transformation. Vladimir On 11/6/14 11:14 AM, Roland Westrelin wrote: >> loopTransform.cpp >> >> insert_castii_before_loop() --> cast_incr_before_loop() >> You missed registration of new CastII nodes - register_new_node(). > > Actually I don?t understand why there?s a need for the register_new_node() call on the CastII nodes since I already _igvn.transform() them. > > Roland. > From john.r.rose at oracle.com Thu Nov 6 20:30:15 2014 From: john.r.rose at oracle.com (John Rose) Date: Thu, 6 Nov 2014 12:30:15 -0800 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <545B4134.4040103@oracle.com> References: <545A960C.7010008@oracle.com> <545AAB7D.3010100@oracle.com> <545B4134.4040103@oracle.com> Message-ID: <9E6FEF1D-370C-4860-A5DA-07CF9384BF07@oracle.com> On Nov 6, 2014, at 1:36 AM, Albert Noll wrote: > It would be good to have data from different applications / architectures. > > Does anyone know more about the motivation for IndexSet, i.e., why it was introduced in the first place? It is a bespoke type for the register allocator, originally called List16Set and written by Cliff Click in 1997 (or earlier, at Rice). Original source file was chaitin.hpp. In 1998 Todd Turnidge did a bunch of cleanups on the source base and factored it as IndexSet into its own header file. Perhaps it was well tuned for the hardware of 1998. :-) ? John From john.r.rose at oracle.com Thu Nov 6 20:40:11 2014 From: john.r.rose at oracle.com (John Rose) Date: Thu, 6 Nov 2014 12:40:11 -0800 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <9E6FEF1D-370C-4860-A5DA-07CF9384BF07@oracle.com> References: <545A960C.7010008@oracle.com> <545AAB7D.3010100@oracle.com> <545B4134.4040103@oracle.com> <9E6FEF1D-370C-4860-A5DA-07CF9384BF07@oracle.com> Message-ID: On Nov 6, 2014, at 12:30 PM, John Rose wrote: > Original source file was chaitin.hpp. Correction: live.hpp. //------------------------------List16Set-------------------------------------- // An abbreviated implementation of Set. Implementation is a list of shorts. // Inserting and deleting is linear (ouch)! Only suitable when lists are // expected to be short. struct List16Set { uint16 _cnt, _max; uint16* _data; ... }; (That's enough software archaeology for today!) -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.r.rose at oracle.com Thu Nov 6 20:40:43 2014 From: john.r.rose at oracle.com (John Rose) Date: Thu, 6 Nov 2014 12:40:43 -0800 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <545A960C.7010008@oracle.com> References: <545A960C.7010008@oracle.com> Message-ID: On Nov 5, 2014, at 1:26 PM, Aleksey Shipilev wrote: > - IndexSetIterator performance is important, and therefore the lookup > table approach from IndexSetIterator was transplanted to new > BitMapIterator. We might want to improve BitMap::get_next_one_offset > with lookup tables as well, but that will contaminate the current > experiment. Curiously, I googled around for info on bitset enumeration, and found the mother lode here: http://chessprogramming.wikispaces.com/BitScan It seems the table-based algorithm is still quite competitive, even with hardware supported ffs/ctz. ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Thu Nov 6 20:57:05 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 06 Nov 2014 23:57:05 +0300 Subject: RFR (XS) : 8063157 : add targets for optimized builds Message-ID: <545BE0A1.7030806@oracle.com> http://cr.openjdk.java.net/~iignatyev/8063157/webrev.00/ 8 lines changed: 4 ins; 0 del; 4 mod; Hi all, Please review the patch which adds optimized build targets and removes mentions of optimized builds from 'jprt.properties' in the top level repo. we did the same in hotspot by 8056072, but after the switch to 'full jdk' we need this change in the top level repo as well. jbs: https://bugs.openjdk.java.net/browse/JDK-8063157 RFR thread for 8056072 : http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2014-August/015299.html -- Igor From vladimir.kozlov at oracle.com Thu Nov 6 22:12:46 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 14:12:46 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> Message-ID: <545BF25E.7010505@oracle.com> On 11/6/14 7:39 AM, Roland Westrelin wrote: > >> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. > > I?ll send an updated webrev. > >>>>>> opaquenode.cpp >>>>>> >>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>> >>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>> >>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>> >>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >> >> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. > > (I know you don?t like when I remove the text from previous emails but that was really getting confusing) At least leave webrev link so I don't need to search for it in previous mails. > > PhiNode::Value() has this code: > > // Check for trip-counted loop. If so, be smarter. > CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; > if( l && l->can_be_counted_loop(phase) && > ((const Node*)l->phi() == this) ) { // Trip counted loop! > // protect against init_trip() or limit() returning NULL > const Node *init = l->init_trip(); > const Node *limit = l->limit(); > if( init != NULL && limit != NULL && l->stride_is_con() ) { > const TypeInt *lo = init ->bottom_type()->isa_int(); > const TypeInt *hi = limit->bottom_type()->isa_int(); > if( lo && hi ) { // Dying loops might have TOP here > int stride = l->stride_con(); > if( stride < 0 ) { // Down-counter loop > const TypeInt *tmp = lo; lo = hi; hi = tmp; > stride = -stride; > } > if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( > return TypeInt::make(lo->_lo,hi->_hi,3); > } > } > } > > That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: // Step B2: Build a zero-trip guard for the main-loop. After leaving the // pre-loop, the main-loop may not execute at all. Later in life this // zero-trip guard will become the minimum-trip guard when we unroll // the main-loop. Node *min_opaq = new Opaque1Node(C, limit); Node *min_cmp = new CmpINode( pre_incr, min_opaq ); But you are talking about pre-loop exit check: // Step B4: Shorten the pre-loop to run only 1 iteration (for now). // RCE and alignment may change this later. Node *cmp_end = pre_end->cmp_node(); assert( cmp_end->in(2) == limit, "" ); Node *pre_limit = new AddINode( init, stride ); // Save the original loop limit in this Opaque1 node for // use by range check elimination. Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); But in this case you can add check (cl->limit() == this) to be clear what you are looking for. Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. Thanks, Vladimir > > In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. > > Roland. > From vladimir.kozlov at oracle.com Fri Nov 7 00:23:20 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 16:23:20 -0800 Subject: RFR (XS) : 8063157 : add targets for optimized builds In-Reply-To: <545BE0A1.7030806@oracle.com> References: <545BE0A1.7030806@oracle.com> Message-ID: <545C10F8.6000604@oracle.com> Good. Important info: > PS the platforms to build w/ optimized flavor will be selected and added > into jprt.properties as the fix for JDK-8013869 after corresponding > changes in JPRT. Thanks, Vladimir On 11/6/14 12:57 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8063157/webrev.00/ > 8 lines changed: 4 ins; 0 del; 4 mod; > > Hi all, > > Please review the patch which adds optimized build targets and removes > mentions of optimized builds from 'jprt.properties' in the top level > repo. we did the same in hotspot by 8056072, but after the switch to > 'full jdk' we need this change in the top level repo as well. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8063157 > > RFR thread for 8056072 : > http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2014-August/015299.html > From vladimir.kozlov at oracle.com Fri Nov 7 00:50:07 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 16:50:07 -0800 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545B4C85.7000109@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> Message-ID: <545C173F.5090403@oracle.com> On 11/6/14 2:25 AM, Tobias Hartmann wrote: > Hi Vladimir, > > thanks for the review. > > On 06.11.2014 01:56, Vladimir Kozlov wrote: >> So the fix is add -XX:-TieredCompilation. >> The rest is fix problems found due to these failure. > > Yes, exactly. > >> May be we should add a new test (other RFE) which would test compilation of >> trivial methods in Tiered mode. Current failure shows that we don't test it. > > I agree. I filed RFE JDK-8056071 and will take care of it. > >> I am not comfortable to rely on data which is set only by C1. This information >> is accessed in non Tiered mode too. > > 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if > TieredCompilation is enabled. Where do you think it is used in non-tiered mode? You are right. NonTieredCompPolicy::is_mature() does not use it. > >> Why not use method->has_loops(), for example? > > We also decide based on the number of blocks and 'MethodData::would_profile' > which is only available if we have a C1 compiled version. Yes, but next code will do the same and only checking MDO in the last case: bool SimpleThresholdPolicy::is_trivial(Method* method) { if (method->is_accessor() || method->is_constant_getter()) { return true; } if (method->has_loops() || (method->code_size() >= 15)) { return false; } MethodData* mdo = method->method_data(); if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && (method->code_size() < 5 || (mdo->num_blocks() < 4))) { return true; } return false; } Also mdo->_would_profile will not be set if C1 compilation is bailed out (Compilation() in c1_Compilation.cpp). So we can't trust it too. May be mdo->_would_profile should be enum {unknown, no_profile, profile} and we use it instead of mdo->stats_valid(). Thanks, Vladimir > > Thanks, > Tobias > >> >> Thanks, >> Vladimir >> >> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>> Hi, >>> >>> please review the following patch. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>> >>> Problem: >>> The test compiles and deoptimizes a method m multiple times. After each >>> deoptimization the test checks if m was compiled with C2 and if so increments a >>> counter. The test assumes that m is compilable until the counter reaches the >>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>> - The first compilation request for m is added to the compile queue. Although we >>> profiled m and the MDO suggests that m is trivial, we do not use this >>> information because due to deoptimization there is no corresponding nmethod [1]. >>> We compile at level 4 and execution continues in the interpreter [2]. >>> - A second compilation request for m is issued and while it is being processed >>> the first compilation request finishes [3]. Because a nmethod now exists, we use >>> the MDO and decide to compile at level 1 since m is trivial. >>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>> >>> After the following deoptimization the counter is not incremented because the >>> nmethod was compiled at level 1. As a result m is set to not compilable before >>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>> >>> Problems with the current implementation: >>> (1) We should always use the MDO if it is valid, even if there is no >>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >>> profiling again. >>> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>> wrongly assume that the method is trivial and re-compile it with C1. >>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>> constant getter methods, like those used by the test, as trivial. >>> (4) We should add verification code to avoid/catch such bad tiered level >>> transitions in the future. >>> >>> Solution: >>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>> always compile the (trivial) methods at level 1 and the test fails. >>> >>> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >>> a method consists of a constant push and a return statement only. If so, we >>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>> >>> I filed JDK-8062913 for (4) because the verification code I added still fails >>> with the current implementation due to 2->2 and 3->2 transitions. >>> >>> Testing: >>> - JPRT including failing whitebox test >>> >>> Thanks, >>> Tobias >>> >>> >>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>> [2] First compilation request: >>> InterpreterRuntime::frequency_counter_overflow >>> InterpreterRuntime::frequency_counter_overflow_inner >>> SimpleThresholdPolicy::event >>> AdvancedThresholdPolicy::method_invocation_event >>> AdvancedThresholdPolicy::call_event >>> AdvancedThresholdPolicy::common >>> [is_trivial() returns false because no nmethod available] >>> [next_level = CompLevel_Full_Optimization] >>> SimpleThresholdPolicy::compile >>> SimpleThresholdPolicy::submit_compile >>> CompileBroker::compile_method >>> CompileBroker::compile_method_base >>> CompileBroker::create_compile_task >>> [continue execution in interpreter] >>> >>> [3] Second compilation request: >>> InterpreterRuntime::frequency_counter_overflow >>> InterpreterRuntime::frequency_counter_overflow_inner >>> SimpleThresholdPolicy::event >>> AdvancedThresholdPolicy::method_invocation_event >>> [First compilation finishes here -> >>> !CompileBroker::compilation_is_in_queue()] >>> AdvancedThresholdPolicy::call_event >>> AdvancedThresholdPolicy::common >>> [is_trivial() returns true because nmethod/mdo now available] >>> [next_level = CompLevel_Simple] >>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>> >>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >>> From vladimir.kozlov at oracle.com Fri Nov 7 01:25:52 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 17:25:52 -0800 Subject: RFR(M) : 8059624 : Test task: WhiteBox API for testing segmented codecache feature In-Reply-To: <545BB7D0.5080601@oracle.com> References: <545A4F63.6010008@oracle.com> <545AB8E7.40400@oracle.com> <545BB7D0.5080601@oracle.com> Message-ID: <545C1FA0.4070107@oracle.com> On 11/6/14 10:02 AM, Igor Ignatyev wrote: > updated webrev: http://cr.openjdk.java.net/~iignatyev/8059624/webrev.01/ > 679 lines changed: 643 ins; 2 del; 34 mod; > > Hi Vladimir/Tobias, > > thanks for review. please see my answers inline. > > On 11/06/2014 02:55 AM, Vladimir Kozlov wrote: >> Hi Igor, >> >> codeBlob.hpp >> >> Why you need?: >> >> + // placement >> + inline void* operator new(size_t s, void* p) throw() { return p; } > I need this to execute ctor in WhiteBox::allocateCodeHeap: >> 887 new (blob) BufferBlob("WB::DummyBlob", full_size); > > it's impossible to use default placement new, since BufferBlob overloads > operator new. We do use ::new() in GrowableArray. Will it work in your case? > >> compileBroker.cpp >> >> I think MonitorLockerEx locker() should be before the loop. Then you may >> not need to get lock in WB_UnlockCompilation. > I moved MonitorLockerEx before the loop, but I still need a lock in > UnlockCompilation to call notify_all(); Should it be like next? Otherwise it looks like deadlock. + WB_ENTRY(void, WB_UnlockCompilation(JNIEnv* env, jobject o)) + WhiteBox::compilation_locked = false; + { + MonitorLockerEx mo(Compilation_lock, Mutex::_no_safepoint_check_flag); + mo.notify_all(); + } + WB_END > >> whitebox.cpp >> >> allocateCodeHeap() - what if CodeCache::allocate() failed to allocate? > it will return null, see AllocationCodeHeapTest.java: >> 93 while ((addr = WHITE_BOX.allocateCodeHeap(size, type.id)) >> != 0) { >> 94 blobs.add(addr); >> 95 } But in other cases you don't check: 73 long addr = WHITE_BOX.allocateCodeHeap(SIZE, type.id); 79 WHITE_BOX.freeCodeHeap(addr); And BufferBlob::free() does not check for NULL: void BufferBlob::free(BufferBlob *blob) { ThreadInVMfromUnknown __tiv; // get to VM state in case we block on CodeCache_lock blob->flush(); > >> In WB_GetCodeHeapEntries() checks that blobs array is not empty after >> the loop which add entries. > should I? I thought that blobs.length() will be 0 and blobs.begin() will > equal blobs.end() in this case, won't it? Yes, but I am asking to add short-cut to bailout early to avoid allocation of new array and other code which follows: + ThreadToNativeFromVM ttn(thread); + jobjectArray result = NULL; + jclass clazz = env->FindClass(vmSymbols::java_lang_Object()->as_C_string()); + CHECK_JNI_EXCEPTION_(env, NULL); + result = env->NewObjectArray(blobs.length(), clazz, NULL); > >> You need to prevent sweeping when you processing codecache blobs >> (getCodeHeapEntries). Otherwise pointers from getCodeBlobs() may point >> to dead or overwritten space, consider -Xcomp -XX:+DeoptimizeALot > sweeper does its work after acquiring CodeCache_lock and there is lock > CodeCache before 1st loop, so I introduced struct CodeBlobStub and store > it in GrowableArray instead of CodeBlob. Yes, this works. Thanks, Vladimir >> >> Thanks, >> Vladimir > > On 11/06/2014 12:11 PM, Tobias Hartmann wrote: >> Hi Igor, >> >> looks good (not a reviewer). In addition to Vladimir's comments: >> >> 'WhiteBox::allocateCodeHeap' should be 'allocate_code_heap'. I think >> the name is >> a bit misleading since we don't allocate a CodeHeap but a CodeBlob in a >> CodeHeap. Maybe use 'code_heap_allocate' or 'allocate_code_blob'. Same >> for >> 'WB_AllocateCodeHeap'. > agree, changed it to AllocateCodeBlob/FreeCodeBlob > // I just tried to mimic WB_AllocateMetaspace >> >> In 'BlobType::getAvailable' you always return the NonProfiledCodeHeap if >> CodeCacheSegmentation is enabled. But if we only use the interpreter >> (-Xint), we >> don't have any method code heaps. Also with 'TieredStopAtLevel <= >> CompLevel_simple' we don't need the ProfiledCodeHeap (see >> 'CodeCache::heap_available'). The corresponding tests will hit the >> "heap is >> null" assert in 'CodeCache::allocate'. > I updated 'BlobType::getAvailable' to handle such the situations. > >> On 11/5/14 8:25 AM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev/8059624/webrev.00/ >>> 660 lines changed: 624 ins; 2 del; 34 mod; >>> >>> Hi all, >>> >>> please review the patch which adds new WhiteBox methods needed for >>> better testing SegmentedCodeCache: >>> - perform allocation in code cache >>> - force code cache sweep >>> - lock/unlock compilation >>> - get a segment id for nmethod. >>> besides these methods, the patch also adds a method to get all entries >>> in code heap. >>> >>> changes in product code: >>> - monitor 'Compilation_lock' was added to implement compilation >>> locking/unlocking >>> - align_code_offset function was made a static member of CodeBlob >>> - WhiteBox was made a friend of several classes >>> >>> testing: jprt, new added tests >>> jbs: https://bugs.openjdk.java.net/browse/JDK-8059624 >>> From vladimir.kozlov at oracle.com Fri Nov 7 01:31:16 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 17:31:16 -0800 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm In-Reply-To: <545B6926.8070906@oracle.com> References: <545B6926.8070906@oracle.com> Message-ID: <545C20E4.1040002@oracle.com> I also asked to add it to TEST.groups to execute in JPRT. Thanks, Vladimir On 11/6/14 4:27 AM, Tatiana Pivovarova wrote: > Hi, > > please review the following patch > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 > Webrev: > http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ > > Problem: > Flag "-XX:+EliminateAutoBox" works on server VM only. The test should > work with Client VM too. > > Solution: > Add flag "-XX:+IgnoreUnrecognizedVMOptions." > > Testing: > Manual > > Thanks, > Tatiana > From vladimir.kozlov at oracle.com Fri Nov 7 01:33:20 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 17:33:20 -0800 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint In-Reply-To: <545B761B.2000505@oracle.com> References: <545B761B.2000505@oracle.com> Message-ID: <545C2160.8000501@oracle.com> Looks good. Thanks, Vladimir On 11/6/14 5:22 AM, Nils Eliasson wrote: > Hi, > > Please review this small change. > > The first implementation of compiler diagnostic commands was asserting > on safepoint or lock. The safepoint was always taken since the dcmd by > default block for a safepoint. That is not correct, the lock must be > taken, and a whether we are at a safepoint doesn't matter. > > Tested with jtreg tests in Hotspot and JDK that exercise this code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 > Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ > > Thanks, > Nils From vladimir.kozlov at oracle.com Fri Nov 7 01:42:45 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 17:42:45 -0800 Subject: RFR (XS): 8062950: Bug in locking code when UseOptoBiasInlining is disabled: assert(dmw->is_neutral()) failed: invariant In-Reply-To: <7C9B87B351A4BA4AA9EC95BB418116566ACE6BEC@DEWDFEMB19C.global.corp.sap> References: <7C9B87B351A4BA4AA9EC95BB418116566ACE597E@DEWDFEMB19C.global.corp.sap> <545AB32E.8070402@oracle.com> <7C9B87B351A4BA4AA9EC95BB418116566ACE6BEC@DEWDFEMB19C.global.corp.sap> Message-ID: <545C2395.8040307@oracle.com> I doubt anyone will look on this code in foreseeable future. But to make it simple for backport (8u40) I will go with your original fix. Thanks, Vladimir On 11/6/14 2:40 AM, Doerr, Martin wrote: > Hi Vladimir, > > thanks for replying quickly. > > Are you sure you want the swap_reg_contains_mark flag to get removed? > There's a TODO in front of the changed line: > // TODO: optimize away redundant LDs of obj->mark and improve the markword triage > // order to reduce the number of conditional branches in the most common cases. > // Beware -- there's a subtle invariant that fetch of the markword > // at [FETCH], below, will never observe a biased encoding (*101b). > // If this invariant is not held we risk exclusion (safety) failure. > > So I'm not sure if the flag may be useful again when somebody works on this TODO. > > Best regards, > Martin > > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Donnerstag, 6. November 2014 00:31 > To: Christian Thalinger; Doerr, Martin > Cc: hotspot-compiler-dev at openjdk.java.net; Hotspot dev runtime > Subject: Re: RFR (XS): 8062950: Bug in locking code when UseOptoBiasInlining is disabled: assert(dmw->is_neutral()) failed: invariant > > It is our (Compiler group) code. This problem was introduced with my > changes for RTM locking. > > Martin your changes are good. But you cleanup a bit this code since we > now never put markword to tmpReg before this call? > > Thanks, > Vladimir > > On 11/5/14 3:13 PM, Christian Thalinger wrote: >> I?m not exactly sure who is our biased locking expert these days but I guess it?s someone from the runtime team. CC?ing them. >> >>> On Nov 5, 2014, at 7:38 AM, Doerr, Martin wrote: >>> >>> Hi, >>> >>> we found a bug in MacroAssembler::fast_lock on x86 which shows up when UseOptoBiasInlining is switched off. >>> The problem is that biased_locking_enter is used with swap_reg_contains_mark==true, which is no longer correct after biased_locking_enter was put in front of check for IsInflated. >>> >>> Please review >>> http://cr.openjdk.java.net/~goetz/webrevs/8062950-lockBug/webrev.00/ >>> >>> Best regards, >>> Martin >> From vladimir.kozlov at oracle.com Fri Nov 7 02:45:45 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 06 Nov 2014 18:45:45 -0800 Subject: [9] RFR(M): 8062011: JT_HS/compiler/7068051 uses jre/lib/javaws.jar In-Reply-To: <545B422D.6070207@oracle.com> References: <545B422D.6070207@oracle.com> Message-ID: <545C3259.10004@oracle.com> Hi Tatiana, I assume you reproduced original problem with old JDK and your new test? Please, say that in "Testing:" so that I don't need to ask it each time. You used Test7068051.java 100 times to create foo.jar. It is fine if it still reproduces the problem. Originally it was suggested to use classes from rt.jar but I switched to javaws.jar because it was smaller and still reproduced the problem. Thanks, Vladimir On 11/6/14 1:41 AM, Tatiana Pivovarova wrote: > Hi all, > > please review the patch > > Bug: https://bugs.openjdk.java.net/browse/JDK-8062011 > Webrev: > http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062011/webrev.00/ > > Problem: > test/compiler/7068051 uses jre/lib/javaws.jar which is removed in jigsaw M2 > > Solution: > We create .jar file by ourselves. > > Testing: > Manual > > Thanks, > Tatiana From tobias.hartmann at oracle.com Fri Nov 7 07:21:28 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 07 Nov 2014 08:21:28 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545BD060.9010705@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545BD060.9010705@oracle.com> Message-ID: <545C72F8.3020106@oracle.com> Hi Igor, thanks for looking at this. See comments inline. On 06.11.2014 20:47, Igor Ignatyev wrote: > Hi Tobias, > > On 11/06/2014 01:25 PM, Tobias Hartmann wrote: >> Hi Vladimir, >> >> thanks for the review. >> >> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>> So the fix is add -XX:-TieredCompilation. >>> The rest is fix problems found due to these failure. >> >> Yes, exactly. >> >>> May be we should add a new test (other RFE) which would test compilation of >>> trivial methods in Tiered mode. Current failure shows that we don't test it. >> >> I agree. I filed RFE JDK-8056071 and will take care of it. > you posted wrong cr. > > actually, PavelP is working on such the tests, it's tracked by 8059575 and/or > 8028590. Thanks, the right RFE is JDK-8063096. I closed it as a duplicate of JDK-8028590 because it is not directly related to the segmented code cache test plan. Best, Tobias >> >>> I am not comfortable to rely on data which is set only by C1. This information >>> is accessed in non Tiered mode too. >> >> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if >> TieredCompilation is enabled. Where do you think it is used in non-tiered mode? >> >>> Why not use method->has_loops(), for example? >> >> We also decide based on the number of blocks and 'MethodData::would_profile' >> which is only available if we have a C1 compiled version. >> >> Thanks, >> Tobias >> >>> >>> Thanks, >>> Vladimir >>> >>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>> Hi, >>>> >>>> please review the following patch. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>> >>>> Problem: >>>> The test compiles and deoptimizes a method m multiple times. After each >>>> deoptimization the test checks if m was compiled with C2 and if so increments a >>>> counter. The test assumes that m is compilable until the counter reaches the >>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>> - The first compilation request for m is added to the compile queue. >>>> Although we >>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>> information because due to deoptimization there is no corresponding nmethod >>>> [1]. >>>> We compile at level 4 and execution continues in the interpreter [2]. >>>> - A second compilation request for m is issued and while it is being processed >>>> the first compilation request finishes [3]. Because a nmethod now exists, we >>>> use >>>> the MDO and decide to compile at level 1 since m is trivial. >>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>> >>>> After the following deoptimization the counter is not incremented because the >>>> nmethod was compiled at level 1. As a result m is set to not compilable before >>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>> >>>> Problems with the current implementation: >>>> (1) We should always use the MDO if it is valid, even if there is no >>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >>>> profiling again. >>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>>> wrongly assume that the method is trivial and re-compile it with C1. >>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>> constant getter methods, like those used by the test, as trivial. >>>> (4) We should add verification code to avoid/catch such bad tiered level >>>> transitions in the future. >>>> >>>> Solution: >>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>>> always compile the (trivial) methods at level 1 and the test fails. >>>> >>>> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >>>> a method consists of a constant push and a return statement only. If so, we >>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>> >>>> I filed JDK-8062913 for (4) because the verification code I added still fails >>>> with the current implementation due to 2->2 and 3->2 transitions. >>>> >>>> Testing: >>>> - JPRT including failing whitebox test >>>> >>>> Thanks, >>>> Tobias >>>> >>>> >>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>> [2] First compilation request: >>>> InterpreterRuntime::frequency_counter_overflow >>>> InterpreterRuntime::frequency_counter_overflow_inner >>>> SimpleThresholdPolicy::event >>>> AdvancedThresholdPolicy::method_invocation_event >>>> AdvancedThresholdPolicy::call_event >>>> AdvancedThresholdPolicy::common >>>> [is_trivial() returns false because no nmethod available] >>>> [next_level = CompLevel_Full_Optimization] >>>> SimpleThresholdPolicy::compile >>>> SimpleThresholdPolicy::submit_compile >>>> CompileBroker::compile_method >>>> CompileBroker::compile_method_base >>>> CompileBroker::create_compile_task >>>> [continue execution in interpreter] >>>> >>>> [3] Second compilation request: >>>> InterpreterRuntime::frequency_counter_overflow >>>> InterpreterRuntime::frequency_counter_overflow_inner >>>> SimpleThresholdPolicy::event >>>> AdvancedThresholdPolicy::method_invocation_event >>>> [First compilation finishes here -> >>>> !CompileBroker::compilation_is_in_queue()] >>>> AdvancedThresholdPolicy::call_event >>>> AdvancedThresholdPolicy::common >>>> [is_trivial() returns true because nmethod/mdo now available] >>>> [next_level = CompLevel_Simple] >>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>> >>>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >>>> From tobias.hartmann at oracle.com Fri Nov 7 09:58:43 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 07 Nov 2014 10:58:43 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545C173F.5090403@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545C173F.5090403@oracle.com> Message-ID: <545C97D3.4010007@oracle.com> Hi Vladimir, thanks for the review. On 07.11.2014 01:50, Vladimir Kozlov wrote: > On 11/6/14 2:25 AM, Tobias Hartmann wrote: >> Hi Vladimir, >> >> thanks for the review. >> >> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>> So the fix is add -XX:-TieredCompilation. >>> The rest is fix problems found due to these failure. >> >> Yes, exactly. >> >>> May be we should add a new test (other RFE) which would test compilation of >>> trivial methods in Tiered mode. Current failure shows that we don't test it. >> >> I agree. I filed RFE JDK-8056071 and will take care of it. >> >>> I am not comfortable to rely on data which is set only by C1. This information >>> is accessed in non Tiered mode too. >> >> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if >> TieredCompilation is enabled. Where do you think it is used in non-tiered mode? > > You are right. NonTieredCompPolicy::is_mature() does not use it. > >> >>> Why not use method->has_loops(), for example? >> >> We also decide based on the number of blocks and 'MethodData::would_profile' >> which is only available if we have a C1 compiled version. > > Yes, but next code will do the same and only checking MDO in the last case: > > bool SimpleThresholdPolicy::is_trivial(Method* method) { > if (method->is_accessor() || > method->is_constant_getter()) { > return true; > } > if (method->has_loops() || (method->code_size() >= 15)) { > return false; > } > MethodData* mdo = method->method_data(); > if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && > (method->code_size() < 5 || (mdo->num_blocks() < 4))) { > return true; > } > return false; > } Okay, I changed the code accordingly. > Also mdo->_would_profile will not be set if C1 compilation is bailed out > (Compilation() in c1_Compilation.cpp). So we can't trust it too. > May be mdo->_would_profile should be enum {unknown, no_profile, profile} and we > use it instead of mdo->stats_valid(). I agree. Like this, we also save the additional '_stats_valid' field in MethodData. I added an enum and changed the implementation of 'stats_valid()'. It now returns true if '_would_profile' is not initialized or set to true. New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.01/ Thanks, Tobias > Thanks, > Vladimir > >> >> Thanks, >> Tobias >> >>> >>> Thanks, >>> Vladimir >>> >>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>> Hi, >>>> >>>> please review the following patch. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>> >>>> Problem: >>>> The test compiles and deoptimizes a method m multiple times. After each >>>> deoptimization the test checks if m was compiled with C2 and if so increments a >>>> counter. The test assumes that m is compilable until the counter reaches the >>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>> - The first compilation request for m is added to the compile queue. >>>> Although we >>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>> information because due to deoptimization there is no corresponding nmethod >>>> [1]. >>>> We compile at level 4 and execution continues in the interpreter [2]. >>>> - A second compilation request for m is issued and while it is being processed >>>> the first compilation request finishes [3]. Because a nmethod now exists, we >>>> use >>>> the MDO and decide to compile at level 1 since m is trivial. >>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>> >>>> After the following deoptimization the counter is not incremented because the >>>> nmethod was compiled at level 1. As a result m is set to not compilable before >>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>> >>>> Problems with the current implementation: >>>> (1) We should always use the MDO if it is valid, even if there is no >>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >>>> profiling again. >>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>>> wrongly assume that the method is trivial and re-compile it with C1. >>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>> constant getter methods, like those used by the test, as trivial. >>>> (4) We should add verification code to avoid/catch such bad tiered level >>>> transitions in the future. >>>> >>>> Solution: >>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>>> always compile the (trivial) methods at level 1 and the test fails. >>>> >>>> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >>>> a method consists of a constant push and a return statement only. If so, we >>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>> >>>> I filed JDK-8062913 for (4) because the verification code I added still fails >>>> with the current implementation due to 2->2 and 3->2 transitions. >>>> >>>> Testing: >>>> - JPRT including failing whitebox test >>>> >>>> Thanks, >>>> Tobias >>>> >>>> >>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>> [2] First compilation request: >>>> InterpreterRuntime::frequency_counter_overflow >>>> InterpreterRuntime::frequency_counter_overflow_inner >>>> SimpleThresholdPolicy::event >>>> AdvancedThresholdPolicy::method_invocation_event >>>> AdvancedThresholdPolicy::call_event >>>> AdvancedThresholdPolicy::common >>>> [is_trivial() returns false because no nmethod available] >>>> [next_level = CompLevel_Full_Optimization] >>>> SimpleThresholdPolicy::compile >>>> SimpleThresholdPolicy::submit_compile >>>> CompileBroker::compile_method >>>> CompileBroker::compile_method_base >>>> CompileBroker::create_compile_task >>>> [continue execution in interpreter] >>>> >>>> [3] Second compilation request: >>>> InterpreterRuntime::frequency_counter_overflow >>>> InterpreterRuntime::frequency_counter_overflow_inner >>>> SimpleThresholdPolicy::event >>>> AdvancedThresholdPolicy::method_invocation_event >>>> [First compilation finishes here -> >>>> !CompileBroker::compilation_is_in_queue()] >>>> AdvancedThresholdPolicy::call_event >>>> AdvancedThresholdPolicy::common >>>> [is_trivial() returns true because nmethod/mdo now available] >>>> [next_level = CompLevel_Simple] >>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>> >>>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >>>> From tatiana.pivovarova at oracle.com Fri Nov 7 13:58:11 2014 From: tatiana.pivovarova at oracle.com (Tatiana Pivovarova) Date: Fri, 07 Nov 2014 16:58:11 +0300 Subject: [9] RFR(M): 8062011: JT_HS/compiler/7068051 uses jre/lib/javaws.jar In-Reply-To: <545C3259.10004@oracle.com> References: <545B422D.6070207@oracle.com> <545C3259.10004@oracle.com> Message-ID: <545CCFF3.1090904@oracle.com> Hi Vladimir, Thank you for your review. On 11/07/2014 05:45 AM, Vladimir Kozlov wrote: > Hi Tatiana, > > I assume you reproduced original problem with old JDK and your new > test? Please, say that in "Testing:" so that I don't need to ask it > each time. I reproduced this bug in jdk 7 b147 (test failed). In jdk 7 b148 and latest version test passed. > > You used Test7068051.java 100 times to create foo.jar. It is fine if > it still reproduces the problem. When I use less than 78-80 files test doesn't reproduce, so I chose empirical number "100". Thanks, Tatiana. > > Originally it was suggested to use classes from rt.jar but I switched > to javaws.jar because it was smaller and still reproduced the problem. > > Thanks, > Vladimir > > On 11/6/14 1:41 AM, Tatiana Pivovarova wrote: >> Hi all, >> >> please review the patch >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8062011 >> Webrev: >> http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062011/webrev.00/ >> >> Problem: >> test/compiler/7068051 uses jre/lib/javaws.jar which is removed in >> jigsaw M2 >> >> Solution: >> We create .jar file by ourselves. >> >> Testing: >> Manual >> >> Thanks, >> Tatiana From tatiana.pivovarova at oracle.com Fri Nov 7 14:23:04 2014 From: tatiana.pivovarova at oracle.com (Tatiana Pivovarova) Date: Fri, 07 Nov 2014 17:23:04 +0300 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm In-Reply-To: <545C20E4.1040002@oracle.com> References: <545B6926.8070906@oracle.com> <545C20E4.1040002@oracle.com> Message-ID: <545CD5C8.5060104@oracle.com> Hi Vladimir, I added this test to TEST.groups. Webrev: http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.01/ Thanks, Tatiana On 11/07/2014 04:31 AM, Vladimir Kozlov wrote: > I also asked to add it to TEST.groups to execute in JPRT. > > Thanks, > Vladimir > > On 11/6/14 4:27 AM, Tatiana Pivovarova wrote: >> Hi, >> >> please review the following patch >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 >> Webrev: >> http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ >> >> Problem: >> Flag "-XX:+EliminateAutoBox" works on server VM only. The test should >> work with Client VM too. >> >> Solution: >> Add flag "-XX:+IgnoreUnrecognizedVMOptions." >> >> Testing: >> Manual >> >> Thanks, >> Tatiana >> From vladimir.kozlov at oracle.com Fri Nov 7 16:21:20 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 07 Nov 2014 08:21:20 -0800 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545C97D3.4010007@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545C173F.5090403@oracle.com> <545C97D3.4010007@oracle.com> Message-ID: <545CF180.5050803@oracle.com> You may change behavior in other places where would_profile() is used. Please, verify. Otherwise looks good. Thanks, Vladimir On 11/7/14 1:58 AM, Tobias Hartmann wrote: > Hi Vladimir, > > thanks for the review. > > On 07.11.2014 01:50, Vladimir Kozlov wrote: >> On 11/6/14 2:25 AM, Tobias Hartmann wrote: >>> Hi Vladimir, >>> >>> thanks for the review. >>> >>> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>>> So the fix is add -XX:-TieredCompilation. >>>> The rest is fix problems found due to these failure. >>> >>> Yes, exactly. >>> >>>> May be we should add a new test (other RFE) which would test compilation of >>>> trivial methods in Tiered mode. Current failure shows that we don't test it. >>> >>> I agree. I filed RFE JDK-8056071 and will take care of it. >>> >>>> I am not comfortable to rely on data which is set only by C1. This information >>>> is accessed in non Tiered mode too. >>> >>> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if >>> TieredCompilation is enabled. Where do you think it is used in non-tiered mode? >> >> You are right. NonTieredCompPolicy::is_mature() does not use it. >> >>> >>>> Why not use method->has_loops(), for example? >>> >>> We also decide based on the number of blocks and 'MethodData::would_profile' >>> which is only available if we have a C1 compiled version. >> >> Yes, but next code will do the same and only checking MDO in the last case: >> >> bool SimpleThresholdPolicy::is_trivial(Method* method) { >> if (method->is_accessor() || >> method->is_constant_getter()) { >> return true; >> } >> if (method->has_loops() || (method->code_size() >= 15)) { >> return false; >> } >> MethodData* mdo = method->method_data(); >> if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && >> (method->code_size() < 5 || (mdo->num_blocks() < 4))) { >> return true; >> } >> return false; >> } > > Okay, I changed the code accordingly. > >> Also mdo->_would_profile will not be set if C1 compilation is bailed out >> (Compilation() in c1_Compilation.cpp). So we can't trust it too. >> May be mdo->_would_profile should be enum {unknown, no_profile, profile} and we >> use it instead of mdo->stats_valid(). > > I agree. Like this, we also save the additional '_stats_valid' field in > MethodData. I added an enum and changed the implementation of 'stats_valid()'. > It now returns true if '_would_profile' is not initialized or set to true. > > New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.01/ > > Thanks, > Tobias > >> Thanks, >> Vladimir >> >>> >>> Thanks, >>> Tobias >>> >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>>> Hi, >>>>> >>>>> please review the following patch. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>>> >>>>> Problem: >>>>> The test compiles and deoptimizes a method m multiple times. After each >>>>> deoptimization the test checks if m was compiled with C2 and if so increments a >>>>> counter. The test assumes that m is compilable until the counter reaches the >>>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>>> - The first compilation request for m is added to the compile queue. >>>>> Although we >>>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>>> information because due to deoptimization there is no corresponding nmethod >>>>> [1]. >>>>> We compile at level 4 and execution continues in the interpreter [2]. >>>>> - A second compilation request for m is issued and while it is being processed >>>>> the first compilation request finishes [3]. Because a nmethod now exists, we >>>>> use >>>>> the MDO and decide to compile at level 1 since m is trivial. >>>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>>> >>>>> After the following deoptimization the counter is not incremented because the >>>>> nmethod was compiled at level 1. As a result m is set to not compilable before >>>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>>> >>>>> Problems with the current implementation: >>>>> (1) We should always use the MDO if it is valid, even if there is no >>>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >>>>> profiling again. >>>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >>>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>>>> wrongly assume that the method is trivial and re-compile it with C1. >>>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>>> constant getter methods, like those used by the test, as trivial. >>>>> (4) We should add verification code to avoid/catch such bad tiered level >>>>> transitions in the future. >>>>> >>>>> Solution: >>>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >>>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>>>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>>>> always compile the (trivial) methods at level 1 and the test fails. >>>>> >>>>> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >>>>> a method consists of a constant push and a return statement only. If so, we >>>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>>> >>>>> I filed JDK-8062913 for (4) because the verification code I added still fails >>>>> with the current implementation due to 2->2 and 3->2 transitions. >>>>> >>>>> Testing: >>>>> - JPRT including failing whitebox test >>>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>> >>>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>>> [2] First compilation request: >>>>> InterpreterRuntime::frequency_counter_overflow >>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>> SimpleThresholdPolicy::event >>>>> AdvancedThresholdPolicy::method_invocation_event >>>>> AdvancedThresholdPolicy::call_event >>>>> AdvancedThresholdPolicy::common >>>>> [is_trivial() returns false because no nmethod available] >>>>> [next_level = CompLevel_Full_Optimization] >>>>> SimpleThresholdPolicy::compile >>>>> SimpleThresholdPolicy::submit_compile >>>>> CompileBroker::compile_method >>>>> CompileBroker::compile_method_base >>>>> CompileBroker::create_compile_task >>>>> [continue execution in interpreter] >>>>> >>>>> [3] Second compilation request: >>>>> InterpreterRuntime::frequency_counter_overflow >>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>> SimpleThresholdPolicy::event >>>>> AdvancedThresholdPolicy::method_invocation_event >>>>> [First compilation finishes here -> >>>>> !CompileBroker::compilation_is_in_queue()] >>>>> AdvancedThresholdPolicy::call_event >>>>> AdvancedThresholdPolicy::common >>>>> [is_trivial() returns true because nmethod/mdo now available] >>>>> [next_level = CompLevel_Simple] >>>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>>> >>>>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >>>>> From vladimir.kozlov at oracle.com Fri Nov 7 16:23:14 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 07 Nov 2014 08:23:14 -0800 Subject: [9] RFR(M): 8062011: JT_HS/compiler/7068051 uses jre/lib/javaws.jar In-Reply-To: <545CCFF3.1090904@oracle.com> References: <545B422D.6070207@oracle.com> <545C3259.10004@oracle.com> <545CCFF3.1090904@oracle.com> Message-ID: <545CF1F2.6020300@oracle.com> Thank you for clarification, Tatiana Changes are good. Thanks, Vladimir On 11/7/14 5:58 AM, Tatiana Pivovarova wrote: > Hi Vladimir, > > Thank you for your review. > > On 11/07/2014 05:45 AM, Vladimir Kozlov wrote: >> Hi Tatiana, >> >> I assume you reproduced original problem with old JDK and your new test? Please, say that in "Testing:" so that I >> don't need to ask it each time. > I reproduced this bug in jdk 7 b147 (test failed). In jdk 7 b148 and latest version test passed. > >> >> You used Test7068051.java 100 times to create foo.jar. It is fine if it still reproduces the problem. > When I use less than 78-80 files test doesn't reproduce, so I chose empirical number "100". > > Thanks, > Tatiana. >> >> Originally it was suggested to use classes from rt.jar but I switched to javaws.jar because it was smaller and still >> reproduced the problem. >> >> Thanks, >> Vladimir >> >> On 11/6/14 1:41 AM, Tatiana Pivovarova wrote: >>> Hi all, >>> >>> please review the patch >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8062011 >>> Webrev: >>> http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062011/webrev.00/ >>> >>> Problem: >>> test/compiler/7068051 uses jre/lib/javaws.jar which is removed in jigsaw M2 >>> >>> Solution: >>> We create .jar file by ourselves. >>> >>> Testing: >>> Manual >>> >>> Thanks, >>> Tatiana > From vladimir.kozlov at oracle.com Fri Nov 7 16:30:11 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 07 Nov 2014 08:30:11 -0800 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm In-Reply-To: <545CD5C8.5060104@oracle.com> References: <545B6926.8070906@oracle.com> <545C20E4.1040002@oracle.com> <545CD5C8.5060104@oracle.com> Message-ID: <545CF393.8010507@oracle.com> Good. Thanks, Vladimir On 11/7/14 6:23 AM, Tatiana Pivovarova wrote: > Hi Vladimir, > > I added this test to TEST.groups. > Webrev: http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.01/ > > Thanks, > Tatiana > > On 11/07/2014 04:31 AM, Vladimir Kozlov wrote: >> I also asked to add it to TEST.groups to execute in JPRT. >> >> Thanks, >> Vladimir >> >> On 11/6/14 4:27 AM, Tatiana Pivovarova wrote: >>> Hi, >>> >>> please review the following patch >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 >>> Webrev: >>> http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ >>> >>> Problem: >>> Flag "-XX:+EliminateAutoBox" works on server VM only. The test should >>> work with Client VM too. >>> >>> Solution: >>> Add flag "-XX:+IgnoreUnrecognizedVMOptions." >>> >>> Testing: >>> Manual >>> >>> Thanks, >>> Tatiana >>> > From david.r.chase at oracle.com Fri Nov 7 21:14:38 2014 From: david.r.chase at oracle.com (David Chase) Date: Fri, 7 Nov 2014 16:14:38 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> Message-ID: <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> New webrev: bug: https://bugs.openjdk.java.net/browse/JDK-8013267 webrevs: http://cr.openjdk.java.net/~drchase/8013267/jdk.06/ http://cr.openjdk.java.net/~drchase/8013267/hotspot.06/ Changes since last: 1) refactored to put ClassData under java.lang.invoke.MemberName 2) split the data structure into two parts; handshake with JVM uses a linked list, which makes for a simpler backout-if-race, and Java side continues to use the simple sorted array. This should allow easier use of (for example) fancier data structures (like ConcurrentHashMap) if this later proves necessary. 3) Cleaned up symbol references in the new hotspot code to go through vmSymbols. 4) renamed oldCapacity to oldSize 5) ran two different benchmarks and saw no change in performance. a) nashorn ScriptTest (see https://bugs.openjdk.java.net/browse/JDK-8014288 ) b) JMH microbenchmarks (see bug comments for details) And it continues to pass the previously-failing tests, as well as the new test which has been added to hotspot/test/compiler/jsr292 . David On 2014-11-04, at 3:54 PM, David Chase wrote: > I?m working on the initial benchmarking, and so far this arrangement (with synchronization > and binary search for lookup, lots of barriers and linear cost insertion) has not yet been any > slower. > > I am nonetheless tempted by the 2-tables solution, because I think the simpler JVM-side > interface that it allows is desirable. > > David > > On 2014-11-04, at 11:48 AM, Peter Levart wrote: > >> On 11/04/2014 04:19 PM, David Chase wrote: >>> On 2014-11-04, at 5:07 AM, Peter Levart wrote: >>>> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? >>> It can?t be an identityHashMap, because we are interning member names. >> >> I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. >> >>> In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. >>> One possibility would be to use two data structures, one for interning, the other for communication with the VM. >>> Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, >>> and the synchronization dance is much simpler. >>> >>> For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: >>> >>> mn = resolve(args) >>> // deal with any errors >>> mn? = chm.get(mn) >>> if (mn? != null) return mn? // hoped-for-common-case >>> >>> synchronized (something) { >>> mn? = chm.get(mn) >>> if (mn? != null) return mn? >>> txn_class = mn.getDeclaringClass() >>> >>> while (true) { >>> redef_count = txn_class.redefCount() >>> mn = resolve(args) >>> >>> shared_array.add(mn); >>> // barrier, because we are a paranoid >>> if (redef_count = redef_count.redefCount()) { >>> chm.add(mn); // safe to publish to other Java threads. >>> return mn; >>> } >>> shared_array.drop_last(); // Try again >>> } >>> } >>> >>> (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). >> >> Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. >> >> A combination of the two approaches is also possible: >> >> - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) >> - use ConcurrentHashMap for interning. >> >> Regards, Peter >> >>> >>> David >>> >>>>> And another way to view this is that we?re now quibbling about performance, when we still >>>>> have an existing correctness problem that this patch solves, so maybe we should just get this >>>>> done and then file an RFE. >>>> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >>>> >>>> Regards, Peter >>>> >>>>> David > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From roland.westrelin at oracle.com Fri Nov 7 22:17:35 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 7 Nov 2014 23:17:35 +0100 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <545BF25E.7010505@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> <545BF25E.7010505@oracle.com> Message-ID: <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> Vladimir, Thanks for the discussion, suggestions and fixes. Here is an updated webrev: http://cr.openjdk.java.net/~roland/8054478/webrev.01/ Roland. > On Nov 6, 2014, at 11:12 PM, Vladimir Kozlov wrote: > > On 11/6/14 7:39 AM, Roland Westrelin wrote: >> >>> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. >> >> I?ll send an updated webrev. >> >>>>>>> opaquenode.cpp >>>>>>> >>>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>>> >>>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>>> >>>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>>> >>>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >>> >>> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. >> >> (I know you don?t like when I remove the text from previous emails but that was really getting confusing) > > At least leave webrev link so I don't need to search for it in previous mails. > >> >> PhiNode::Value() has this code: >> >> // Check for trip-counted loop. If so, be smarter. >> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >> if( l && l->can_be_counted_loop(phase) && >> ((const Node*)l->phi() == this) ) { // Trip counted loop! >> // protect against init_trip() or limit() returning NULL >> const Node *init = l->init_trip(); >> const Node *limit = l->limit(); >> if( init != NULL && limit != NULL && l->stride_is_con() ) { >> const TypeInt *lo = init ->bottom_type()->isa_int(); >> const TypeInt *hi = limit->bottom_type()->isa_int(); >> if( lo && hi ) { // Dying loops might have TOP here >> int stride = l->stride_con(); >> if( stride < 0 ) { // Down-counter loop >> const TypeInt *tmp = lo; lo = hi; hi = tmp; >> stride = -stride; >> } >> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >> return TypeInt::make(lo->_lo,hi->_hi,3); >> } >> } >> } >> >> That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. > > My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: > > // Step B2: Build a zero-trip guard for the main-loop. After leaving the > // pre-loop, the main-loop may not execute at all. Later in life this > // zero-trip guard will become the minimum-trip guard when we unroll > // the main-loop. > Node *min_opaq = new Opaque1Node(C, limit); > Node *min_cmp = new CmpINode( pre_incr, min_opaq ); > > > But you are talking about pre-loop exit check: > > // Step B4: Shorten the pre-loop to run only 1 iteration (for now). > // RCE and alignment may change this later. > Node *cmp_end = pre_end->cmp_node(); > assert( cmp_end->in(2) == limit, "" ); > Node *pre_limit = new AddINode( init, stride ); > > // Save the original loop limit in this Opaque1 node for > // use by range check elimination. > Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); > > > But in this case you can add check (cl->limit() == this) to be clear what you are looking for. > Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. > > Thanks, > Vladimir > >> >> In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. >> >> Roland. From vladimir.kozlov at oracle.com Fri Nov 7 23:08:51 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 07 Nov 2014 15:08:51 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> <545BF25E.7010505@oracle.com> <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> Message-ID: <545D5103.9020309@oracle.com> This looks good but I just realized that what you are doing in Opaque1Node::Ideal() we usually do in PhaseIterGVN::add_users_to_worklist(). For example you can do there (for Phi): + uint use_op = use->Opcode(); if( use->is_Cmp() ) { // Enable CMP/BOOL optimization add_users_to_worklist(use); // Put Bool on worklist - // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the - // phi merging either 0 or 1 onto the worklist if (use->outcnt() > 0) { Node* bol = use->raw_out(0); if (bol->outcnt() > 0) { Node* iff = bol->raw_out(0); if (use_op == Op_CmpI && iff->is_CountedLoopEnd() && n->Opcode() == Op_CastII) { + // If this opaque node feeds into the limit condition of a + // CountedLoop, we need to process the Phi node for the + // induction variable: the range of values taken by the Phi is + // known now and so its type is also known. + CountedLoopEndNode* cle = iff->as_CountedLoopEnd(); + if (cle->limit() == this) { + _worklist.push(cle->phi()); + } - if (iff->outcnt() == 2) { + } else if (iff->outcnt() == 2) { + // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the + // phi merging either 0 or 1 onto the worklist Thanks, Vladimir On 11/7/14 2:17 PM, Roland Westrelin wrote: > Vladimir, > > Thanks for the discussion, suggestions and fixes. Here is an updated webrev: > > http://cr.openjdk.java.net/~roland/8054478/webrev.01/ > > Roland. > >> On Nov 6, 2014, at 11:12 PM, Vladimir Kozlov wrote: >> >> On 11/6/14 7:39 AM, Roland Westrelin wrote: >>> >>>> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. >>> >>> I?ll send an updated webrev. >>> >>>>>>>> opaquenode.cpp >>>>>>>> >>>>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>>>> >>>>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>>>> >>>>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>>>> >>>>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >>>> >>>> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. >>> >>> (I know you don?t like when I remove the text from previous emails but that was really getting confusing) >> >> At least leave webrev link so I don't need to search for it in previous mails. >> >>> >>> PhiNode::Value() has this code: >>> >>> // Check for trip-counted loop. If so, be smarter. >>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>> if( l && l->can_be_counted_loop(phase) && >>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>> // protect against init_trip() or limit() returning NULL >>> const Node *init = l->init_trip(); >>> const Node *limit = l->limit(); >>> if( init != NULL && limit != NULL && l->stride_is_con() ) { >>> const TypeInt *lo = init ->bottom_type()->isa_int(); >>> const TypeInt *hi = limit->bottom_type()->isa_int(); >>> if( lo && hi ) { // Dying loops might have TOP here >>> int stride = l->stride_con(); >>> if( stride < 0 ) { // Down-counter loop >>> const TypeInt *tmp = lo; lo = hi; hi = tmp; >>> stride = -stride; >>> } >>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>> return TypeInt::make(lo->_lo,hi->_hi,3); >>> } >>> } >>> } >>> >>> That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. >> >> My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: >> >> // Step B2: Build a zero-trip guard for the main-loop. After leaving the >> // pre-loop, the main-loop may not execute at all. Later in life this >> // zero-trip guard will become the minimum-trip guard when we unroll >> // the main-loop. >> Node *min_opaq = new Opaque1Node(C, limit); >> Node *min_cmp = new CmpINode( pre_incr, min_opaq ); >> >> >> But you are talking about pre-loop exit check: >> >> // Step B4: Shorten the pre-loop to run only 1 iteration (for now). >> // RCE and alignment may change this later. >> Node *cmp_end = pre_end->cmp_node(); >> assert( cmp_end->in(2) == limit, "" ); >> Node *pre_limit = new AddINode( init, stride ); >> >> // Save the original loop limit in this Opaque1 node for >> // use by range check elimination. >> Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); >> >> >> But in this case you can add check (cl->limit() == this) to be clear what you are looking for. >> Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. >> >> Thanks, >> Vladimir >> >>> >>> In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. >>> >>> Roland. > From igor.veresov at oracle.com Sat Nov 8 08:23:26 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Fri, 7 Nov 2014 22:23:26 -1000 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <545C97D3.4010007@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545C173F.5090403@oracle.com> <545C97D3.4010007@oracle.com> Message-ID: <9F6CF186-841D-41A6-AC4C-0D9541BA07AC@oracle.com> In your change _would_profile is still a bool, I guess it should be of the enum type above or at least an int? igor On Nov 6, 2014, at 11:58 PM, Tobias Hartmann wrote: > Hi Vladimir, > > thanks for the review. > > On 07.11.2014 01:50, Vladimir Kozlov wrote: >> On 11/6/14 2:25 AM, Tobias Hartmann wrote: >>> Hi Vladimir, >>> >>> thanks for the review. >>> >>> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>>> So the fix is add -XX:-TieredCompilation. >>>> The rest is fix problems found due to these failure. >>> >>> Yes, exactly. >>> >>>> May be we should add a new test (other RFE) which would test compilation of >>>> trivial methods in Tiered mode. Current failure shows that we don't test it. >>> >>> I agree. I filed RFE JDK-8056071 and will take care of it. >>> >>>> I am not comfortable to rely on data which is set only by C1. This information >>>> is accessed in non Tiered mode too. >>> >>> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if >>> TieredCompilation is enabled. Where do you think it is used in non-tiered mode? >> >> You are right. NonTieredCompPolicy::is_mature() does not use it. >> >>> >>>> Why not use method->has_loops(), for example? >>> >>> We also decide based on the number of blocks and 'MethodData::would_profile' >>> which is only available if we have a C1 compiled version. >> >> Yes, but next code will do the same and only checking MDO in the last case: >> >> bool SimpleThresholdPolicy::is_trivial(Method* method) { >> if (method->is_accessor() || >> method->is_constant_getter()) { >> return true; >> } >> if (method->has_loops() || (method->code_size() >= 15)) { >> return false; >> } >> MethodData* mdo = method->method_data(); >> if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && >> (method->code_size() < 5 || (mdo->num_blocks() < 4))) { >> return true; >> } >> return false; >> } > > Okay, I changed the code accordingly. > >> Also mdo->_would_profile will not be set if C1 compilation is bailed out >> (Compilation() in c1_Compilation.cpp). So we can't trust it too. >> May be mdo->_would_profile should be enum {unknown, no_profile, profile} and we >> use it instead of mdo->stats_valid(). > > I agree. Like this, we also save the additional '_stats_valid' field in > MethodData. I added an enum and changed the implementation of 'stats_valid()'. > It now returns true if '_would_profile' is not initialized or set to true. > > New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.01/ > > Thanks, > Tobias > >> Thanks, >> Vladimir >> >>> >>> Thanks, >>> Tobias >>> >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>>> Hi, >>>>> >>>>> please review the following patch. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>>> >>>>> Problem: >>>>> The test compiles and deoptimizes a method m multiple times. After each >>>>> deoptimization the test checks if m was compiled with C2 and if so increments a >>>>> counter. The test assumes that m is compilable until the counter reaches the >>>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>>> - The first compilation request for m is added to the compile queue. >>>>> Although we >>>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>>> information because due to deoptimization there is no corresponding nmethod >>>>> [1]. >>>>> We compile at level 4 and execution continues in the interpreter [2]. >>>>> - A second compilation request for m is issued and while it is being processed >>>>> the first compilation request finishes [3]. Because a nmethod now exists, we >>>>> use >>>>> the MDO and decide to compile at level 1 since m is trivial. >>>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>>> >>>>> After the following deoptimization the counter is not incremented because the >>>>> nmethod was compiled at level 1. As a result m is set to not compilable before >>>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>>> >>>>> Problems with the current implementation: >>>>> (1) We should always use the MDO if it is valid, even if there is no >>>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or start >>>>> profiling again. >>>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a method >>>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>>>> wrongly assume that the method is trivial and re-compile it with C1. >>>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>>> constant getter methods, like those used by the test, as trivial. >>>>> (4) We should add verification code to avoid/catch such bad tiered level >>>>> transitions in the future. >>>>> >>>>> Solution: >>>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that determines >>>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>>>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>>>> always compile the (trivial) methods at level 1 and the test fails. >>>>> >>>>> To fix (3) I added the method 'Method::is_constant_getter()' that determines if >>>>> a method consists of a constant push and a return statement only. If so, we >>>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>>> >>>>> I filed JDK-8062913 for (4) because the verification code I added still fails >>>>> with the current implementation due to 2->2 and 3->2 transitions. >>>>> >>>>> Testing: >>>>> - JPRT including failing whitebox test >>>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>> >>>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>>> [2] First compilation request: >>>>> InterpreterRuntime::frequency_counter_overflow >>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>> SimpleThresholdPolicy::event >>>>> AdvancedThresholdPolicy::method_invocation_event >>>>> AdvancedThresholdPolicy::call_event >>>>> AdvancedThresholdPolicy::common >>>>> [is_trivial() returns false because no nmethod available] >>>>> [next_level = CompLevel_Full_Optimization] >>>>> SimpleThresholdPolicy::compile >>>>> SimpleThresholdPolicy::submit_compile >>>>> CompileBroker::compile_method >>>>> CompileBroker::compile_method_base >>>>> CompileBroker::create_compile_task >>>>> [continue execution in interpreter] >>>>> >>>>> [3] Second compilation request: >>>>> InterpreterRuntime::frequency_counter_overflow >>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>> SimpleThresholdPolicy::event >>>>> AdvancedThresholdPolicy::method_invocation_event >>>>> [First compilation finishes here -> >>>>> !CompileBroker::compilation_is_in_queue()] >>>>> AdvancedThresholdPolicy::call_event >>>>> AdvancedThresholdPolicy::common >>>>> [is_trivial() returns true because nmethod/mdo now available] >>>>> [next_level = CompLevel_Simple] >>>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>>> >>>>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.ignatyev at oracle.com Sat Nov 8 18:41:55 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sat, 08 Nov 2014 21:41:55 +0300 Subject: RFR(M) : 8059624 : Test task: WhiteBox API for testing segmented codecache feature In-Reply-To: <545C1FA0.4070107@oracle.com> References: <545A4F63.6010008@oracle.com> <545AB8E7.40400@oracle.com> <545BB7D0.5080601@oracle.com> <545C1FA0.4070107@oracle.com> Message-ID: <545E63F3.2080001@oracle.com> http://cr.openjdk.java.net/~iignatyev/8059624/webrev.02/ On 11/07/2014 04:25 AM, Vladimir Kozlov wrote: > On 11/6/14 10:02 AM, Igor Ignatyev wrote: >> updated webrev: http://cr.openjdk.java.net/~iignatyev/8059624/webrev.01/ >> 679 lines changed: 643 ins; 2 del; 34 mod; >> >> Hi Vladimir/Tobias, >> >> thanks for review. please see my answers inline. >> >> On 11/06/2014 02:55 AM, Vladimir Kozlov wrote: >>> Hi Igor, >>> >>> codeBlob.hpp >>> >>> Why you need?: >>> >>> + // placement >>> + inline void* operator new(size_t s, void* p) throw() { return p; } >> I need this to execute ctor in WhiteBox::allocateCodeHeap: >>> 887 new (blob) BufferBlob("WB::DummyBlob", full_size); >> >> it's impossible to use default placement new, since BufferBlob overloads >> operator new. > > We do use ::new() in GrowableArray. Will it work in your case? y, it works, thanks. > >> >>> compileBroker.cpp >>> >>> I think MonitorLockerEx locker() should be before the loop. Then you may >>> not need to get lock in WB_UnlockCompilation. >> I moved MonitorLockerEx before the loop, but I still need a lock in >> UnlockCompilation to call notify_all(); > > Should it be like next? Otherwise it looks like deadlock. > > + WB_ENTRY(void, WB_UnlockCompilation(JNIEnv* env, jobject o)) > + WhiteBox::compilation_locked = false; > + { > + MonitorLockerEx mo(Compilation_lock, > Mutex::_no_safepoint_check_flag); > + mo.notify_all(); > + } > + WB_END there's no deadlock since Monitor::wait release the lock. >> >>> whitebox.cpp >>> >>> allocateCodeHeap() - what if CodeCache::allocate() failed to allocate? >> it will return null, see AllocationCodeHeapTest.java: >>> 93 while ((addr = WHITE_BOX.allocateCodeHeap(size, type.id)) >>> != 0) { >>> 94 blobs.add(addr); >>> 95 } > > But in other cases you don't check: > > 73 long addr = WHITE_BOX.allocateCodeHeap(SIZE, type.id); > > 79 WHITE_BOX.freeCodeHeap(addr); > > And BufferBlob::free() does not check for NULL: > > void BufferBlob::free(BufferBlob *blob) { > ThreadInVMfromUnknown __tiv; // get to VM state in case we block on > CodeCache_lock > blob->flush(); I've added checks that addr isn't 0 to the tests. >> >>> In WB_GetCodeHeapEntries() checks that blobs array is not empty after >>> the loop which add entries. >> should I? I thought that blobs.length() will be 0 and blobs.begin() will >> equal blobs.end() in this case, won't it? > > Yes, but I am asking to add short-cut to bailout early to avoid > allocation of new array and other code which follows: > > + ThreadToNativeFromVM ttn(thread); > + jobjectArray result = NULL; > + jclass clazz = > env->FindClass(vmSymbols::java_lang_Object()->as_C_string()); > + CHECK_JNI_EXCEPTION_(env, NULL); > + result = env->NewObjectArray(blobs.length(), clazz, NULL); fixed >> >>> You need to prevent sweeping when you processing codecache blobs >>> (getCodeHeapEntries). Otherwise pointers from getCodeBlobs() may point >>> to dead or overwritten space, consider -Xcomp -XX:+DeoptimizeALot >> sweeper does its work after acquiring CodeCache_lock and there is lock >> CodeCache before 1st loop, so I introduced struct CodeBlobStub and store >> it in GrowableArray instead of CodeBlob. > > Yes, this works. > > Thanks, > Vladimir > >>> >>> Thanks, >>> Vladimir >> >> On 11/06/2014 12:11 PM, Tobias Hartmann wrote: >>> Hi Igor, >>> >>> looks good (not a reviewer). In addition to Vladimir's comments: >>> >>> 'WhiteBox::allocateCodeHeap' should be 'allocate_code_heap'. I think >>> the name is >>> a bit misleading since we don't allocate a CodeHeap but a CodeBlob in a >>> CodeHeap. Maybe use 'code_heap_allocate' or 'allocate_code_blob'. Same >>> for >>> 'WB_AllocateCodeHeap'. >> agree, changed it to AllocateCodeBlob/FreeCodeBlob >> // I just tried to mimic WB_AllocateMetaspace >>> >>> In 'BlobType::getAvailable' you always return the NonProfiledCodeHeap if >>> CodeCacheSegmentation is enabled. But if we only use the interpreter >>> (-Xint), we >>> don't have any method code heaps. Also with 'TieredStopAtLevel <= >>> CompLevel_simple' we don't need the ProfiledCodeHeap (see >>> 'CodeCache::heap_available'). The corresponding tests will hit the >>> "heap is >>> null" assert in 'CodeCache::allocate'. >> I updated 'BlobType::getAvailable' to handle such the situations. >> >>> On 11/5/14 8:25 AM, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev/8059624/webrev.00/ >>>> 660 lines changed: 624 ins; 2 del; 34 mod; >>>> >>>> Hi all, >>>> >>>> please review the patch which adds new WhiteBox methods needed for >>>> better testing SegmentedCodeCache: >>>> - perform allocation in code cache >>>> - force code cache sweep >>>> - lock/unlock compilation >>>> - get a segment id for nmethod. >>>> besides these methods, the patch also adds a method to get all entries >>>> in code heap. >>>> >>>> changes in product code: >>>> - monitor 'Compilation_lock' was added to implement compilation >>>> locking/unlocking >>>> - align_code_offset function was made a static member of CodeBlob >>>> - WhiteBox was made a friend of several classes >>>> >>>> testing: jprt, new added tests >>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8059624 >>>> From vladimir.kozlov at oracle.com Sat Nov 8 19:35:14 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sat, 08 Nov 2014 11:35:14 -0800 Subject: RFR(M) : 8059624 : Test task: WhiteBox API for testing segmented codecache feature In-Reply-To: <545E63F3.2080001@oracle.com> References: <545A4F63.6010008@oracle.com> <545AB8E7.40400@oracle.com> <545BB7D0.5080601@oracle.com> <545C1FA0.4070107@oracle.com> <545E63F3.2080001@oracle.com> Message-ID: <545E7072.1020406@oracle.com> You missed next null check case in AllocationCodeBlobTest.java: 87 addr = WHITE_BOX.allocateCodeBlob(SIZE, type.id); 88 long secondAllocation = getUsage(); 89 Asserts.assertEQ(firstAllocation, secondAllocation); 90 91 WHITE_BOX.freeCodeBlob(addr); Otherwise it is good. Thanks, Vladimir On 11/8/14 10:41 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8059624/webrev.02/ > > On 11/07/2014 04:25 AM, Vladimir Kozlov wrote: >> On 11/6/14 10:02 AM, Igor Ignatyev wrote: >>> updated webrev: http://cr.openjdk.java.net/~iignatyev/8059624/webrev.01/ >>> 679 lines changed: 643 ins; 2 del; 34 mod; >>> >>> Hi Vladimir/Tobias, >>> >>> thanks for review. please see my answers inline. >>> >>> On 11/06/2014 02:55 AM, Vladimir Kozlov wrote: >>>> Hi Igor, >>>> >>>> codeBlob.hpp >>>> >>>> Why you need?: >>>> >>>> + // placement >>>> + inline void* operator new(size_t s, void* p) throw() { return p; } >>> I need this to execute ctor in WhiteBox::allocateCodeHeap: >>>> 887 new (blob) BufferBlob("WB::DummyBlob", full_size); >>> >>> it's impossible to use default placement new, since BufferBlob overloads >>> operator new. >> >> We do use ::new() in GrowableArray. Will it work in your case? > y, it works, thanks. >> >>> >>>> compileBroker.cpp >>>> >>>> I think MonitorLockerEx locker() should be before the loop. Then you may >>>> not need to get lock in WB_UnlockCompilation. >>> I moved MonitorLockerEx before the loop, but I still need a lock in >>> UnlockCompilation to call notify_all(); >> >> Should it be like next? Otherwise it looks like deadlock. >> >> + WB_ENTRY(void, WB_UnlockCompilation(JNIEnv* env, jobject o)) >> + WhiteBox::compilation_locked = false; >> + { >> + MonitorLockerEx mo(Compilation_lock, >> Mutex::_no_safepoint_check_flag); >> + mo.notify_all(); >> + } >> + WB_END > there's no deadlock since Monitor::wait release the lock. >>> >>>> whitebox.cpp >>>> >>>> allocateCodeHeap() - what if CodeCache::allocate() failed to allocate? >>> it will return null, see AllocationCodeHeapTest.java: >>>> 93 while ((addr = WHITE_BOX.allocateCodeHeap(size, type.id)) >>>> != 0) { >>>> 94 blobs.add(addr); >>>> 95 } >> >> But in other cases you don't check: >> >> 73 long addr = WHITE_BOX.allocateCodeHeap(SIZE, type.id); >> >> 79 WHITE_BOX.freeCodeHeap(addr); >> >> And BufferBlob::free() does not check for NULL: >> >> void BufferBlob::free(BufferBlob *blob) { >> ThreadInVMfromUnknown __tiv; // get to VM state in case we block on >> CodeCache_lock >> blob->flush(); > I've added checks that addr isn't 0 to the tests. >>> >>>> In WB_GetCodeHeapEntries() checks that blobs array is not empty after >>>> the loop which add entries. >>> should I? I thought that blobs.length() will be 0 and blobs.begin() will >>> equal blobs.end() in this case, won't it? >> >> Yes, but I am asking to add short-cut to bailout early to avoid >> allocation of new array and other code which follows: >> >> + ThreadToNativeFromVM ttn(thread); >> + jobjectArray result = NULL; >> + jclass clazz = >> env->FindClass(vmSymbols::java_lang_Object()->as_C_string()); >> + CHECK_JNI_EXCEPTION_(env, NULL); >> + result = env->NewObjectArray(blobs.length(), clazz, NULL); > fixed >>> >>>> You need to prevent sweeping when you processing codecache blobs >>>> (getCodeHeapEntries). Otherwise pointers from getCodeBlobs() may point >>>> to dead or overwritten space, consider -Xcomp -XX:+DeoptimizeALot >>> sweeper does its work after acquiring CodeCache_lock and there is lock >>> CodeCache before 1st loop, so I introduced struct CodeBlobStub and store >>> it in GrowableArray instead of CodeBlob. >> >> Yes, this works. >> >> Thanks, >> Vladimir >> >>>> >>>> Thanks, >>>> Vladimir >>> >>> On 11/06/2014 12:11 PM, Tobias Hartmann wrote: >>>> Hi Igor, >>>> >>>> looks good (not a reviewer). In addition to Vladimir's comments: >>>> >>>> 'WhiteBox::allocateCodeHeap' should be 'allocate_code_heap'. I think >>>> the name is >>>> a bit misleading since we don't allocate a CodeHeap but a CodeBlob in a >>>> CodeHeap. Maybe use 'code_heap_allocate' or 'allocate_code_blob'. Same >>>> for >>>> 'WB_AllocateCodeHeap'. >>> agree, changed it to AllocateCodeBlob/FreeCodeBlob >>> // I just tried to mimic WB_AllocateMetaspace >>>> >>>> In 'BlobType::getAvailable' you always return the NonProfiledCodeHeap if >>>> CodeCacheSegmentation is enabled. But if we only use the interpreter >>>> (-Xint), we >>>> don't have any method code heaps. Also with 'TieredStopAtLevel <= >>>> CompLevel_simple' we don't need the ProfiledCodeHeap (see >>>> 'CodeCache::heap_available'). The corresponding tests will hit the >>>> "heap is >>>> null" assert in 'CodeCache::allocate'. >>> I updated 'BlobType::getAvailable' to handle such the situations. >>> >>>> On 11/5/14 8:25 AM, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev/8059624/webrev.00/ >>>>> 660 lines changed: 624 ins; 2 del; 34 mod; >>>>> >>>>> Hi all, >>>>> >>>>> please review the patch which adds new WhiteBox methods needed for >>>>> better testing SegmentedCodeCache: >>>>> - perform allocation in code cache >>>>> - force code cache sweep >>>>> - lock/unlock compilation >>>>> - get a segment id for nmethod. >>>>> besides these methods, the patch also adds a method to get all entries >>>>> in code heap. >>>>> >>>>> changes in product code: >>>>> - monitor 'Compilation_lock' was added to implement compilation >>>>> locking/unlocking >>>>> - align_code_offset function was made a static member of CodeBlob >>>>> - WhiteBox was made a friend of several classes >>>>> >>>>> testing: jprt, new added tests >>>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8059624 >>>>> From tobias.hartmann at oracle.com Mon Nov 10 06:44:21 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 10 Nov 2014 07:44:21 +0100 Subject: RFR(M) : 8059624 : Test task: WhiteBox API for testing segmented codecache feature In-Reply-To: <545E63F3.2080001@oracle.com> References: <545A4F63.6010008@oracle.com> <545AB8E7.40400@oracle.com> <545BB7D0.5080601@oracle.com> <545C1FA0.4070107@oracle.com> <545E63F3.2080001@oracle.com> Message-ID: <54605EC5.3010201@oracle.com> Hi Igor, looks good (not a reviewer). Best, Tobias On 08.11.2014 19:41, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8059624/webrev.02/ > > On 11/07/2014 04:25 AM, Vladimir Kozlov wrote: >> On 11/6/14 10:02 AM, Igor Ignatyev wrote: >>> updated webrev: http://cr.openjdk.java.net/~iignatyev/8059624/webrev.01/ >>> 679 lines changed: 643 ins; 2 del; 34 mod; >>> >>> Hi Vladimir/Tobias, >>> >>> thanks for review. please see my answers inline. >>> >>> On 11/06/2014 02:55 AM, Vladimir Kozlov wrote: >>>> Hi Igor, >>>> >>>> codeBlob.hpp >>>> >>>> Why you need?: >>>> >>>> + // placement >>>> + inline void* operator new(size_t s, void* p) throw() { return p; } >>> I need this to execute ctor in WhiteBox::allocateCodeHeap: >>>> 887 new (blob) BufferBlob("WB::DummyBlob", full_size); >>> >>> it's impossible to use default placement new, since BufferBlob overloads >>> operator new. >> >> We do use ::new() in GrowableArray. Will it work in your case? > y, it works, thanks. >> >>> >>>> compileBroker.cpp >>>> >>>> I think MonitorLockerEx locker() should be before the loop. Then you may >>>> not need to get lock in WB_UnlockCompilation. >>> I moved MonitorLockerEx before the loop, but I still need a lock in >>> UnlockCompilation to call notify_all(); >> >> Should it be like next? Otherwise it looks like deadlock. >> >> + WB_ENTRY(void, WB_UnlockCompilation(JNIEnv* env, jobject o)) >> + WhiteBox::compilation_locked = false; >> + { >> + MonitorLockerEx mo(Compilation_lock, >> Mutex::_no_safepoint_check_flag); >> + mo.notify_all(); >> + } >> + WB_END > there's no deadlock since Monitor::wait release the lock. >>> >>>> whitebox.cpp >>>> >>>> allocateCodeHeap() - what if CodeCache::allocate() failed to allocate? >>> it will return null, see AllocationCodeHeapTest.java: >>>> 93 while ((addr = WHITE_BOX.allocateCodeHeap(size, type.id)) >>>> != 0) { >>>> 94 blobs.add(addr); >>>> 95 } >> >> But in other cases you don't check: >> >> 73 long addr = WHITE_BOX.allocateCodeHeap(SIZE, type.id); >> >> 79 WHITE_BOX.freeCodeHeap(addr); >> >> And BufferBlob::free() does not check for NULL: >> >> void BufferBlob::free(BufferBlob *blob) { >> ThreadInVMfromUnknown __tiv; // get to VM state in case we block on >> CodeCache_lock >> blob->flush(); > I've added checks that addr isn't 0 to the tests. >>> >>>> In WB_GetCodeHeapEntries() checks that blobs array is not empty after >>>> the loop which add entries. >>> should I? I thought that blobs.length() will be 0 and blobs.begin() will >>> equal blobs.end() in this case, won't it? >> >> Yes, but I am asking to add short-cut to bailout early to avoid >> allocation of new array and other code which follows: >> >> + ThreadToNativeFromVM ttn(thread); >> + jobjectArray result = NULL; >> + jclass clazz = >> env->FindClass(vmSymbols::java_lang_Object()->as_C_string()); >> + CHECK_JNI_EXCEPTION_(env, NULL); >> + result = env->NewObjectArray(blobs.length(), clazz, NULL); > fixed >>> >>>> You need to prevent sweeping when you processing codecache blobs >>>> (getCodeHeapEntries). Otherwise pointers from getCodeBlobs() may point >>>> to dead or overwritten space, consider -Xcomp -XX:+DeoptimizeALot >>> sweeper does its work after acquiring CodeCache_lock and there is lock >>> CodeCache before 1st loop, so I introduced struct CodeBlobStub and store >>> it in GrowableArray instead of CodeBlob. >> >> Yes, this works. >> >> Thanks, >> Vladimir >> >>>> >>>> Thanks, >>>> Vladimir >>> >>> On 11/06/2014 12:11 PM, Tobias Hartmann wrote: >>>> Hi Igor, >>>> >>>> looks good (not a reviewer). In addition to Vladimir's comments: >>>> >>>> 'WhiteBox::allocateCodeHeap' should be 'allocate_code_heap'. I think >>>> the name is >>>> a bit misleading since we don't allocate a CodeHeap but a CodeBlob in a >>>> CodeHeap. Maybe use 'code_heap_allocate' or 'allocate_code_blob'. Same >>>> for >>>> 'WB_AllocateCodeHeap'. >>> agree, changed it to AllocateCodeBlob/FreeCodeBlob >>> // I just tried to mimic WB_AllocateMetaspace >>>> >>>> In 'BlobType::getAvailable' you always return the NonProfiledCodeHeap if >>>> CodeCacheSegmentation is enabled. But if we only use the interpreter >>>> (-Xint), we >>>> don't have any method code heaps. Also with 'TieredStopAtLevel <= >>>> CompLevel_simple' we don't need the ProfiledCodeHeap (see >>>> 'CodeCache::heap_available'). The corresponding tests will hit the >>>> "heap is >>>> null" assert in 'CodeCache::allocate'. >>> I updated 'BlobType::getAvailable' to handle such the situations. >>> >>>> On 11/5/14 8:25 AM, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev/8059624/webrev.00/ >>>>> 660 lines changed: 624 ins; 2 del; 34 mod; >>>>> >>>>> Hi all, >>>>> >>>>> please review the patch which adds new WhiteBox methods needed for >>>>> better testing SegmentedCodeCache: >>>>> - perform allocation in code cache >>>>> - force code cache sweep >>>>> - lock/unlock compilation >>>>> - get a segment id for nmethod. >>>>> besides these methods, the patch also adds a method to get all entries >>>>> in code heap. >>>>> >>>>> changes in product code: >>>>> - monitor 'Compilation_lock' was added to implement compilation >>>>> locking/unlocking >>>>> - align_code_offset function was made a static member of CodeBlob >>>>> - WhiteBox was made a friend of several classes >>>>> >>>>> testing: jprt, new added tests >>>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8059624 >>>>> From goetz.lindenmaier at sap.com Mon Nov 10 07:40:35 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 10 Nov 2014 07:40:35 +0000 Subject: gc bugs after 8060252: JDK-7173584 compiler changes regress SPECjvm2008 on SPARC Message-ID: <4295855A5C1DE049A61835A1887419CC2CF26315@DEWDFEMB12A.global.corp.sap> Hi, on ppc, I see immediate bugs in g1: ppc_vm/bin/java -XX:+UseG1GC -XX:SurvivorRatio=4 -classpath .../benchmarks/jvm98/ SPECjvm98All .../benchmarks/jvm98/ jvm98.log.txt jvm98.result.txt javac # Internal Error (/sapmnt/home1/d045726/oJ/g1Bug-hs-comp/src/share/vm/oops/oop.inline.hpp:199), pid=1554, tid=4398079689280 # assert(check_obj_alignment(result)) failed: address not aligned: 0x00000000baadbabe V [libjvm.so+0xaa5150] report_vm_error(char const*, int, char const*, char const*)+0xdc V [libjvm.so+0x78e738] oopDesc::decode_heap_oop_not_null(unsigned int)+0x11c V [libjvm.so+0x78e7f8] oopDesc::decode_heap_oop(unsigned int)+0x6c V [libjvm.so+0xc41238] void G1SATBCardTableModRefBS::write_ref_field_pre_static(unsigned int*, oopDesc*)+0x7c V [libjvm.so+0xc412cc] void G1SATBCardTableModRefBS::inline_write_ref_field_pre(unsigned int*, oopDesc*)+0x40 V [libjvm.so+0xc41354] G1SATBCardTableModRefBS::write_ref_field_pre_work(unsigned int*, oopDesc*)+0x44 V [libjvm.so+0x8dadcc] void BarrierSet::write_ref_field_pre(unsigned int*, oopDesc*)+0xac V [libjvm.so+0x1195044] void ObjArrayKlass::do_copy(arrayOopDesc*, unsigned int*, arrayOopDesc*, unsigned int*, int, Thread*)+0x3f4 V [libjvm.so+0x1191c68] ObjArrayKlass::copy_array(arrayOopDesc*, int, arrayOopDesc*, int, int, Thread*)+0x27c V [libjvm.so+0xeac84c] JVM_ArrayCopy+0xff51fc1c J 17 java.lang.System.arraycopy(Ljava/lang/Object;ILjava/lang/Object;II)V (0 bytes) @ 0x00000400021bd104 [0x00000400021bd000+0x104] j spec.benchmarks._213_javac.Parser.exprArgs(I)[Lspec/benchmarks/_213_javac/Expression;+23 j spec.benchmarks._213_javac.Parser.parseMethodExpression(Lspec/benchmarks/_213_javac/Expression;Lspec/benchmarks/_213_javac/Identifier;)Lspec/benchmarks/_213_javac/Expression;+69 j spec.benchmarks._213_javac.Parser.parseExpression()Lspec/benchmarks/_213_javac/Expression;+426 Did anybody see similar problems? Best regards, Goetz. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.hartmann at oracle.com Mon Nov 10 08:37:58 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 10 Nov 2014 09:37:58 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <9F6CF186-841D-41A6-AC4C-0D9541BA07AC@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545C173F.5090403@oracle.com> <545C97D3.4010007@oracle.com> <9F6CF186-841D-41A6-AC4C-0D9541BA07AC@oracle.com> Message-ID: <54607966.7080205@oracle.com> Vladimir, Igor, thanks for the reviews. On 07.11.2014 17:21, Vladimir Kozlov wrote: > You may change behavior in other places where would_profile() is used. Please, verify. Otherwise looks good Would_profile() is only used by the 'common' methods of the Simple and AdvancedThresholdPolicies to decide if we need more profiling or should switch to full optimization. In the current implementation _would_profile is initialized to false and therefore would_profile() returns false for methods that were never compiled with C1. The fixed implementation returns true in this case. However, the 'common' methods only invoke would_profile() from level 2 and 3, i.e. if a C1 compiled version exists. There should be no behavioural difference. > Thanks, > Vladimir On 08.11.2014 09:23, Igor Veresov wrote: > In your change _would_profile is still a bool, I guess it should be of the enum > type above or at least an int? Thanks! I missed that. New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.02/ Thanks, Tobias > igor > > On Nov 6, 2014, at 11:58 PM, Tobias Hartmann > wrote: > >> Hi Vladimir, >> >> thanks for the review. >> >> On 07.11.2014 01:50, Vladimir Kozlov wrote: >>> On 11/6/14 2:25 AM, Tobias Hartmann wrote: >>>> Hi Vladimir, >>>> >>>> thanks for the review. >>>> >>>> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>>>> So the fix is add -XX:-TieredCompilation. >>>>> The rest is fix problems found due to these failure. >>>> >>>> Yes, exactly. >>>> >>>>> May be we should add a new test (other RFE) which would test compilation of >>>>> trivial methods in Tiered mode. Current failure shows that we don't test it. >>>> >>>> I agree. I filed RFE JDK-8056071 and will take care of it. >>>> >>>>> I am not comfortable to rely on data which is set only by C1. This information >>>>> is accessed in non Tiered mode too. >>>> >>>> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if >>>> TieredCompilation is enabled. Where do you think it is used in non-tiered mode? >>> >>> You are right. NonTieredCompPolicy::is_mature() does not use it. >>> >>>> >>>>> Why not use method->has_loops(), for example? >>>> >>>> We also decide based on the number of blocks and 'MethodData::would_profile' >>>> which is only available if we have a C1 compiled version. >>> >>> Yes, but next code will do the same and only checking MDO in the last case: >>> >>> bool SimpleThresholdPolicy::is_trivial(Method* method) { >>> if (method->is_accessor() || >>> method->is_constant_getter()) { >>> return true; >>> } >>> if (method->has_loops() || (method->code_size() >= 15)) { >>> return false; >>> } >>> MethodData* mdo = method->method_data(); >>> if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && >>> (method->code_size() < 5 || (mdo->num_blocks() < 4))) { >>> return true; >>> } >>> return false; >>> } >> >> Okay, I changed the code accordingly. >> >>> Also mdo->_would_profile will not be set if C1 compilation is bailed out >>> (Compilation() in c1_Compilation.cpp). So we can't trust it too. >>> May be mdo->_would_profile should be enum {unknown, no_profile, profile} and we >>> use it instead of mdo->stats_valid(). >> >> I agree. Like this, we also save the additional '_stats_valid' field in >> MethodData. I added an enum and changed the implementation of 'stats_valid()'. >> It now returns true if '_would_profile' is not initialized or set to true. >> >> New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.01/ >> >> Thanks, >> Tobias >> >>> Thanks, >>> Vladimir >>> >>>> >>>> Thanks, >>>> Tobias >>>> >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>>>> Hi, >>>>>> >>>>>> please review the following patch. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>>>> >>>>>> Problem: >>>>>> The test compiles and deoptimizes a method m multiple times. After each >>>>>> deoptimization the test checks if m was compiled with C2 and if so >>>>>> increments a >>>>>> counter. The test assumes that m is compilable until the counter reaches the >>>>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>>>> - The first compilation request for m is added to the compile queue. >>>>>> Although we >>>>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>>>> information because due to deoptimization there is no corresponding nmethod >>>>>> [1]. >>>>>> We compile at level 4 and execution continues in the interpreter [2]. >>>>>> - A second compilation request for m is issued and while it is being processed >>>>>> the first compilation request finishes [3]. Because a nmethod now exists, we >>>>>> use >>>>>> the MDO and decide to compile at level 1 since m is trivial. >>>>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>>>> >>>>>> After the following deoptimization the counter is not incremented because the >>>>>> nmethod was compiled at level 1. As a result m is set to not compilable before >>>>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>>>> >>>>>> Problems with the current implementation: >>>>>> (1) We should always use the MDO if it is valid, even if there is no >>>>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or >>>>>> start >>>>>> profiling again. >>>>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a >>>>>> method >>>>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>>>>> wrongly assume that the method is trivial and re-compile it with C1. >>>>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>>>> constant getter methods, like those used by the test, as trivial. >>>>>> (4) We should add verification code to avoid/catch such bad tiered level >>>>>> transitions in the future. >>>>>> >>>>>> Solution: >>>>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that >>>>>> determines >>>>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>>>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>>>>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>>>>> always compile the (trivial) methods at level 1 and the test fails. >>>>>> >>>>>> To fix (3) I added the method 'Method::is_constant_getter()' that >>>>>> determines if >>>>>> a method consists of a constant push and a return statement only. If so, we >>>>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>>>> >>>>>> I filed JDK-8062913 for (4) because the verification code I added still fails >>>>>> with the current implementation due to 2->2 and 3->2 transitions. >>>>>> >>>>>> Testing: >>>>>> - JPRT including failing whitebox test >>>>>> >>>>>> Thanks, >>>>>> Tobias >>>>>> >>>>>> >>>>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>>>> [2] First compilation request: >>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>> SimpleThresholdPolicy::event >>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>> AdvancedThresholdPolicy::call_event >>>>>> AdvancedThresholdPolicy::common >>>>>> [is_trivial() returns false because no nmethod available] >>>>>> [next_level = CompLevel_Full_Optimization] >>>>>> SimpleThresholdPolicy::compile >>>>>> SimpleThresholdPolicy::submit_compile >>>>>> CompileBroker::compile_method >>>>>> CompileBroker::compile_method_base >>>>>> CompileBroker::create_compile_task >>>>>> [continue execution in interpreter] >>>>>> >>>>>> [3] Second compilation request: >>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>> SimpleThresholdPolicy::event >>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>> [First compilation finishes here -> >>>>>> !CompileBroker::compilation_is_in_queue()] >>>>>> AdvancedThresholdPolicy::call_event >>>>>> AdvancedThresholdPolicy::common >>>>>> [is_trivial() returns true because nmethod/mdo now available] >>>>>> [next_level = CompLevel_Simple] >>>>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>>>> >>>>>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log > From tatiana.pivovarova at oracle.com Mon Nov 10 09:24:12 2014 From: tatiana.pivovarova at oracle.com (Tatiana Pivovarova) Date: Mon, 10 Nov 2014 12:24:12 +0300 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm In-Reply-To: <545CF393.8010507@oracle.com> References: <545B6926.8070906@oracle.com> <545C20E4.1040002@oracle.com> <545CD5C8.5060104@oracle.com> <545CF393.8010507@oracle.com> Message-ID: <5460843C.2070706@oracle.com> Hi, Thank you for your review! Tatiana On 11/07/2014 07:30 PM, Vladimir Kozlov wrote: > Good. > > Thanks, > Vladimir > > On 11/7/14 6:23 AM, Tatiana Pivovarova wrote: >> Hi Vladimir, >> >> I added this test to TEST.groups. >> Webrev: >> http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.01/ >> >> Thanks, >> Tatiana >> >> On 11/07/2014 04:31 AM, Vladimir Kozlov wrote: >>> I also asked to add it to TEST.groups to execute in JPRT. >>> >>> Thanks, >>> Vladimir >>> >>> On 11/6/14 4:27 AM, Tatiana Pivovarova wrote: >>>> Hi, >>>> >>>> please review the following patch >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 >>>> Webrev: >>>> http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ >>>> >>>> Problem: >>>> Flag "-XX:+EliminateAutoBox" works on server VM only. The test should >>>> work with Client VM too. >>>> >>>> Solution: >>>> Add flag "-XX:+IgnoreUnrecognizedVMOptions." >>>> >>>> Testing: >>>> Manual >>>> >>>> Thanks, >>>> Tatiana >>>> >> From tatiana.pivovarova at oracle.com Mon Nov 10 09:25:20 2014 From: tatiana.pivovarova at oracle.com (Tatiana Pivovarova) Date: Mon, 10 Nov 2014 12:25:20 +0300 Subject: [9] RFR(M): 8062011: JT_HS/compiler/7068051 uses jre/lib/javaws.jar In-Reply-To: <545CF1F2.6020300@oracle.com> References: <545B422D.6070207@oracle.com> <545C3259.10004@oracle.com> <545CCFF3.1090904@oracle.com> <545CF1F2.6020300@oracle.com> Message-ID: <54608480.7080100@oracle.com> Hi Vladimir And thanks for this review! Tatiana On 11/07/2014 07:23 PM, Vladimir Kozlov wrote: > Thank you for clarification, Tatiana > > Changes are good. > > Thanks, > Vladimir > > On 11/7/14 5:58 AM, Tatiana Pivovarova wrote: >> Hi Vladimir, >> >> Thank you for your review. >> >> On 11/07/2014 05:45 AM, Vladimir Kozlov wrote: >>> Hi Tatiana, >>> >>> I assume you reproduced original problem with old JDK and your new >>> test? Please, say that in "Testing:" so that I >>> don't need to ask it each time. >> I reproduced this bug in jdk 7 b147 (test failed). In jdk 7 b148 and >> latest version test passed. >> >>> >>> You used Test7068051.java 100 times to create foo.jar. It is fine if >>> it still reproduces the problem. >> When I use less than 78-80 files test doesn't reproduce, so I chose >> empirical number "100". >> >> Thanks, >> Tatiana. >>> >>> Originally it was suggested to use classes from rt.jar but I >>> switched to javaws.jar because it was smaller and still >>> reproduced the problem. >>> >>> Thanks, >>> Vladimir >>> >>> On 11/6/14 1:41 AM, Tatiana Pivovarova wrote: >>>> Hi all, >>>> >>>> please review the patch >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8062011 >>>> Webrev: >>>> http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062011/webrev.00/ >>>> >>>> Problem: >>>> test/compiler/7068051 uses jre/lib/javaws.jar which is removed in >>>> jigsaw M2 >>>> >>>> Solution: >>>> We create .jar file by ourselves. >>>> >>>> Testing: >>>> Manual >>>> >>>> Thanks, >>>> Tatiana >> From tatiana.pivovarova at oracle.com Mon Nov 10 09:28:18 2014 From: tatiana.pivovarova at oracle.com (Tatiana Pivovarova) Date: Mon, 10 Nov 2014 12:28:18 +0300 Subject: RFR(XS) : 8062742: compiler/EliminateAutoBox/UnsignedLoads.java fails with client vm In-Reply-To: <20141106134839.GD27319@rbackman> References: <545B6926.8070906@oracle.com> <20141106134839.GD27319@rbackman> Message-ID: <54608532.2060104@oracle.com> Hi Albert, Ricard, Thank you for your reviews! Tatiana On 11/06/2014 04:48 PM, Rickard B?ckman wrote: > Reviewed. > > /R > > On 11/06, Tatiana Pivovarova wrote: >> Hi, >> >> please review the following patch >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8062742 >> Webrev: http://cr.openjdk.java.net/~iignatyev/tpivovarova/8062742/webrev.00/ >> >> Problem: >> Flag "-XX:+EliminateAutoBox" works on server VM only. The test >> should work with Client VM too. >> >> Solution: >> Add flag "-XX:+IgnoreUnrecognizedVMOptions." >> >> Testing: >> Manual >> >> Thanks, >> Tatiana >> From nils.eliasson at oracle.com Mon Nov 10 09:43:28 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Mon, 10 Nov 2014 10:43:28 +0100 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint In-Reply-To: <545B761B.2000505@oracle.com> References: <545B761B.2000505@oracle.com> Message-ID: <546088C0.6030907@oracle.com> Hi, I need a second review on this change. Regards, Nils On 2014-11-06 14:22, Nils Eliasson wrote: > Hi, > > Please review this small change. > > The first implementation of compiler diagnostic commands was asserting > on safepoint or lock. The safepoint was always taken since the dcmd by > default block for a safepoint. That is not correct, the lock must be > taken, and a whether we are at a safepoint doesn't matter. > > Tested with jtreg tests in Hotspot and JDK that exercise this code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 > Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ > > Thanks, > Nils From albert.noll at oracle.com Mon Nov 10 10:10:21 2014 From: albert.noll at oracle.com (Albert Noll) Date: Mon, 10 Nov 2014 11:10:21 +0100 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint In-Reply-To: <545B761B.2000505@oracle.com> References: <545B761B.2000505@oracle.com> Message-ID: <54608F0D.1050101@oracle.com> Hi Nils, On 11/06/2014 02:22 PM, Nils Eliasson wrote: > Hi, > > Please review this small change. > > The first implementation of compiler diagnostic commands was asserting > on safepoint or lock. The safepoint was always taken since the dcmd by > default block for a safepoint. That is not correct, the lock must be > taken, and a whether we are at a safepoint doesn't matter. > Could you please explain why we always need to take the lock? Thanks, Albert > Tested with jtreg tests in Hotspot and JDK that exercise this code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 > Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ > > Thanks, > Nils From nils.eliasson at oracle.com Mon Nov 10 10:38:01 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Mon, 10 Nov 2014 11:38:01 +0100 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint In-Reply-To: <54608F0D.1050101@oracle.com> References: <545B761B.2000505@oracle.com> <54608F0D.1050101@oracle.com> Message-ID: <54609589.9090008@oracle.com> Hi Albert, On 2014-11-10 11:10, Albert Noll wrote: > Hi Nils, > > > On 11/06/2014 02:22 PM, Nils Eliasson wrote: >> Hi, >> >> Please review this small change. >> >> The first implementation of compiler diagnostic commands was >> asserting on safepoint or lock. The safepoint was always taken since >> the dcmd by default block for a safepoint. That is not correct, the >> lock must be taken, and a whether we are at a safepoint doesn't matter. >> > Could you please explain why we always need to take the lock? Mutation of the code cache can happen at any time, for example when a new compiled method is installed. The codelist dcmd iterates the code cache and might see a dangling pointer. The code cache stats dcmd only print statisticss, so it won't break but can see bad data. Regards, Nils > > Thanks, > Albert >> Tested with jtreg tests in Hotspot and JDK that exercise this code. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 >> Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ >> >> Thanks, >> Nils > From albert.noll at oracle.com Mon Nov 10 11:57:09 2014 From: albert.noll at oracle.com (Albert Noll) Date: Mon, 10 Nov 2014 12:57:09 +0100 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint In-Reply-To: <54609589.9090008@oracle.com> References: <545B761B.2000505@oracle.com> <54608F0D.1050101@oracle.com> <54609589.9090008@oracle.com> Message-ID: <5460A815.2010403@oracle.com> Hi Nils, thanks for your response. On 11/10/2014 11:38 AM, Nils Eliasson wrote: > Hi Albert, > > On 2014-11-10 11:10, Albert Noll wrote: >> Hi Nils, >> >> >> On 11/06/2014 02:22 PM, Nils Eliasson wrote: >>> Hi, >>> >>> Please review this small change. >>> >>> The first implementation of compiler diagnostic commands was >>> asserting on safepoint or lock. The safepoint was always taken since >>> the dcmd by default block for a safepoint. That is not correct, the >>> lock must be taken, and a whether we are at a safepoint doesn't matter. >>> >> Could you please explain why we always need to take the lock? > > Mutation of the code cache can happen at any time, for example when a > new compiled method is installed. The codelist dcmd iterates the code > cache and might see a dangling pointer. The code cache stats dcmd only > print statisticss, so it won't break but can see bad data. > Here are two things that I don't understand: 1) assert_locked_or_safepoint(CodeCache_lock) checks if we are at a safepoint and/or we own the 'CodeCache_lock'. If we are at a safepoint, how can another thread possibly modify data structures in the code cache? 2) Why is it safe to use assert_locked_or_safepoint(CodeCache_lock) in so many places except for CodeCache::print_codelist(outputStream* st)? 3) To make sure that no other thread modifies the code cache, wouldn't you you have to make sure that every modification to the code cache is guarded by assert(CodeCache_lock->owned_by_self(), "Checking") instead of assert_locked_or_safepoint(CodeCache_lock)? I.e., a lock is only useful if every modification to the code cache is guarded by that lock. Best, Albert > Regards, > Nils > >> >> Thanks, >> Albert >>> Tested with jtreg tests in Hotspot and JDK that exercise this code. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 >>> Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ >>> >>> Thanks, >>> Nils >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nils.eliasson at oracle.com Mon Nov 10 14:36:30 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Mon, 10 Nov 2014 15:36:30 +0100 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint In-Reply-To: <5460A815.2010403@oracle.com> References: <545B761B.2000505@oracle.com> <54608F0D.1050101@oracle.com> <54609589.9090008@oracle.com> <5460A815.2010403@oracle.com> Message-ID: <5460CD6E.1060407@oracle.com> Hi Albert, On 2014-11-10 12:57, Albert Noll wrote: > Hi Nils, > [...] >> > Here are two things that I don't understand: > 1) assert_locked_or_safepoint(CodeCache_lock) checks if we are at a > safepoint and/or we own the 'CodeCache_lock'. If we are at a > safepoint, how can another thread possibly modify data structures in > the code cache? Looking at the code one more time the contract seem to be lock and no_safepoint for mutators. But that is not what's asserted. We lock with _no_safepoint_check - but that just means we don't verify anything. We want to verify the no_safepoint. Many of the asserts are also not really helpful. Example: CodeCache::allocate - assert_locked_or_safepoint(CodeCache_lock) - The lock guarantees correctness for multiple threads, the safepoint don't. But we have locking at all the code paths that call on CodeCache::allocate, and we never do it at a safepoint. So it works ok. Writers should do: assert_locked_and_verify_no_safepoint() Readers: assert locked_or_safepoint(). > > 2) Why is it safe to use assert_locked_or_safepoint(CodeCache_lock) in > so many places except for CodeCache::print_codelist(outputStream* st)? On second thought is should be enough with assert_locked_or_safepoint(CodeCache_lock) on all readers if we assert locked_and_verify_not_at_safepoint on all writers. So the old implementation was correct. > > 3) To make sure that no other thread modifies the code cache, wouldn't > you you have to make sure that every modification to the code cache is > guarded by assert(CodeCache_lock->owned_by_self(), "Checking") instead > of assert_locked_or_safepoint(CodeCache_lock)? I.e., a lock is only > useful if every modification to the code cache is guarded by that lock. We should assert on lock and verify not at safepoint at the places modifying the code cache if we want to keep the contract that it is ok to read at safepoints. Thanks, Nils > Best, > Albert > >> Regards, >> Nils >> >>> >>> Thanks, >>> Albert >>>> Tested with jtreg tests in Hotspot and JDK that exercise this code. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 >>>> Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ >>>> >>>> Thanks, >>>> Nils >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kozlov at oracle.com Mon Nov 10 15:44:17 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 10 Nov 2014 07:44:17 -0800 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <54607966.7080205@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545C173F.5090403@oracle.com> <545C97D3.4010007@oracle.com> <9F6CF186-841D-41A6-AC4C-0D9541BA07AC@oracle.com> <54607966.7080205@oracle.com> Message-ID: <5460DD51.90202@oracle.com> Thank you, Tobias, for verification. This looks good. Thanks, Vladimir On 11/10/14 12:37 AM, Tobias Hartmann wrote: > Vladimir, Igor, thanks for the reviews. > > On 07.11.2014 17:21, Vladimir Kozlov wrote: >> You may change behavior in other places where would_profile() is used. Please, verify. Otherwise looks good > > Would_profile() is only used by the 'common' methods of the Simple and > AdvancedThresholdPolicies to decide if we need more profiling or should switch > to full optimization. > In the current implementation _would_profile is initialized to false and > therefore would_profile() returns false for methods that were never compiled > with C1. The fixed implementation returns true in this case. > > However, the 'common' methods only invoke would_profile() from level 2 and 3, > i.e. if a C1 compiled version exists. There should be no behavioural difference. > >> Thanks, >> Vladimir > > On 08.11.2014 09:23, Igor Veresov wrote: >> In your change _would_profile is still a bool, I guess it should be of the enum >> type above or at least an int? > > Thanks! I missed that. > > New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.02/ > > Thanks, > Tobias > >> igor >> >> On Nov 6, 2014, at 11:58 PM, Tobias Hartmann > > wrote: >> >>> Hi Vladimir, >>> >>> thanks for the review. >>> >>> On 07.11.2014 01:50, Vladimir Kozlov wrote: >>>> On 11/6/14 2:25 AM, Tobias Hartmann wrote: >>>>> Hi Vladimir, >>>>> >>>>> thanks for the review. >>>>> >>>>> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>>>>> So the fix is add -XX:-TieredCompilation. >>>>>> The rest is fix problems found due to these failure. >>>>> >>>>> Yes, exactly. >>>>> >>>>>> May be we should add a new test (other RFE) which would test compilation of >>>>>> trivial methods in Tiered mode. Current failure shows that we don't test it. >>>>> >>>>> I agree. I filed RFE JDK-8056071 and will take care of it. >>>>> >>>>>> I am not comfortable to rely on data which is set only by C1. This information >>>>>> is accessed in non Tiered mode too. >>>>> >>>>> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if >>>>> TieredCompilation is enabled. Where do you think it is used in non-tiered mode? >>>> >>>> You are right. NonTieredCompPolicy::is_mature() does not use it. >>>> >>>>> >>>>>> Why not use method->has_loops(), for example? >>>>> >>>>> We also decide based on the number of blocks and 'MethodData::would_profile' >>>>> which is only available if we have a C1 compiled version. >>>> >>>> Yes, but next code will do the same and only checking MDO in the last case: >>>> >>>> bool SimpleThresholdPolicy::is_trivial(Method* method) { >>>> if (method->is_accessor() || >>>> method->is_constant_getter()) { >>>> return true; >>>> } >>>> if (method->has_loops() || (method->code_size() >= 15)) { >>>> return false; >>>> } >>>> MethodData* mdo = method->method_data(); >>>> if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && >>>> (method->code_size() < 5 || (mdo->num_blocks() < 4))) { >>>> return true; >>>> } >>>> return false; >>>> } >>> >>> Okay, I changed the code accordingly. >>> >>>> Also mdo->_would_profile will not be set if C1 compilation is bailed out >>>> (Compilation() in c1_Compilation.cpp). So we can't trust it too. >>>> May be mdo->_would_profile should be enum {unknown, no_profile, profile} and we >>>> use it instead of mdo->stats_valid(). >>> >>> I agree. Like this, we also save the additional '_stats_valid' field in >>> MethodData. I added an enum and changed the implementation of 'stats_valid()'. >>> It now returns true if '_would_profile' is not initialized or set to true. >>> >>> New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.01/ >>> >>> Thanks, >>> Tobias >>> >>>> Thanks, >>>> Vladimir >>>> >>>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>>>>> Hi, >>>>>>> >>>>>>> please review the following patch. >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>>>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>>>>> >>>>>>> Problem: >>>>>>> The test compiles and deoptimizes a method m multiple times. After each >>>>>>> deoptimization the test checks if m was compiled with C2 and if so >>>>>>> increments a >>>>>>> counter. The test assumes that m is compilable until the counter reaches the >>>>>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>>>>> - The first compilation request for m is added to the compile queue. >>>>>>> Although we >>>>>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>>>>> information because due to deoptimization there is no corresponding nmethod >>>>>>> [1]. >>>>>>> We compile at level 4 and execution continues in the interpreter [2]. >>>>>>> - A second compilation request for m is issued and while it is being processed >>>>>>> the first compilation request finishes [3]. Because a nmethod now exists, we >>>>>>> use >>>>>>> the MDO and decide to compile at level 1 since m is trivial. >>>>>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>>>>> >>>>>>> After the following deoptimization the counter is not incremented because the >>>>>>> nmethod was compiled at level 1. As a result m is set to not compilable before >>>>>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>>>>> >>>>>>> Problems with the current implementation: >>>>>>> (1) We should always use the MDO if it is valid, even if there is no >>>>>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or >>>>>>> start >>>>>>> profiling again. >>>>>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a >>>>>>> method >>>>>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>>>>>> wrongly assume that the method is trivial and re-compile it with C1. >>>>>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>>>>> constant getter methods, like those used by the test, as trivial. >>>>>>> (4) We should add verification code to avoid/catch such bad tiered level >>>>>>> transitions in the future. >>>>>>> >>>>>>> Solution: >>>>>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that >>>>>>> determines >>>>>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>>>>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>>>>>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>>>>>> always compile the (trivial) methods at level 1 and the test fails. >>>>>>> >>>>>>> To fix (3) I added the method 'Method::is_constant_getter()' that >>>>>>> determines if >>>>>>> a method consists of a constant push and a return statement only. If so, we >>>>>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>>>>> >>>>>>> I filed JDK-8062913 for (4) because the verification code I added still fails >>>>>>> with the current implementation due to 2->2 and 3->2 transitions. >>>>>>> >>>>>>> Testing: >>>>>>> - JPRT including failing whitebox test >>>>>>> >>>>>>> Thanks, >>>>>>> Tobias >>>>>>> >>>>>>> >>>>>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>>>>> [2] First compilation request: >>>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>>> SimpleThresholdPolicy::event >>>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>>> AdvancedThresholdPolicy::call_event >>>>>>> AdvancedThresholdPolicy::common >>>>>>> [is_trivial() returns false because no nmethod available] >>>>>>> [next_level = CompLevel_Full_Optimization] >>>>>>> SimpleThresholdPolicy::compile >>>>>>> SimpleThresholdPolicy::submit_compile >>>>>>> CompileBroker::compile_method >>>>>>> CompileBroker::compile_method_base >>>>>>> CompileBroker::create_compile_task >>>>>>> [continue execution in interpreter] >>>>>>> >>>>>>> [3] Second compilation request: >>>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>>> SimpleThresholdPolicy::event >>>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>>> [First compilation finishes here -> >>>>>>> !CompileBroker::compilation_is_in_queue()] >>>>>>> AdvancedThresholdPolicy::call_event >>>>>>> AdvancedThresholdPolicy::common >>>>>>> [is_trivial() returns true because nmethod/mdo now available] >>>>>>> [next_level = CompLevel_Simple] >>>>>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>>>>> >>>>>>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>>>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>>>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >> From tobias.hartmann at oracle.com Mon Nov 10 16:18:47 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 10 Nov 2014 17:18:47 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <5460DD51.90202@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545C173F.5090403@oracle.com> <545C97D3.4010007@oracle.com> <9F6CF186-841D-41A6-AC4C-0D9541BA07AC@oracle.com> <54607966.7080205@oracle.com> <5460DD51.90202@oracle.com> Message-ID: <5460E567.9050804@oracle.com> Thanks, Vladimir. Best, Tobias On 10.11.2014 16:44, Vladimir Kozlov wrote: > Thank you, Tobias, for verification. > This looks good. > > Thanks, > Vladimir > > On 11/10/14 12:37 AM, Tobias Hartmann wrote: >> Vladimir, Igor, thanks for the reviews. >> >> On 07.11.2014 17:21, Vladimir Kozlov wrote: >>> You may change behavior in other places where would_profile() is used. >>> Please, verify. Otherwise looks good >> >> Would_profile() is only used by the 'common' methods of the Simple and >> AdvancedThresholdPolicies to decide if we need more profiling or should switch >> to full optimization. >> In the current implementation _would_profile is initialized to false and >> therefore would_profile() returns false for methods that were never compiled >> with C1. The fixed implementation returns true in this case. >> >> However, the 'common' methods only invoke would_profile() from level 2 and 3, >> i.e. if a C1 compiled version exists. There should be no behavioural difference. >> >>> Thanks, >>> Vladimir >> >> On 08.11.2014 09:23, Igor Veresov wrote: >>> In your change _would_profile is still a bool, I guess it should be of the enum >>> type above or at least an int? >> >> Thanks! I missed that. >> >> New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.02/ >> >> Thanks, >> Tobias >> >>> igor >>> >>> On Nov 6, 2014, at 11:58 PM, Tobias Hartmann >> > wrote: >>> >>>> Hi Vladimir, >>>> >>>> thanks for the review. >>>> >>>> On 07.11.2014 01:50, Vladimir Kozlov wrote: >>>>> On 11/6/14 2:25 AM, Tobias Hartmann wrote: >>>>>> Hi Vladimir, >>>>>> >>>>>> thanks for the review. >>>>>> >>>>>> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>>>>>> So the fix is add -XX:-TieredCompilation. >>>>>>> The rest is fix problems found due to these failure. >>>>>> >>>>>> Yes, exactly. >>>>>> >>>>>>> May be we should add a new test (other RFE) which would test compilation of >>>>>>> trivial methods in Tiered mode. Current failure shows that we don't test it. >>>>>> >>>>>> I agree. I filed RFE JDK-8056071 and will take care of it. >>>>>> >>>>>>> I am not comfortable to rely on data which is set only by C1. This >>>>>>> information >>>>>>> is accessed in non Tiered mode too. >>>>>> >>>>>> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, >>>>>> i.e., if >>>>>> TieredCompilation is enabled. Where do you think it is used in non-tiered >>>>>> mode? >>>>> >>>>> You are right. NonTieredCompPolicy::is_mature() does not use it. >>>>> >>>>>> >>>>>>> Why not use method->has_loops(), for example? >>>>>> >>>>>> We also decide based on the number of blocks and 'MethodData::would_profile' >>>>>> which is only available if we have a C1 compiled version. >>>>> >>>>> Yes, but next code will do the same and only checking MDO in the last case: >>>>> >>>>> bool SimpleThresholdPolicy::is_trivial(Method* method) { >>>>> if (method->is_accessor() || >>>>> method->is_constant_getter()) { >>>>> return true; >>>>> } >>>>> if (method->has_loops() || (method->code_size() >= 15)) { >>>>> return false; >>>>> } >>>>> MethodData* mdo = method->method_data(); >>>>> if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && >>>>> (method->code_size() < 5 || (mdo->num_blocks() < 4))) { >>>>> return true; >>>>> } >>>>> return false; >>>>> } >>>> >>>> Okay, I changed the code accordingly. >>>> >>>>> Also mdo->_would_profile will not be set if C1 compilation is bailed out >>>>> (Compilation() in c1_Compilation.cpp). So we can't trust it too. >>>>> May be mdo->_would_profile should be enum {unknown, no_profile, profile} >>>>> and we >>>>> use it instead of mdo->stats_valid(). >>>> >>>> I agree. Like this, we also save the additional '_stats_valid' field in >>>> MethodData. I added an enum and changed the implementation of 'stats_valid()'. >>>> It now returns true if '_would_profile' is not initialized or set to true. >>>> >>>> New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.01/ >>>> >>>> Thanks, >>>> Tobias >>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>>> >>>>>> Thanks, >>>>>> Tobias >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> please review the following patch. >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>>>>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>>>>>> >>>>>>>> Problem: >>>>>>>> The test compiles and deoptimizes a method m multiple times. After each >>>>>>>> deoptimization the test checks if m was compiled with C2 and if so >>>>>>>> increments a >>>>>>>> counter. The test assumes that m is compilable until the counter reaches >>>>>>>> the >>>>>>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>>>>>> - The first compilation request for m is added to the compile queue. >>>>>>>> Although we >>>>>>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>>>>>> information because due to deoptimization there is no corresponding nmethod >>>>>>>> [1]. >>>>>>>> We compile at level 4 and execution continues in the interpreter [2]. >>>>>>>> - A second compilation request for m is issued and while it is being >>>>>>>> processed >>>>>>>> the first compilation request finishes [3]. Because a nmethod now >>>>>>>> exists, we >>>>>>>> use >>>>>>>> the MDO and decide to compile at level 1 since m is trivial. >>>>>>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>>>>>> >>>>>>>> After the following deoptimization the counter is not incremented >>>>>>>> because the >>>>>>>> nmethod was compiled at level 1. As a result m is set to not compilable >>>>>>>> before >>>>>>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>>>>>> >>>>>>>> Problems with the current implementation: >>>>>>>> (1) We should always use the MDO if it is valid, even if there is no >>>>>>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or >>>>>>>> start >>>>>>>> profiling again. >>>>>>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a >>>>>>>> method >>>>>>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this >>>>>>>> case we >>>>>>>> wrongly assume that the method is trivial and re-compile it with C1. >>>>>>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>>>>>> constant getter methods, like those used by the test, as trivial. >>>>>>>> (4) We should add verification code to avoid/catch such bad tiered level >>>>>>>> transitions in the future. >>>>>>>> >>>>>>>> Solution: >>>>>>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that >>>>>>>> determines >>>>>>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was >>>>>>>> previously C1 >>>>>>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks >>>>>>>> if the >>>>>>>> MDO is valid. I added -XX:-TieredCompilation to the test because >>>>>>>> otherwise we >>>>>>>> always compile the (trivial) methods at level 1 and the test fails. >>>>>>>> >>>>>>>> To fix (3) I added the method 'Method::is_constant_getter()' that >>>>>>>> determines if >>>>>>>> a method consists of a constant push and a return statement only. If so, we >>>>>>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>>>>>> >>>>>>>> I filed JDK-8062913 for (4) because the verification code I added still >>>>>>>> fails >>>>>>>> with the current implementation due to 2->2 and 3->2 transitions. >>>>>>>> >>>>>>>> Testing: >>>>>>>> - JPRT including failing whitebox test >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Tobias >>>>>>>> >>>>>>>> >>>>>>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>>>>>> [2] First compilation request: >>>>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>>>> SimpleThresholdPolicy::event >>>>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>>>> AdvancedThresholdPolicy::call_event >>>>>>>> AdvancedThresholdPolicy::common >>>>>>>> [is_trivial() returns false because no nmethod available] >>>>>>>> [next_level = CompLevel_Full_Optimization] >>>>>>>> SimpleThresholdPolicy::compile >>>>>>>> SimpleThresholdPolicy::submit_compile >>>>>>>> CompileBroker::compile_method >>>>>>>> CompileBroker::compile_method_base >>>>>>>> CompileBroker::create_compile_task >>>>>>>> [continue execution in interpreter] >>>>>>>> >>>>>>>> [3] Second compilation request: >>>>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>>>> SimpleThresholdPolicy::event >>>>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>>>> [First compilation finishes here -> >>>>>>>> !CompileBroker::compilation_is_in_queue()] >>>>>>>> AdvancedThresholdPolicy::call_event >>>>>>>> AdvancedThresholdPolicy::common >>>>>>>> [is_trivial() returns true because nmethod/mdo now available] >>>>>>>> [next_level = CompLevel_Simple] >>>>>>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>>>>>> >>>>>>>> [4] >>>>>>>> https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>>>>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>>>>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >>> From albert.noll at oracle.com Mon Nov 10 17:06:53 2014 From: albert.noll at oracle.com (Albert Noll) Date: Mon, 10 Nov 2014 18:06:53 +0100 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint In-Reply-To: <5460CD6E.1060407@oracle.com> References: <545B761B.2000505@oracle.com> <54608F0D.1050101@oracle.com> <54609589.9090008@oracle.com> <5460A815.2010403@oracle.com> <5460CD6E.1060407@oracle.com> Message-ID: <5460F0AD.3070605@oracle.com> Hi Nils, thanks for the explanations. Please see comment inline: On 11/10/2014 03:36 PM, Nils Eliasson wrote: > Hi Albert, > > On 2014-11-10 12:57, Albert Noll wrote: >> Hi Nils, >> > [...] >>> >> Here are two things that I don't understand: >> 1) assert_locked_or_safepoint(CodeCache_lock) checks if we are at a >> safepoint and/or we own the 'CodeCache_lock'. If we are at a >> safepoint, how can another thread possibly modify data structures in >> the code cache? > > Looking at the code one more time the contract seem to be lock and > no_safepoint for mutators. But that is not what's asserted. We lock > with _no_safepoint_check - but that just means we don't verify > anything. We want to verify the no_safepoint. > > Many of the asserts are also not really helpful. Example: > CodeCache::allocate - assert_locked_or_safepoint(CodeCache_lock) - > The lock guarantees correctness for multiple threads, the safepoint > don't. But we have locking at all the code paths that call on > CodeCache::allocate, and we never do it at a safepoint. So it works ok. > > Writers should do: assert_locked_and_verify_no_safepoint() > Readers: assert locked_or_safepoint(). > >> >> 2) Why is it safe to use assert_locked_or_safepoint(CodeCache_lock) >> in so many places except for CodeCache::print_codelist(outputStream* >> st)? > > On second thought is should be enough with > assert_locked_or_safepoint(CodeCache_lock) on all readers if we assert > locked_and_verify_not_at_safepoint on all writers. So the old > implementation was correct. Does that mean you do no intend to push this change? Best, Albert > >> >> 3) To make sure that no other thread modifies the code cache, >> wouldn't you you have to make sure that every modification to the >> code cache is guarded by assert(CodeCache_lock->owned_by_self(), >> "Checking") instead of assert_locked_or_safepoint(CodeCache_lock)? >> I.e., a lock is only useful if every modification to the code cache >> is guarded by that lock. > > We should assert on lock and verify not at safepoint at the places > modifying the code cache if we want to keep the contract that it is ok > to read at safepoints. > > Thanks, > Nils > >> Best, >> Albert >> >>> Regards, >>> Nils >>> >>>> >>>> Thanks, >>>> Albert >>>>> Tested with jtreg tests in Hotspot and JDK that exercise this code. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 >>>>> Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ >>>>> >>>>> Thanks, >>>>> Nils >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.veresov at oracle.com Mon Nov 10 18:58:00 2014 From: igor.veresov at oracle.com (Igor Veresov) Date: Mon, 10 Nov 2014 10:58:00 -0800 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <54607966.7080205@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545C173F.5090403@oracle.com> <545C97D3.4010007@oracle.com> <9F6CF186-841D-41A6-AC4C-0D9541BA07AC@oracle.com> <54607966.7080205@oracle.com> Message-ID: <4053B441-2185-4F2F-8AED-1AE75C5E0B89@oracle.com> That looks good. igor > On Nov 10, 2014, at 12:37 AM, Tobias Hartmann wrote: > > Vladimir, Igor, thanks for the reviews. > > On 07.11.2014 17:21, Vladimir Kozlov wrote: >> You may change behavior in other places where would_profile() is used. Please, verify. Otherwise looks good > > Would_profile() is only used by the 'common' methods of the Simple and > AdvancedThresholdPolicies to decide if we need more profiling or should switch > to full optimization. > In the current implementation _would_profile is initialized to false and > therefore would_profile() returns false for methods that were never compiled > with C1. The fixed implementation returns true in this case. > > However, the 'common' methods only invoke would_profile() from level 2 and 3, > i.e. if a C1 compiled version exists. There should be no behavioural difference. > >> Thanks, >> Vladimir > > On 08.11.2014 09:23, Igor Veresov wrote: >> In your change _would_profile is still a bool, I guess it should be of the enum >> type above or at least an int? > > Thanks! I missed that. > > New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.02/ > > Thanks, > Tobias > >> igor >> >> On Nov 6, 2014, at 11:58 PM, Tobias Hartmann > > wrote: >> >>> Hi Vladimir, >>> >>> thanks for the review. >>> >>> On 07.11.2014 01:50, Vladimir Kozlov wrote: >>>> On 11/6/14 2:25 AM, Tobias Hartmann wrote: >>>>> Hi Vladimir, >>>>> >>>>> thanks for the review. >>>>> >>>>> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>>>>> So the fix is add -XX:-TieredCompilation. >>>>>> The rest is fix problems found due to these failure. >>>>> >>>>> Yes, exactly. >>>>> >>>>>> May be we should add a new test (other RFE) which would test compilation of >>>>>> trivial methods in Tiered mode. Current failure shows that we don't test it. >>>>> >>>>> I agree. I filed RFE JDK-8056071 and will take care of it. >>>>> >>>>>> I am not comfortable to rely on data which is set only by C1. This information >>>>>> is accessed in non Tiered mode too. >>>>> >>>>> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if >>>>> TieredCompilation is enabled. Where do you think it is used in non-tiered mode? >>>> >>>> You are right. NonTieredCompPolicy::is_mature() does not use it. >>>> >>>>> >>>>>> Why not use method->has_loops(), for example? >>>>> >>>>> We also decide based on the number of blocks and 'MethodData::would_profile' >>>>> which is only available if we have a C1 compiled version. >>>> >>>> Yes, but next code will do the same and only checking MDO in the last case: >>>> >>>> bool SimpleThresholdPolicy::is_trivial(Method* method) { >>>> if (method->is_accessor() || >>>> method->is_constant_getter()) { >>>> return true; >>>> } >>>> if (method->has_loops() || (method->code_size() >= 15)) { >>>> return false; >>>> } >>>> MethodData* mdo = method->method_data(); >>>> if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && >>>> (method->code_size() < 5 || (mdo->num_blocks() < 4))) { >>>> return true; >>>> } >>>> return false; >>>> } >>> >>> Okay, I changed the code accordingly. >>> >>>> Also mdo->_would_profile will not be set if C1 compilation is bailed out >>>> (Compilation() in c1_Compilation.cpp). So we can't trust it too. >>>> May be mdo->_would_profile should be enum {unknown, no_profile, profile} and we >>>> use it instead of mdo->stats_valid(). >>> >>> I agree. Like this, we also save the additional '_stats_valid' field in >>> MethodData. I added an enum and changed the implementation of 'stats_valid()'. >>> It now returns true if '_would_profile' is not initialized or set to true. >>> >>> New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.01/ >>> >>> Thanks, >>> Tobias >>> >>>> Thanks, >>>> Vladimir >>>> >>>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>>>>> Hi, >>>>>>> >>>>>>> please review the following patch. >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>>>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>>>>> >>>>>>> Problem: >>>>>>> The test compiles and deoptimizes a method m multiple times. After each >>>>>>> deoptimization the test checks if m was compiled with C2 and if so >>>>>>> increments a >>>>>>> counter. The test assumes that m is compilable until the counter reaches the >>>>>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>>>>> - The first compilation request for m is added to the compile queue. >>>>>>> Although we >>>>>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>>>>> information because due to deoptimization there is no corresponding nmethod >>>>>>> [1]. >>>>>>> We compile at level 4 and execution continues in the interpreter [2]. >>>>>>> - A second compilation request for m is issued and while it is being processed >>>>>>> the first compilation request finishes [3]. Because a nmethod now exists, we >>>>>>> use >>>>>>> the MDO and decide to compile at level 1 since m is trivial. >>>>>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>>>>> >>>>>>> After the following deoptimization the counter is not incremented because the >>>>>>> nmethod was compiled at level 1. As a result m is set to not compilable before >>>>>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>>>>> >>>>>>> Problems with the current implementation: >>>>>>> (1) We should always use the MDO if it is valid, even if there is no >>>>>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or >>>>>>> start >>>>>>> profiling again. >>>>>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a >>>>>>> method >>>>>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>>>>>> wrongly assume that the method is trivial and re-compile it with C1. >>>>>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>>>>> constant getter methods, like those used by the test, as trivial. >>>>>>> (4) We should add verification code to avoid/catch such bad tiered level >>>>>>> transitions in the future. >>>>>>> >>>>>>> Solution: >>>>>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that >>>>>>> determines >>>>>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>>>>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>>>>>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>>>>>> always compile the (trivial) methods at level 1 and the test fails. >>>>>>> >>>>>>> To fix (3) I added the method 'Method::is_constant_getter()' that >>>>>>> determines if >>>>>>> a method consists of a constant push and a return statement only. If so, we >>>>>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>>>>> >>>>>>> I filed JDK-8062913 for (4) because the verification code I added still fails >>>>>>> with the current implementation due to 2->2 and 3->2 transitions. >>>>>>> >>>>>>> Testing: >>>>>>> - JPRT including failing whitebox test >>>>>>> >>>>>>> Thanks, >>>>>>> Tobias >>>>>>> >>>>>>> >>>>>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>>>>> [2] First compilation request: >>>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>>> SimpleThresholdPolicy::event >>>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>>> AdvancedThresholdPolicy::call_event >>>>>>> AdvancedThresholdPolicy::common >>>>>>> [is_trivial() returns false because no nmethod available] >>>>>>> [next_level = CompLevel_Full_Optimization] >>>>>>> SimpleThresholdPolicy::compile >>>>>>> SimpleThresholdPolicy::submit_compile >>>>>>> CompileBroker::compile_method >>>>>>> CompileBroker::compile_method_base >>>>>>> CompileBroker::create_compile_task >>>>>>> [continue execution in interpreter] >>>>>>> >>>>>>> [3] Second compilation request: >>>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>>> SimpleThresholdPolicy::event >>>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>>> [First compilation finishes here -> >>>>>>> !CompileBroker::compilation_is_in_queue()] >>>>>>> AdvancedThresholdPolicy::call_event >>>>>>> AdvancedThresholdPolicy::common >>>>>>> [is_trivial() returns true because nmethod/mdo now available] >>>>>>> [next_level = CompLevel_Simple] >>>>>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>>>>> >>>>>>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>>>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>>>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >> From paul.j.elschot at gmail.com Tue Nov 11 08:33:44 2014 From: paul.j.elschot at gmail.com (Paul Elschot) Date: Tue, 11 Nov 2014 09:33:44 +0100 Subject: Processor support for select operations Message-ID: <5461C9E8.1070809@gmail.com> Dear readers, I recently asked this question on the lucene-dev list (http://lucene.apache.org/core/discussion.html), and in reply David Weiss pointed me here. He also pointed to some recent discussions here on the use of bit operations to implement an IndexSetIterator, so you may be interested because of that. This was the question, also somewhat off topic here: For LUCENE-6040 it would be good to have better processor support for selecting the i-th set bit from a 64-bit integer. Not too long ago Long.bitCount() was intrinsified in JVM's. I hope something similar will happen to a select(long x, int i) method. However, better processor support is needed first. This is somewhat off topic here, but does anyone know how to request better processor support for select operations? Regards, Paul Elschot To be complete here are some references: - http://vigna.di.unimi.it/Sux/select.php on the currently fastest select code in C, - https://issues.apache.org/jira/browse/LUCENE-6040 the above lucene issue with select code in Java. From tobias.hartmann at oracle.com Tue Nov 11 09:07:17 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 11 Nov 2014 10:07:17 +0100 Subject: [9] RFR(S): 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <4053B441-2185-4F2F-8AED-1AE75C5E0B89@oracle.com> References: <545A1445.2050200@oracle.com> <545AC741.2040803@oracle.com> <545B4C85.7000109@oracle.com> <545C173F.5090403@oracle.com> <545C97D3.4010007@oracle.com> <9F6CF186-841D-41A6-AC4C-0D9541BA07AC@oracle.com> <54607966.7080205@oracle.com> <4053B441-2185-4F2F-8AED-1AE75C5E0B89@oracle.com> Message-ID: <5461D1C5.6070406@oracle.com> Thanks, Igor. Best, Tobias On 10.11.2014 19:58, Igor Veresov wrote: > That looks good. > > igor > >> On Nov 10, 2014, at 12:37 AM, Tobias Hartmann wrote: >> >> Vladimir, Igor, thanks for the reviews. >> >> On 07.11.2014 17:21, Vladimir Kozlov wrote: >>> You may change behavior in other places where would_profile() is used. Please, verify. Otherwise looks good >> >> Would_profile() is only used by the 'common' methods of the Simple and >> AdvancedThresholdPolicies to decide if we need more profiling or should switch >> to full optimization. >> In the current implementation _would_profile is initialized to false and >> therefore would_profile() returns false for methods that were never compiled >> with C1. The fixed implementation returns true in this case. >> >> However, the 'common' methods only invoke would_profile() from level 2 and 3, >> i.e. if a C1 compiled version exists. There should be no behavioural difference. >> >>> Thanks, >>> Vladimir >> >> On 08.11.2014 09:23, Igor Veresov wrote: >>> In your change _would_profile is still a bool, I guess it should be of the enum >>> type above or at least an int? >> >> Thanks! I missed that. >> >> New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.02/ >> >> Thanks, >> Tobias >> >>> igor >>> >>> On Nov 6, 2014, at 11:58 PM, Tobias Hartmann >> > wrote: >>> >>>> Hi Vladimir, >>>> >>>> thanks for the review. >>>> >>>> On 07.11.2014 01:50, Vladimir Kozlov wrote: >>>>> On 11/6/14 2:25 AM, Tobias Hartmann wrote: >>>>>> Hi Vladimir, >>>>>> >>>>>> thanks for the review. >>>>>> >>>>>> On 06.11.2014 01:56, Vladimir Kozlov wrote: >>>>>>> So the fix is add -XX:-TieredCompilation. >>>>>>> The rest is fix problems found due to these failure. >>>>>> >>>>>> Yes, exactly. >>>>>> >>>>>>> May be we should add a new test (other RFE) which would test compilation of >>>>>>> trivial methods in Tiered mode. Current failure shows that we don't test it. >>>>>> >>>>>> I agree. I filed RFE JDK-8056071 and will take care of it. >>>>>> >>>>>>> I am not comfortable to rely on data which is set only by C1. This information >>>>>>> is accessed in non Tiered mode too. >>>>>> >>>>>> 'is_trivial' is only used with the Simple and AdvancedThresholdPolicy, i.e., if >>>>>> TieredCompilation is enabled. Where do you think it is used in non-tiered mode? >>>>> >>>>> You are right. NonTieredCompPolicy::is_mature() does not use it. >>>>> >>>>>> >>>>>>> Why not use method->has_loops(), for example? >>>>>> >>>>>> We also decide based on the number of blocks and 'MethodData::would_profile' >>>>>> which is only available if we have a C1 compiled version. >>>>> >>>>> Yes, but next code will do the same and only checking MDO in the last case: >>>>> >>>>> bool SimpleThresholdPolicy::is_trivial(Method* method) { >>>>> if (method->is_accessor() || >>>>> method->is_constant_getter()) { >>>>> return true; >>>>> } >>>>> if (method->has_loops() || (method->code_size() >= 15)) { >>>>> return false; >>>>> } >>>>> MethodData* mdo = method->method_data(); >>>>> if (mdo != NULL && mdo->stats_valid() && !mdo->would_profile() && >>>>> (method->code_size() < 5 || (mdo->num_blocks() < 4))) { >>>>> return true; >>>>> } >>>>> return false; >>>>> } >>>> >>>> Okay, I changed the code accordingly. >>>> >>>>> Also mdo->_would_profile will not be set if C1 compilation is bailed out >>>>> (Compilation() in c1_Compilation.cpp). So we can't trust it too. >>>>> May be mdo->_would_profile should be enum {unknown, no_profile, profile} and we >>>>> use it instead of mdo->stats_valid(). >>>> >>>> I agree. Like this, we also save the additional '_stats_valid' field in >>>> MethodData. I added an enum and changed the implementation of 'stats_valid()'. >>>> It now returns true if '_would_profile' is not initialized or set to true. >>>> >>>> New webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.01/ >>>> >>>> Thanks, >>>> Tobias >>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>>> >>>>>> Thanks, >>>>>> Tobias >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>> On 11/5/14 4:12 AM, Tobias Hartmann wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> please review the following patch. >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8056071 >>>>>>>> Webrev: http://cr.openjdk.java.net/~thartmann/8056071/webrev.00/ >>>>>>>> >>>>>>>> Problem: >>>>>>>> The test compiles and deoptimizes a method m multiple times. After each >>>>>>>> deoptimization the test checks if m was compiled with C2 and if so >>>>>>>> increments a >>>>>>>> counter. The test assumes that m is compilable until the counter reaches the >>>>>>>> 'PerMethodRecompilationCutoff'. This does not hold in the following case: >>>>>>>> - The first compilation request for m is added to the compile queue. >>>>>>>> Although we >>>>>>>> profiled m and the MDO suggests that m is trivial, we do not use this >>>>>>>> information because due to deoptimization there is no corresponding nmethod >>>>>>>> [1]. >>>>>>>> We compile at level 4 and execution continues in the interpreter [2]. >>>>>>>> - A second compilation request for m is issued and while it is being processed >>>>>>>> the first compilation request finishes [3]. Because a nmethod now exists, we >>>>>>>> use >>>>>>>> the MDO and decide to compile at level 1 since m is trivial. >>>>>>>> - The level 4 compiled nmethod is replaced by the new level 1 nmethod [4]. >>>>>>>> >>>>>>>> After the following deoptimization the counter is not incremented because the >>>>>>>> nmethod was compiled at level 1. As a result m is set to not compilable before >>>>>>>> the counter reaches the 'PerMethodRecompilationCutoff'. The test fails. >>>>>>>> >>>>>>>> Problems with the current implementation: >>>>>>>> (1) We should always use the MDO if it is valid, even if there is no >>>>>>>> corresponding nmethod. Otherwise we compile trivial methods at level 4 or >>>>>>>> start >>>>>>>> profiling again. >>>>>>>> (2) If there is a nmethod the MDO may still be invalid: We can compile a >>>>>>>> method >>>>>>>> immediately at C2, deoptimize, and the MDO is uninitialized. In this case we >>>>>>>> wrongly assume that the method is trivial and re-compile it with C1. >>>>>>>> (3) To avoid a level 2 or 3 compilation and profiling we should mark simple >>>>>>>> constant getter methods, like those used by the test, as trivial. >>>>>>>> (4) We should add verification code to avoid/catch such bad tiered level >>>>>>>> transitions in the future. >>>>>>>> >>>>>>>> Solution: >>>>>>>> To fix (1) and (2) I added a field '_stats_valid' to MethodData that >>>>>>>> determines >>>>>>>> if '_num_loops' and '_num_blocks' are set (i.e. the method was previously C1 >>>>>>>> compiled). Instead of checking for a nmethod, 'is_trivial()' now checks if the >>>>>>>> MDO is valid. I added -XX:-TieredCompilation to the test because otherwise we >>>>>>>> always compile the (trivial) methods at level 1 and the test fails. >>>>>>>> >>>>>>>> To fix (3) I added the method 'Method::is_constant_getter()' that >>>>>>>> determines if >>>>>>>> a method consists of a constant push and a return statement only. If so, we >>>>>>>> treat the method as trivial. See trivial.log [5] and trivial_fixed.log [6]. >>>>>>>> >>>>>>>> I filed JDK-8062913 for (4) because the verification code I added still fails >>>>>>>> with the current implementation due to 2->2 and 3->2 transitions. >>>>>>>> >>>>>>>> Testing: >>>>>>>> - JPRT including failing whitebox test >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Tobias >>>>>>>> >>>>>>>> >>>>>>>> [1] See implementation of 'SimpleThresholdPolicy::is_trivial(..)' >>>>>>>> [2] First compilation request: >>>>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>>>> SimpleThresholdPolicy::event >>>>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>>>> AdvancedThresholdPolicy::call_event >>>>>>>> AdvancedThresholdPolicy::common >>>>>>>> [is_trivial() returns false because no nmethod available] >>>>>>>> [next_level = CompLevel_Full_Optimization] >>>>>>>> SimpleThresholdPolicy::compile >>>>>>>> SimpleThresholdPolicy::submit_compile >>>>>>>> CompileBroker::compile_method >>>>>>>> CompileBroker::compile_method_base >>>>>>>> CompileBroker::create_compile_task >>>>>>>> [continue execution in interpreter] >>>>>>>> >>>>>>>> [3] Second compilation request: >>>>>>>> InterpreterRuntime::frequency_counter_overflow >>>>>>>> InterpreterRuntime::frequency_counter_overflow_inner >>>>>>>> SimpleThresholdPolicy::event >>>>>>>> AdvancedThresholdPolicy::method_invocation_event >>>>>>>> [First compilation finishes here -> >>>>>>>> !CompileBroker::compilation_is_in_queue()] >>>>>>>> AdvancedThresholdPolicy::call_event >>>>>>>> AdvancedThresholdPolicy::common >>>>>>>> [is_trivial() returns true because nmethod/mdo now available] >>>>>>>> [next_level = CompLevel_Simple] >>>>>>>> [CompLevel_Simple nmethod replaced CompLevel_Full_Optimization method] >>>>>>>> >>>>>>>> [4] https://bugs.openjdk.java.net/secure/attachment/23321/PrintCompilation.log >>>>>>>> [5] https://bugs.openjdk.java.net/secure/attachment/23327/trivial.log >>>>>>>> [6] https://bugs.openjdk.java.net/secure/attachment/23328/trivial_fixed.log >>> > From nils.eliasson at oracle.com Tue Nov 11 12:57:14 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Tue, 11 Nov 2014 13:57:14 +0100 Subject: RFR(S): 8063112: Compiler diagnostic commands should have locking instead of safepoint In-Reply-To: <5460F0AD.3070605@oracle.com> References: <545B761B.2000505@oracle.com> <54608F0D.1050101@oracle.com> <54609589.9090008@oracle.com> <5460A815.2010403@oracle.com> <5460CD6E.1060407@oracle.com> <5460F0AD.3070605@oracle.com> Message-ID: <546207AA.9040203@oracle.com> Not now at least. I'll investigate the asserts some more. //N On 2014-11-10 18:06, Albert Noll wrote: > Hi Nils, > thanks for the explanations. Please see comment inline: > On 11/10/2014 03:36 PM, Nils Eliasson wrote: >> Hi Albert, >> >> On 2014-11-10 12:57, Albert Noll wrote: >>> Hi Nils, >>> >> [...] >>>> >>> Here are two things that I don't understand: >>> 1) assert_locked_or_safepoint(CodeCache_lock) checks if we are at a >>> safepoint and/or we own the 'CodeCache_lock'. If we are at a >>> safepoint, how can another thread possibly modify data structures in >>> the code cache? >> >> Looking at the code one more time the contract seem to be lock and >> no_safepoint for mutators. But that is not what's asserted. We lock >> with _no_safepoint_check - but that just means we don't verify >> anything. We want to verify the no_safepoint. >> >> Many of the asserts are also not really helpful. Example: >> CodeCache::allocate - assert_locked_or_safepoint(CodeCache_lock) - >> The lock guarantees correctness for multiple threads, the safepoint >> don't. But we have locking at all the code paths that call on >> CodeCache::allocate, and we never do it at a safepoint. So it works ok. >> >> Writers should do: assert_locked_and_verify_no_safepoint() >> Readers: assert locked_or_safepoint(). >> >>> >>> 2) Why is it safe to use assert_locked_or_safepoint(CodeCache_lock) >>> in so many places except for CodeCache::print_codelist(outputStream* >>> st)? >> >> On second thought is should be enough with >> assert_locked_or_safepoint(CodeCache_lock) on all readers if we >> assert locked_and_verify_not_at_safepoint on all writers. So the old >> implementation was correct. > Does that mean you do no intend to push this change? > > Best, > Albert > >> >>> >>> 3) To make sure that no other thread modifies the code cache, >>> wouldn't you you have to make sure that every modification to the >>> code cache is guarded by assert(CodeCache_lock->owned_by_self(), >>> "Checking") instead of assert_locked_or_safepoint(CodeCache_lock)? >>> I.e., a lock is only useful if every modification to the code cache >>> is guarded by that lock. >> >> We should assert on lock and verify not at safepoint at the places >> modifying the code cache if we want to keep the contract that it is >> ok to read at safepoints. >> >> Thanks, >> Nils >> >>> Best, >>> Albert >>> >>>> Regards, >>>> Nils >>>> >>>>> >>>>> Thanks, >>>>> Albert >>>>>> Tested with jtreg tests in Hotspot and JDK that exercise this code. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8063112 >>>>>> Webrev: http://cr.openjdk.java.net/~neliasso/8063112/webrev.01/ >>>>>> >>>>>> Thanks, >>>>>> Nils >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksey.shipilev at oracle.com Tue Nov 11 13:27:07 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 11 Nov 2014 16:27:07 +0300 Subject: Processor support for select operations In-Reply-To: <5461C9E8.1070809@gmail.com> References: <5461C9E8.1070809@gmail.com> Message-ID: <54620EAB.2080000@oracle.com> Hi Paul, On 11.11.2014 11:33, Paul Elschot wrote: > He also pointed to some recent discussions here on the use of bit > operations to implement an IndexSetIterator, so you may be interested > because of that. Notice we are using lookup tables there instead of sophisticated math there. In fact, we would like to use TZCNT/LZCNT and friends, but it is hard to do in a cross-platform way from C++ code. > This was the question, also somewhat off topic here: > > For LUCENE-6040 it would be good to have better processor support for > selecting the i-th set bit from a 64-bit integer. I would say this is an offtopic for this list. People on this list are not hardware people. > Not too long ago Long.bitCount() was intrinsified in JVM's. As well as countLeadingZeros/countTrailingZeros, on platforms where LZCNT/TZCNT are available. You can probably use that to enumerate bits in a Integer/Long faster. > I hope something similar will happen to a select(long x, int i) > method. However, better processor support is needed first. What method are referring to? We can only safely intrinsify methods with clear and stable semantics, i.e. those in standard library. I don't see any, and so I guess the very first thing to do is prototype the Java function like that in, say, java.lang.Long. -Aleksey. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From vitalyd at gmail.com Tue Nov 11 13:47:05 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 11 Nov 2014 08:47:05 -0500 Subject: Processor support for select operations In-Reply-To: <5461C9E8.1070809@gmail.com> References: <5461C9E8.1070809@gmail.com> Message-ID: Have you tried doing the array access via Unsafe? It's not clear from the citation what exactly is the issue with java array access, but I'm assuming it's the range checks. Sent from my phone On Nov 11, 2014 3:34 AM, "Paul Elschot" wrote: > Dear readers, > > I recently asked this question on the lucene-dev list > (http://lucene.apache.org/core/discussion.html), and in reply David > Weiss pointed me here. > He also pointed to some recent discussions here on the use of bit > operations to implement an IndexSetIterator, so you may be interested > because of that. > > This was the question, also somewhat off topic here: > > For LUCENE-6040 it would be good to have better processor support for > selecting the i-th set bit from a 64-bit integer. > > Not too long ago Long.bitCount() was intrinsified in JVM's. > > I hope something similar will happen to a select(long x, int i) > method. However, better processor support is needed first. > > This is somewhat off topic here, but does anyone know how to request > better processor support for select operations? > > > Regards, > Paul Elschot > > > To be complete here are some references: > - http://vigna.di.unimi.it/Sux/select.php on the currently fastest > select code in C, > - https://issues.apache.org/jira/browse/LUCENE-6040 the above lucene > issue with select code in Java. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.j.elschot at gmail.com Tue Nov 11 18:08:47 2014 From: paul.j.elschot at gmail.com (Paul Elschot) Date: Tue, 11 Nov 2014 19:08:47 +0100 Subject: Processor support for select operations In-Reply-To: <54620EAB.2080000@oracle.com> References: <5461C9E8.1070809@gmail.com> <54620EAB.2080000@oracle.com> Message-ID: <546250AF.7060003@gmail.com> Aleksey, On 11-11-14 14:27, Aleksey Shipilev wrote: > ... > What method are referring to? We can only safely intrinsify methods with > clear and stable semantics, i.e. those in standard library. I don't see > any, and so I guess the very first thing to do is prototype the Java > function like that in, say, java.lang.Long. Java code, also available from: https://svn.apache.org/viewvc/lucene/dev/trunk/lucene/core/src/java/org/apache/lucene/util/BitUtil.java?revision=1636913 /** Select a 1-bit from a long. See also LUCENE-6040. * @return The index of the r-th 1 bit in x. This bit must exist. */ public static int select(long x, int r) { long s = x - ((x & 0xAAAAAAAAAAAAAAAAL) >>> 1); // pairwise bitsums // nibblewise bitsums s = (s & 0x3333333333333333L) + ((s >>> 2) & 0x3333333333333333L); // bytewise bitsums, cumulative s = ((s + (s >>> 4)) & 0x0F0F0F0F0F0F0F0FL) * L8_L; // bit position of byte with r-th 1 bit. int b = (Long.numberOfTrailingZeros( (s + psOverflow[r-1]) & (L8_L << 7)) >> 3) << 3; long l = r - (((s << 8) >>> b) & 0xFFL); // bit rank in byte at b // Select bit l from byte (x >>> b): int selectIndex = (int) (((x >>> b) & 0xFFL) | ((l-1) << 8)); int res = b + select256[selectIndex]; return res; } private final static long L8_L = 0x0101010101010101L; The java code to initialize the arrays is at the link above. When the bit to be selected does not exist, the semantics are undefined. Normally a Long.bitCount(x) is done beforehand to avoid that. There is C code ("The fastest code") here, it works in much the same way: http://vigna.di.unimi.it/Sux/select.php Regards, Paul Elschot From paul.j.elschot at gmail.com Tue Nov 11 18:16:13 2014 From: paul.j.elschot at gmail.com (Paul Elschot) Date: Tue, 11 Nov 2014 19:16:13 +0100 Subject: Processor support for select operations In-Reply-To: References: <5461C9E8.1070809@gmail.com> Message-ID: <5462526D.1010302@gmail.com> On 11-11-14 14:47, Vitaly Davidovich wrote: > Have you tried doing the array access via Unsafe? It's not clear from > the citation what exactly is the issue with java array access, but I'm > assuming it's the range checks. I have not tried that, it would probably not be acceptable for Lucene. A little masking might be enough to enforce the ranges, see the code in my other reply. Regards, Paul Elschot From john.r.rose at oracle.com Tue Nov 11 18:54:21 2014 From: john.r.rose at oracle.com (John Rose) Date: Tue, 11 Nov 2014 10:54:21 -0800 Subject: Processor support for select operations In-Reply-To: <5462526D.1010302@gmail.com> References: <5461C9E8.1070809@gmail.com> <5462526D.1010302@gmail.com> Message-ID: <535F47A2-C478-4A16-AC57-FE3DF2FC7918@oracle.com> On Nov 11, 2014, at 10:16 AM, Paul Elschot wrote: > A little masking might be enough to enforce the ranges, see the code in > my other reply. Power-of-two array indexing is an important use case which can be optimized without resorting to Unsafe. We're working on it. See https://bugs.openjdk.java.net/browse/JDK-8003585 ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.r.chase at oracle.com Tue Nov 11 18:58:30 2014 From: david.r.chase at oracle.com (David Chase) Date: Tue, 11 Nov 2014 13:58:30 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <545E31BA.3070500@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545E31BA.3070500@gmail.com> Message-ID: <1D96462F-87D3-4794-A818-27DD2EE10046@oracle.com> On 2014-11-08, at 10:07 AM, Peter Levart wrote: > > Now let's take for example one of the MemberName.make() methods that return interned MemberNames: > > 206 public static MemberName make(Method m, boolean wantSpecial) { > 207 // Unreflected member names are resolved so intern them here. > 208 MemberName tmp0 = null; > 209 InternTransaction tx = new InternTransaction(m.getDeclaringClass()); > 210 while (tmp0 == null) { > 211 MemberName tmp = new MemberName(m, wantSpecial); > 212 tmp0 = tx.tryIntern(tmp); > 213 } > 214 return tmp0; > 215 } > > I'm trying to understand the workings of InternTransaction helper class (and find an example that breaks it). You create an instance of it, passing Method's declaringClass. You then (in retry loop) create a resolved MemberName from the Method and wantSpecial flag. This MemberName's clazz can apparently differ from Method's declaringClass. I don't know when and why this happens, but apparently it can (super method?), so in InternTransaction.tryIntern() you do... > > 363 if (member_name.isResolved()) { > 364 if (member_name.clazz != tx_class) { > 365 Class prev_tx_class = tx_class; > 366 int prev_txn_token = txn_token; > 367 tx_class = member_name.clazz; > 368 txn_token = internTxnToken(tx_class); > 369 // Zero is a special case. > 370 if (txn_token != 0 || > 371 prev_txn_token != internTxnToken(prev_tx_class)) { > 372 // Resolved class is different and at least one > 373 // redef of it occurred, therefore repeat with > 374 // proper class for race consistency checking. > 375 return null; > 376 } > 377 } > 378 member_name = member_name.intern(txn_token); > 379 if (member_name == null) { > 380 // Update the token for the next try. > 381 txn_token = internTxnToken(tx_class); > 382 } > 383 } > > > Now let's assume that the resolved member_name.clazz differs from Method's declaringClass. Let's assume also that either member_name.clazz has had at least one redefinition or Method's declaringClass has been redefined between creating InternTransaction and reading member_name.clazz's txn_token. You return 'null' in such case, concluding that not only the resolved member_name.clazz redefinition matters, but Method's declaringClass redefinition can also invalidate resolved MemberName am I right? It would be helpful if I could understand when and how Method's declaringClass redefinition can affect member_name. Can it affect which clazz is resolved for member_name? If a declaring class is redefined before a MemberName is ?published? to the VM, then there is a risk that its secret fields will have gone stale because the referenced VM methods changed but were not updated. Therefore, the resolution must be retried to get a fresh resolution that is known not to be stale. There is sort of a glitch in the race-checking protocol; I don?t have certain knowledge which class will be resolved, so if I guessed wrong (and the common-case no redefinition at all check fails) then I am forced to retry and get a fresh, known-good resolution. However, based on my understanding of what is (not) allowed in class redefinition, what differs after redefinition is only the code of the method, and not the owner ? that is, if D.m resolved to B.m before redefinition of D, C, or B, then it will always resolve to B.m ? but the definition of B.m itself might have changed (from the test cases, it might print ?foo? instead of ?bar?). Or to put it differently, the methods change, but their hierarchy does not. > Anyway, you return null in such case from an updated InternTransaction (tx_class and txn_token are now updated to have values for resolved member_name.clazz). In next round the checks of newly constructed and resolved member_name are not performed against Method's declaringClass but against previous round's member_name.clazz. Is this what is intended? > I can see there has to be a stop condition for loop to end, but shouldn't checks for Method's declaringClass redefinition be performed in every iteration (in addition to the check for member_name.clazz redefinition if it differs from Method's declaringClass)? To the best of my understanding (see restrictions above) the tx_class ought to be wrong at most once; all subsequent resolutions including those that span a class redefinition should return the same class, so it suffices to detect redefinition of the method itself. I?ve incorporated your other changes (not yet the linear-scan hash table) and will be retesting. One thing I wonder about for both hash table and binary search is if the first try should be attempted with no lock to avoid the overhead of synchronization; I expect that looking will be more common than interning, which in turn will be (vastly) more common than class redefinition. David -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From vitalyd at gmail.com Tue Nov 11 19:00:54 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 11 Nov 2014 14:00:54 -0500 Subject: Processor support for select operations In-Reply-To: <535F47A2-C478-4A16-AC57-FE3DF2FC7918@oracle.com> References: <5461C9E8.1070809@gmail.com> <5462526D.1010302@gmail.com> <535F47A2-C478-4A16-AC57-FE3DF2FC7918@oracle.com> Message-ID: But in the meantime, just trying Unsafe to see what the perf benefit would be is still interesting, no? Sent from my phone On Nov 11, 2014 1:54 PM, "John Rose" wrote: > On Nov 11, 2014, at 10:16 AM, Paul Elschot > wrote: > > A little masking might be enough to enforce the ranges, see the code in > my other reply. > > > Power-of-two array indexing is an important use case which can be > optimized without resorting to Unsafe. > We're working on it. See https://bugs.openjdk.java.net/browse/JDK-8003585 > > ? John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.r.rose at oracle.com Tue Nov 11 19:17:18 2014 From: john.r.rose at oracle.com (John Rose) Date: Tue, 11 Nov 2014 11:17:18 -0800 Subject: Processor support for select operations In-Reply-To: References: <5461C9E8.1070809@gmail.com> <5462526D.1010302@gmail.com> <535F47A2-C478-4A16-AC57-FE3DF2FC7918@oracle.com> Message-ID: On Nov 11, 2014, at 11:00 AM, Vitaly Davidovich wrote: > But in the meantime, just trying Unsafe to see what the perf benefit would be is still interesting, no? > Yep. It's worth repeating: Unsafe is only for internal use by the JDK. The access restrictions on it, which are often circumvented, are going to be strengthened by the JDK 9 module system, and the restrictions will become harder to circumvent. Use it for experiments if that's useful, but don't build it into your product. For more information, check out Paul Sandoz's talk on unsafe and varhandles at Java One 2014. ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksey.shipilev at oracle.com Tue Nov 11 19:20:11 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 11 Nov 2014 22:20:11 +0300 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <545AAB7D.3010100@oracle.com> References: <545A960C.7010008@oracle.com> <545AAB7D.3010100@oracle.com> Message-ID: <5462616B.3030003@oracle.com> Thanks for review, Vladimir! FTR, here's an updated webrev: http://cr.openjdk.java.net/~shade/8059461/webrev.03/ On 11/06/2014 01:58 AM, Vladimir Kozlov wrote: > Aleksey, can you compare average memory consumed by IndexSet before and > after? Any ideas how to estimate this reliably? Current code has some instrumentation, but only about usage, not the concrete footprint. NMT shows very flaky numbers for Nashorn/Octane: $ grep "Compiler" before/* Compiler (reserved=165KB, committed=165KB) Compiler (reserved=174KB, committed=174KB) Compiler (reserved=201KB, committed=201KB) Compiler (reserved=189KB, committed=189KB) Compiler (reserved=192KB, committed=192KB) $ grep "Compiler" after/* Compiler (reserved=162KB, committed=162KB) Compiler (reserved=167KB, committed=167KB) Compiler (reserved=198KB, committed=198KB) Compiler (reserved=240KB, committed=240KB) Compiler (reserved=188KB, committed=188KB) $ grep "Arena" before/* Arena Chunk (reserved=10952KB, committed=10952KB) Arena Chunk (reserved=11433KB, committed=11433KB) Arena Chunk (reserved=10506KB, committed=10506KB) Arena Chunk (reserved=10825KB, committed=10825KB) Arena Chunk (reserved=7179KB, committed=7179KB) $ grep "Arena" after/* Arena Chunk (reserved=19971KB, committed=19971KB) Arena Chunk (reserved=20738KB, committed=20738KB) Arena Chunk (reserved=21090KB, committed=21090KB) Arena Chunk (reserved=22304KB, committed=22304KB) Arena Chunk (reserved=14342KB, committed=14342KB) It *looks* like a memory leak in Arenas, but it might be as well the larger peak usage. Or, it might be NMT not telling me the whole story. I am running Footprint tests from our performance regression suites now -- they will show if there is something real. > Why you need initialize_in_resource_arena()? by default BitMap() uses > resource area: > > BitMap(idx_t size_in_bits, bool in_resource_area = true); Yes, pruned that. > Make lrg_union() PhaseConservativeCoalesce class's method. Yes, did that. Thanks, -Aleksey. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From aleksey.shipilev at oracle.com Tue Nov 11 19:21:39 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 11 Nov 2014 22:21:39 +0300 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <545B4134.4040103@oracle.com> References: <545A960C.7010008@oracle.com> <545AAB7D.3010100@oracle.com> <545B4134.4040103@oracle.com> Message-ID: <546261C3.9080905@oracle.com> Thanks for taking a look, Albert! Of course, the performance runs for other workloads and platforms are underway. I am not expecting that dramatic boosts though, since Nashorn is one of the few compilation-heavy workloads in our belt. -Aleksey. On 11/06/2014 12:36 PM, Albert Noll wrote: > It would be good to have data from different applications / architectures. > > Does anyone know more about the motivation for IndexSet, i.e., why it > was introduced in the first place? > > Thanks, > Albert > > On 11/05/2014 11:58 PM, Vladimir Kozlov wrote: >> I think it is nice cleanup with performance benefits :) >> >> Aleksey, can you compare average memory consumed by IndexSet before >> and after? >> >> Why you need initialize_in_resource_arena()? by default BitMap() uses >> resource area: >> >> BitMap(idx_t size_in_bits, bool in_resource_area = true); >> >> Make lrg_union() PhaseConservativeCoalesce class's method. >> >> Thanks, >> Vladimir >> >> On 11/5/14 1:26 PM, Aleksey Shipilev wrote: >>> Hi, >>> >>> Long story short: current implementation of IndexSet, while smart, is >>> too smart for its own good. Trying to be sparse, it loses locality. >>> Trying to be smart about bit tricks, it loses the native word length. >>> >>> Because of that, sophisticated IndexSet does not yield a desired >>> performance benefit on compilation-heavy workloads like Nashorn. >>> Delegating the work to already existing BitMap both conserves the source >>> code, and brings more performance. (C1 also uses the BitMap adapter like >>> that for ValueMap-s). >>> >>> IndexSet is a major data structure for representing IFG in C2 Regalloc, >>> and that is why improvements in IndexSet translate to faster register >>> allocation, and faster C2 compiles. If you gut the IndexSet internals, >>> and replace it with BitMap, the sample performance runs on Nashorn >>> running Octane suite yield reliable improvements in compilation speed >>> (average: 22.7 Kb/s -> 24.4 Kb/s), explained by the decrease in C2 >>> regalloc time (average: 155s -> 132s). >>> >>> These improvements are in line with predicted improvements from a trial >>> experiment of "coarsening" the IndexSet, see the relevant RFE: >>> https://bugs.openjdk.java.net/browse/JDK-8059461 >>> >>> In other words, we can have a performance-improving change which also >>> removes lots of code. Or, a cleanup change, which also improves >>> performance. Here it is: >>> http://cr.openjdk.java.net/~shade/8059461/webrev.02/ >>> >>> The patch is mostly proof-of-concept, and not ready for commit. Please >>> let me know what you think about the approach. Code/style/idea >>> suggestions are welcome. >>> >>> Brief summary of changes: >>> - Almost all contents of IndexSet are purged, and delegated to BitMap >>> - IndexSetIterator performance is important, and therefore the lookup >>> table approach from IndexSetIterator was transplanted to new >>> BitMapIterator. We might want to improve BitMap::get_next_one_offset >>> with lookup tables as well, but that will contaminate the current >>> experiment. >>> - lrg_union was moved to appropriate place (why was it in IndexSet to >>> begin with?) >>> - some of IndexSet memory management / tracing functions were purged >>> >>> The testing so far was very light: >>> - smoke tests with JPRT >>> - Nashorn/Octane benchmarks >>> >>> Thanks, >>> -Aleksey. >>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From vitalyd at gmail.com Tue Nov 11 19:27:30 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Tue, 11 Nov 2014 14:27:30 -0500 Subject: Processor support for select operations In-Reply-To: References: <5461C9E8.1070809@gmail.com> <5462526D.1010302@gmail.com> <535F47A2-C478-4A16-AC57-FE3DF2FC7918@oracle.com> Message-ID: Don't mean to sidetrack this thread, but I really hope Unsafe can still be obtained by non-JDK code in java 9. Besides the effort required to move existing code off it, there's also the need to then have "native" java code performance match what Unsafe offers, which is a tall order since Unsafe gets its performance advantage by dropping all the safety belts. Therefore, I hope you guys don't fully lock it up and still allow access to it. It already has warning labels attached to it and given that java does not provide language based "unsafe" regions, this is the only way to achieve that. On Tue, Nov 11, 2014 at 2:17 PM, John Rose wrote: > On Nov 11, 2014, at 11:00 AM, Vitaly Davidovich wrote: > > But in the meantime, just trying Unsafe to see what the perf benefit would > be is still interesting, no? > > Yep. > > It's worth repeating: Unsafe is only for internal use by the JDK. The > access restrictions on it, > which are often circumvented, are going to be strengthened by the JDK 9 > module system, > and the restrictions will become harder to circumvent. > > Use it for experiments if that's useful, but don't build it into your > product. > > For more information, check out Paul Sandoz's talk on unsafe and > varhandles at Java One 2014. > > ? John > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.levart at gmail.com Sat Nov 8 15:07:38 2014 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 08 Nov 2014 16:07:38 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> Message-ID: <545E31BA.3070500@gmail.com> Hi David, As previously, I feel competent to comment only the Java side of the patch. Using linked list to publish to VM is a safer and easier way for the desired purpose and using a separate data structure for interning makes it easier to change it in the future should need arise. That need may just be around the corner as there are people (Martin Buchholz, Duncan MacGregor, for example) that use classes with huge number of methods... Here are few comments for the new webrev (MemberName): 78 @SuppressWarnings("rawtypes") //Comparable in next line 79 /*non-public*/ final class MemberName implements Member, Comparable, Cloneable { Since MemberName is final and can only be compared to itself, you could make it implement Comparable and eliminate the @SuppressWarnings more MemberName: 84 private volatile MemberName next; // used for a linked list of MemberNames known to VM ... 1375 private static class ClassData { 1376 /** 1377 * This needs to be a simple data structure because we need to access 1378 * and update its elements from the JVM. Note that the Java side controls 1379 * the allocation and order of elements in the array; the JVM modifies 1380 * fields of those elements during class redefinition. 1381 */ 1382 private volatile MemberName[] elementData; 1383 private volatile MemberName publishedToVM; 1384 private volatile int size; 1385 1386 /** 1387 * Interns a member name in the member name table. 1388 * Returns null if a race with the jvm occurred. Races are detected 1389 * by checking for changes in the class redefinition count that occur 1390 * before an intern is complete. 1391 * 1392 * @param klass class whose redefinition count is checked. 1393 * @param memberName member name to be interned 1394 * @param redefined_count the value of classRedefinedCount() observed before 1395 * creation of the MemberName that is being interned. 1396 * @return null if a race occurred, otherwise the interned MemberName. 1397 */ 1398 @SuppressWarnings({"unchecked","rawtypes"}) 1399 public MemberName intern(Class klass, MemberName memberName, int redefined_count) { 1400 if (elementData == null) { 1401 synchronized (this) { 1402 if (elementData == null) { 1403 elementData = new MemberName[1]; 1404 } 1405 } 1406 } 1407 synchronized (this) { // this == ClassData 1408 final int index = Arrays.binarySearch(elementData, 0, size, memberName); 1409 if (index >= 0) { 1410 return elementData[index]; 1411 } 1412 // Not found, add carefully. 1413 return add(klass, ~index, memberName, redefined_count); 1414 } 1415 } ... 1426 private MemberName add(Class klass, int index, MemberName e, int redefined_count) { 1427 // First attempt publication to JVM, if that succeeds, 1428 // then record internally. 1429 e.next = publishedToVM; 1430 publishedToVM = e; 1431 storeFence(); 1432 if (redefined_count != jla.getClassRedefinedCount(klass)) { 1433 // Lost a race, back out publication and report failure. 1434 publishedToVM = e.next; 1435 return null; 1436 } Since you now synchronize lookup/add *and* lazy elementData construction on the same object (the ClassData instance), you can merge two synchronized blocks and simplify code. You can make MemberName.next, ClassData.elementData and ClassData.size be non-volatile (just ClassData.publishedToVM needs to be volatile) and ClassData.intern() can look something like that: public synchronized MemberName intern(Class klass, MemberName memberName, int redefined_count) { final int index; if (elementData == null) { elementData = new MemberName[1]; index = ~0; } else { index = Arrays.binarySearch(elementData, 0, size, memberName); if (index >= 0) return elementData[index]; } // Not found, add carefully. return add(klass, ~index, memberName, redefined_count); } // Note: no need for additional storeFence() in add()... private MemberName add(Class klass, int index, MemberName e, int redefined_count) { // First attempt publication to JVM, if that succeeds, // then record internally. e.next = publishedToVM; // volatile read of publishedToVM, followed by normal write of e.next... publishedToVM = e; // ...which is ordered before volatile write of publishedToVM... if (redefined_count != jla.getClassRedefinedCount(klass)) { // ...which is ordered before volatile read of klass.classRedefinedCount. // Lost a race, back out publication and report failure. publishedToVM = e.next; return null; } ... Now let's take for example one of the MemberName.make() methods that return interned MemberNames: 206 public static MemberName make(Method m, boolean wantSpecial) { 207 // Unreflected member names are resolved so intern them here. 208 MemberName tmp0 = null; 209 InternTransaction tx = new InternTransaction(m.getDeclaringClass()); 210 while (tmp0 == null) { 211 MemberName tmp = new MemberName(m, wantSpecial); 212 tmp0 = tx.tryIntern(tmp); 213 } 214 return tmp0; 215 } I'm trying to understand the workings of InternTransaction helper class (and find an example that breaks it). You create an instance of it, passing Method's declaringClass. You then (in retry loop) create a resolved MemberName from the Method and wantSpecial flag. This MemberName's clazz can apparently differ from Method's declaringClass. I don't know when and why this happens, but apparently it can (super method?), so in InternTransaction.tryIntern() you do... 363 if (member_name.isResolved()) { 364 if (member_name.clazz != tx_class) { 365 Class prev_tx_class = tx_class; 366 int prev_txn_token = txn_token; 367 tx_class = member_name.clazz; 368 txn_token = internTxnToken(tx_class); 369 // Zero is a special case. 370 if (txn_token != 0 || 371 prev_txn_token != internTxnToken(prev_tx_class)) { 372 // Resolved class is different and at least one 373 // redef of it occurred, therefore repeat with 374 // proper class for race consistency checking. 375 return null; 376 } 377 } 378 member_name = member_name.intern(txn_token); 379 if (member_name == null) { 380 // Update the token for the next try. 381 txn_token = internTxnToken(tx_class); 382 } 383 } Now let's assume that the resolved member_name.clazz differs from Method's declaringClass. Let's assume also that either member_name.clazz has had at least one redefinition or Method's declaringClass has been redefined between creating InternTransaction and reading member_name.clazz's txn_token. You return 'null' in such case, concluding that not only the resolved member_name.clazz redefinition matters, but Method's declaringClass redefinition can also invalidate resolved MemberName am I right? It would be helpful if I could understand when and how Method's declaringClass redefinition can affect member_name. Can it affect which clazz is resolved for member_name? Anyway, you return null in such case from an updated InternTransaction (tx_class and txn_token are now updated to have values for resolved member_name.clazz). In next round the checks of newly constructed and resolved member_name are not performed against Method's declaringClass but against previous round's member_name.clazz. Is this what is intended? I can see there has to be a stop condition for loop to end, but shouldn't checks for Method's declaringClass redefinition be performed in every iteration (in addition to the check for member_name.clazz redefinition if it differs from Method's declaringClass)? Regards, Peter On 11/07/2014 10:14 PM, David Chase wrote: > New webrev: > > bug:https://bugs.openjdk.java.net/browse/JDK-8013267 > > webrevs: > http://cr.openjdk.java.net/~drchase/8013267/jdk.06/ > http://cr.openjdk.java.net/~drchase/8013267/hotspot.06/ > > Changes since last: > > 1) refactored to put ClassData under java.lang.invoke.MemberName > > 2) split the data structure into two parts; handshake with JVM uses a linked list, > which makes for a simpler backout-if-race, and Java side continues to use the > simple sorted array. This should allow easier use of (for example) fancier > data structures (like ConcurrentHashMap) if this later proves necessary. > > 3) Cleaned up symbol references in the new hotspot code to go through vmSymbols. > > 4) renamed oldCapacity to oldSize > > 5) ran two different benchmarks and saw no change in performance. > a) nashorn ScriptTest (seehttps://bugs.openjdk.java.net/browse/JDK-8014288 ) > b) JMH microbenchmarks > (see bug comments for details) > > And it continues to pass the previously-failing tests, as well as the new test > which has been added to hotspot/test/compiler/jsr292 . > > David > > On 2014-11-04, at 3:54 PM, David Chase wrote: > >> I?m working on the initial benchmarking, and so far this arrangement (with synchronization >> and binary search for lookup, lots of barriers and linear cost insertion) has not yet been any >> slower. >> >> I am nonetheless tempted by the 2-tables solution, because I think the simpler JVM-side >> interface that it allows is desirable. >> >> David >> >> On 2014-11-04, at 11:48 AM, Peter Levart wrote: >> >>> On 11/04/2014 04:19 PM, David Chase wrote: >>>> On 2014-11-04, at 5:07 AM, Peter Levart wrote: >>>>> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? >>>> It can?t be an identityHashMap, because we are interning member names. >>> I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. >>> >>>> In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. >>>> One possibility would be to use two data structures, one for interning, the other for communication with the VM. >>>> Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, >>>> and the synchronization dance is much simpler. >>>> >>>> For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: >>>> >>>> mn = resolve(args) >>>> // deal with any errors >>>> mn? = chm.get(mn) >>>> if (mn? != null) return mn? // hoped-for-common-case >>>> >>>> synchronized (something) { >>>> mn? = chm.get(mn) >>>> if (mn? != null) return mn? >>>> txn_class = mn.getDeclaringClass() >>>> >>>> while (true) { >>>> redef_count = txn_class.redefCount() >>>> mn = resolve(args) >>>> >>>> shared_array.add(mn); >>>> // barrier, because we are a paranoid >>>> if (redef_count = redef_count.redefCount()) { >>>> chm.add(mn); // safe to publish to other Java threads. >>>> return mn; >>>> } >>>> shared_array.drop_last(); // Try again >>>> } >>>> } >>>> >>>> (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). >>> Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. >>> >>> A combination of the two approaches is also possible: >>> >>> - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) >>> - use ConcurrentHashMap for interning. >>> >>> Regards, Peter >>> >>>> David >>>> >>>>>> And another way to view this is that we?re now quibbling about performance, when we still >>>>>> have an existing correctness problem that this patch solves, so maybe we should just get this >>>>>> done and then file an RFE. >>>>> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >>>>> >>>>> Regards, Peter >>>>> >>>>>> David -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.levart at gmail.com Sun Nov 9 12:55:10 2014 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 09 Nov 2014 13:55:10 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> Message-ID: <545F642E.30205@gmail.com> Hi David, I played a little with the idea of having a hash table instead of packed sorted array for interning. Using ConcurrentHashMap would present quite some memory overhead. A more compact representation is possible in the form of a linear-scan hash table where elements of array are MemberNames themselves: http://cr.openjdk.java.net/~plevart/misc/MemberName.intern/jdk.06.diff/ This is a drop-in replacement for MemberName on top of your jdk.06 patch. If you have some time, you can run this with your performance tests to see if it presents any difference. If not, then perhaps this interning is not so performance critical after all. Regards, Peter On 11/07/2014 10:14 PM, David Chase wrote: > New webrev: > > bug: https://bugs.openjdk.java.net/browse/JDK-8013267 > > webrevs: > http://cr.openjdk.java.net/~drchase/8013267/jdk.06/ > http://cr.openjdk.java.net/~drchase/8013267/hotspot.06/ > > Changes since last: > > 1) refactored to put ClassData under java.lang.invoke.MemberName > > 2) split the data structure into two parts; handshake with JVM uses a linked list, > which makes for a simpler backout-if-race, and Java side continues to use the > simple sorted array. This should allow easier use of (for example) fancier > data structures (like ConcurrentHashMap) if this later proves necessary. > > 3) Cleaned up symbol references in the new hotspot code to go through vmSymbols. > > 4) renamed oldCapacity to oldSize > > 5) ran two different benchmarks and saw no change in performance. > a) nashorn ScriptTest (see https://bugs.openjdk.java.net/browse/JDK-8014288 ) > b) JMH microbenchmarks > (see bug comments for details) > > And it continues to pass the previously-failing tests, as well as the new test > which has been added to hotspot/test/compiler/jsr292 . > > David > > On 2014-11-04, at 3:54 PM, David Chase wrote: > >> I?m working on the initial benchmarking, and so far this arrangement (with synchronization >> and binary search for lookup, lots of barriers and linear cost insertion) has not yet been any >> slower. >> >> I am nonetheless tempted by the 2-tables solution, because I think the simpler JVM-side >> interface that it allows is desirable. >> >> David >> >> On 2014-11-04, at 11:48 AM, Peter Levart wrote: >> >>> On 11/04/2014 04:19 PM, David Chase wrote: >>>> On 2014-11-04, at 5:07 AM, Peter Levart wrote: >>>>> Are you thinking of an IdentityHashMap type of hash table (no linked-list of elements for same bucket, just search for 1st free slot on insert)? The problem would be how to pre-size the array. Count declared members? >>>> It can?t be an identityHashMap, because we are interning member names. >>> I know it can't be IdentityHashMap - I just wondered if you were thinking of an IdentityHashMap-like data structure in contrast to standard HashMap-like. Not in terms of equality/hashCode used, but in terms of internal data structure. IdentityHashMap is just an array of elements (well pairs of them - key, value are placed in two consecutive array slots). Lookup searches for element linearly in the array starting from hashCode based index to the element if found or 1st empty array slot. It's very easy to implement if the only operations are get() and put() and could be used for interning and as a shared structure for VM to scan, but array has to be sized to at least 3/2 the number of elements for performance to not degrade. >>> >>>> In spite of my grumbling about benchmarking, I?m inclined to do that and try a couple of experiments. >>>> One possibility would be to use two data structures, one for interning, the other for communication with the VM. >>>> Because there?s no lookup in the VM data stucture it can just be an array that gets elements appended, >>>> and the synchronization dance is much simpler. >>>> >>>> For interning, maybe I use a ConcurrentHashMap, and I try the following idiom: >>>> >>>> mn = resolve(args) >>>> // deal with any errors >>>> mn? = chm.get(mn) >>>> if (mn? != null) return mn? // hoped-for-common-case >>>> >>>> synchronized (something) { >>>> mn? = chm.get(mn) >>>> if (mn? != null) return mn? >>>> txn_class = mn.getDeclaringClass() >>>> >>>> while (true) { >>>> redef_count = txn_class.redefCount() >>>> mn = resolve(args) >>>> >>>> shared_array.add(mn); >>>> // barrier, because we are a paranoid >>>> if (redef_count = redef_count.redefCount()) { >>>> chm.add(mn); // safe to publish to other Java threads. >>>> return mn; >>>> } >>>> shared_array.drop_last(); // Try again >>>> } >>>> } >>>> >>>> (Idiom gets slightly revised for the one or two other intern use cases, but this is the basic idea). >>> Yes, that's similar to what I suggested by using a linked-list of MemberName(s) instead of the "shared_array" (easier to reason about ordering of writes) and a sorted array of MemberName(s) instead of the "chm" in your scheme above. ConcurrentHashMap would certainly be the most performant solution in terms of lookup/insertion-time and concurrent throughput, but it will use more heap than a simple packed array of MemberNames. CHM is much better now in JDK8 though regarding heap use. >>> >>> A combination of the two approaches is also possible: >>> >>> - instead of maintaining a "shared_array" of MemberName(s), have them form a linked-list (you trade a slot in array for 'next' pointer in MemberName) >>> - use ConcurrentHashMap for interning. >>> >>> Regards, Peter >>> >>>> David >>>> >>>>>> And another way to view this is that we?re now quibbling about performance, when we still >>>>>> have an existing correctness problem that this patch solves, so maybe we should just get this >>>>>> done and then file an RFE. >>>>> Perhaps, yes. But note that questions about JMM and ordering of writes to array elements are about correctness, not performance. >>>>> >>>>> Regards, Peter >>>>> >>>>>> David -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.levart at gmail.com Tue Nov 11 19:30:10 2014 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 11 Nov 2014 20:30:10 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <1D96462F-87D3-4794-A818-27DD2EE10046@oracle.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545E31BA.3070500@gmail.com> <1D96462F-87D3-4794-A818-27DD2EE10046@oracle.com> Message-ID: <546263C2.1060508@gmail.com> On 11/11/2014 07:58 PM, David Chase wrote: > I?ve incorporated your other changes (not yet the linear-scan hash table) and will be retesting. > One thing I wonder about for both hash table and binary search is if the first try should be attempted with no lock to avoid the overhead of synchronization; I expect that looking will be more common than interning, which in turn will be (vastly) more common than class redefinition. Hi David, Yes, that's why I implemented the hash table in a way where lookups are lock-free. Binary-search would be trickier to implement without locking, but maybe not impossible. Surely not with Arrays.binarySearch() but perhaps with a separate implementation. The problem with Arrays.binarySearch is that it returns an index. By the time you retrieve the element at that index, it can already move. I'm also not sure that "careful" concurrent insertion of new element would not break the correctness of binary search. But there is another way I showed before: using StampedLock. It is a kind of optimistic/pessimistic read-write lock. Its beauty is in that optimistic read part is almost free (just a volatile read at start and a readFence followed by another volatile read at the end). You just have to be sure that the algorithm guarded by an optimistic read lock terminates normally (that it doesn't spin in an endless loop or throw exceptions) in the presence of arbitrary concurrent modifications of looked-up state. Well, binary search is such an algorithm. Regards, Peter > David -------------- next part -------------- An HTML attachment was scrubbed... URL: From roland.westrelin at oracle.com Wed Nov 12 10:16:17 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Wed, 12 Nov 2014 11:16:17 +0100 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <545D5103.9020309@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> <545BF25E.7010505@oracle.com> <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> <545D5103.9020309@oracle.com> Message-ID: Vladimir, New webrev with your suggested change: http://cr.openjdk.java.net/~roland/8054478/webrev.02/ I also had to make BoolTest::dump_on available in product builds for that code: 138 } else { 139 stringStream ss; 140 test.dump_on(&ss); 141 fatal(err_msg_res("unexpected comparison %s", ss.as_string())); 142 } in castnode.cpp I did a refworkload run on x64: ============================================================================ tbase: reference_server Benchmark Samples Mean Stdev jetstream 10 102.74 0.10 Write 10 202.90 0.94 Parse 10 37.10 0.05 Read 10 23.20 0.13 rw.runtime 10 10.60 0.05 Copy 10 641.70 0.10 scimark 10 1600.34 0.02 LU 10 3776.61 0.00 FFT 10 405.76 0.02 Monte 10 775.88 0.00 SOR 10 1275.16 0.01 Sparse 10 1768.28 0.08 rw.runtime 10 27.20 0.02 specjbb2000 10 487411.56 0.04 Last_Warehouse 10 487411.58 0.04 First_Warehouse 10 95725.01 0.02 rw.runtime 10 603.90 0.00 specjbb2005 10 480939.78 0.01 last 10 480939.79 0.01 interval_average 10 5937.40 0.01 peak 10 534158.07 0.02 overall_average 10 431366.64 0.02 last_warehouse 10 8.00 0.00 peak_warehouse 10 2.30 0.21 first 10 50951.62 0.03 rw.runtime 10 461.60 0.00 specjvm98 10 801.92 0.04 compress 10 604.68 0.10 javac 10 405.68 0.17 db 10 449.78 0.00 jack 10 631.14 0.05 mtrt 10 2342.41 0.03 jess 10 957.35 0.02 rw.runtime 10 41.10 0.04 mpegaudio 10 1385.76 0.00 volano25 10 327418.69 0.06 time 10 2.45 0.05 connections 10 400.00 0.00 rw.runtime 10 26.10 0.01 -------------------------------------------------------------------------- Weighted Geomean 31199.92 ============================================================================ tnew: reference_server Benchmark Samples Mean Stdev %Diff P Significant jetstream 10 85.82 0.12 -16.47 0.002 Yes Write 10 219.60 0.31 -8.23 0.799 * Parse 10 36.90 0.06 0.54 0.836 * Read 10 23.60 0.10 -1.72 0.741 * rw.runtime 10 13.80 0.03 -30.19 0.000 Yes Copy 10 1058.70 0.26 -64.98 0.001 Yes scimark 10 1583.22 0.00 -1.07 0.110 * LU 10 3778.49 0.00 0.05 0.160 * FFT 10 407.80 0.01 0.50 0.498 * Monte 10 775.71 0.00 -0.02 0.279 * SOR 10 1276.75 0.01 0.12 0.742 * Sparse 10 1677.37 0.01 -5.14 0.086 * rw.runtime 10 27.50 0.02 -1.10 0.265 * specjbb2000 10 491672.03 0.04 0.87 0.624 * Last_Warehouse 10 491672.04 0.04 0.87 0.624 * First_Warehouse 10 94172.03 0.02 -1.62 0.050 * rw.runtime 10 604.00 0.00 -0.02 0.585 * specjbb2005 10 481051.28 0.01 0.02 0.945 * last 10 481051.29 0.01 0.02 0.945 * interval_average 10 5938.90 0.01 0.03 0.940 * peak 10 538706.19 0.03 0.85 0.461 * overall_average 10 434244.96 0.02 0.67 0.535 * last_warehouse 10 8.00 0.00 -0.00 0.000 * peak_warehouse 10 2.00 0.00 13.04 0.081 * first 10 51039.34 0.03 0.17 0.889 * rw.runtime 10 462.20 0.00 -0.13 0.120 * specjvm98 10 806.31 0.04 0.55 0.738 * compress 10 623.61 0.10 3.13 0.494 * javac 10 402.37 0.10 -0.81 0.898 * db 10 450.58 0.00 0.18 0.327 * jack 10 627.02 0.05 -0.65 0.785 * mtrt 10 2281.86 0.03 -2.58 0.106 * jess 10 1000.65 0.10 4.52 0.220 * rw.runtime 10 40.90 0.03 0.49 0.761 * mpegaudio 10 1384.19 0.00 -0.11 0.514 * volano25 10 324906.00 0.08 -0.77 0.799 * time 10 2.47 0.07 -1.00 0.733 * connections 10 400.00 0.00 0.00 0.000 * rw.runtime 10 26.50 0.02 -1.53 0.058 * -------------------------------------------------------------------------- Weighted Geomean 30613.61 -1.88 ============================================================================ tbase is current hotspot-comp. tnew is with the change. It?s ok, right? Roland. > On Nov 8, 2014, at 12:08 AM, Vladimir Kozlov wrote: > > This looks good but I just realized that what you are doing in Opaque1Node::Ideal() we usually do in PhaseIterGVN::add_users_to_worklist(). > > For example you can do there (for Phi): > > + uint use_op = use->Opcode(); > if( use->is_Cmp() ) { // Enable CMP/BOOL optimization > add_users_to_worklist(use); // Put Bool on worklist > - // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the > - // phi merging either 0 or 1 onto the worklist > if (use->outcnt() > 0) { > Node* bol = use->raw_out(0); > if (bol->outcnt() > 0) { > Node* iff = bol->raw_out(0); > if (use_op == Op_CmpI && iff->is_CountedLoopEnd() && > n->Opcode() == Op_CastII) { > + // If this opaque node feeds into the limit condition of a > + // CountedLoop, we need to process the Phi node for the > + // induction variable: the range of values taken by the Phi is > + // known now and so its type is also known. > + CountedLoopEndNode* cle = iff->as_CountedLoopEnd(); > + if (cle->limit() == this) { > + _worklist.push(cle->phi()); > + } > - if (iff->outcnt() == 2) { > + } else if (iff->outcnt() == 2) { > + // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the > + // phi merging either 0 or 1 onto the worklist > > > Thanks, > Vladimir > > > On 11/7/14 2:17 PM, Roland Westrelin wrote: >> Vladimir, >> >> Thanks for the discussion, suggestions and fixes. Here is an updated webrev: >> >> http://cr.openjdk.java.net/~roland/8054478/webrev.01/ >> >> Roland. >> >>> On Nov 6, 2014, at 11:12 PM, Vladimir Kozlov wrote: >>> >>> On 11/6/14 7:39 AM, Roland Westrelin wrote: >>>> >>>>> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. >>>> >>>> I?ll send an updated webrev. >>>> >>>>>>>>> opaquenode.cpp >>>>>>>>> >>>>>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>>>>> >>>>>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>>>>> >>>>>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>>>>> >>>>>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >>>>> >>>>> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. >>>> >>>> (I know you don?t like when I remove the text from previous emails but that was really getting confusing) >>> >>> At least leave webrev link so I don't need to search for it in previous mails. >>> >>>> >>>> PhiNode::Value() has this code: >>>> >>>> // Check for trip-counted loop. If so, be smarter. >>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>> if( l && l->can_be_counted_loop(phase) && >>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>> // protect against init_trip() or limit() returning NULL >>>> const Node *init = l->init_trip(); >>>> const Node *limit = l->limit(); >>>> if( init != NULL && limit != NULL && l->stride_is_con() ) { >>>> const TypeInt *lo = init ->bottom_type()->isa_int(); >>>> const TypeInt *hi = limit->bottom_type()->isa_int(); >>>> if( lo && hi ) { // Dying loops might have TOP here >>>> int stride = l->stride_con(); >>>> if( stride < 0 ) { // Down-counter loop >>>> const TypeInt *tmp = lo; lo = hi; hi = tmp; >>>> stride = -stride; >>>> } >>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>> } >>>> } >>>> } >>>> >>>> That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. >>> >>> My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: >>> >>> // Step B2: Build a zero-trip guard for the main-loop. After leaving the >>> // pre-loop, the main-loop may not execute at all. Later in life this >>> // zero-trip guard will become the minimum-trip guard when we unroll >>> // the main-loop. >>> Node *min_opaq = new Opaque1Node(C, limit); >>> Node *min_cmp = new CmpINode( pre_incr, min_opaq ); >>> >>> >>> But you are talking about pre-loop exit check: >>> >>> // Step B4: Shorten the pre-loop to run only 1 iteration (for now). >>> // RCE and alignment may change this later. >>> Node *cmp_end = pre_end->cmp_node(); >>> assert( cmp_end->in(2) == limit, "" ); >>> Node *pre_limit = new AddINode( init, stride ); >>> >>> // Save the original loop limit in this Opaque1 node for >>> // use by range check elimination. >>> Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); >>> >>> >>> But in this case you can add check (cl->limit() == this) to be clear what you are looking for. >>> Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. >>> >>> Thanks, >>> Vladimir >>> >>>> >>>> In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. >>>> >>>> Roland. >> From roland.westrelin at oracle.com Wed Nov 12 10:50:30 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Wed, 12 Nov 2014 11:50:30 +0100 Subject: gc bugs after 8060252: JDK-7173584 compiler changes regress SPECjvm2008 on SPARC In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF26315@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF26315@DEWDFEMB12A.global.corp.sap> Message-ID: Hi Goetz, > on ppc, I see immediate bugs in g1: > > ppc_vm/bin/java -XX:+UseG1GC -XX:SurvivorRatio=4 -classpath ?/benchmarks/jvm98/ SPECjvm98All ?/benchmarks/jvm98/ jvm98.log.txt jvm98.result.txt javac > > # Internal Error (/sapmnt/home1/d045726/oJ/g1Bug-hs-comp/src/share/vm/oops/oop.inline.hpp:199), pid=1554, tid=4398079689280 > # assert(check_obj_alignment(result)) failed: address not aligned: 0x00000000baadbabe > > V [libjvm.so+0xaa5150] report_vm_error(char const*, int, char const*, char const*)+0xdc > V [libjvm.so+0x78e738] oopDesc::decode_heap_oop_not_null(unsigned int)+0x11c > V [libjvm.so+0x78e7f8] oopDesc::decode_heap_oop(unsigned int)+0x6c > V [libjvm.so+0xc41238] void G1SATBCardTableModRefBS::write_ref_field_pre_static(unsigned int*, oopDesc*)+0x7c > V [libjvm.so+0xc412cc] void G1SATBCardTableModRefBS::inline_write_ref_field_pre(unsigned int*, oopDesc*)+0x40 > V [libjvm.so+0xc41354] G1SATBCardTableModRefBS::write_ref_field_pre_work(unsigned int*, oopDesc*)+0x44 > V [libjvm.so+0x8dadcc] void BarrierSet::write_ref_field_pre(unsigned int*, oopDesc*)+0xac > V [libjvm.so+0x1195044] void ObjArrayKlass::do_copy(arrayOopDesc*, unsigned int*, arrayOopDesc*, unsigned int*, int, Thread*)+0x3f4 > V [libjvm.so+0x1191c68] ObjArrayKlass::copy_array(arrayOopDesc*, int, arrayOopDesc*, int, int, Thread*)+0x27c > V [libjvm.so+0xeac84c] JVM_ArrayCopy+0xff51fc1c > J 17 java.lang.System.arraycopy(Ljava/lang/Object;ILjava/lang/Object;II)V (0 bytes) @ 0x00000400021bd104 [0x00000400021bd000+0x104] > j spec.benchmarks._213_javac.Parser.exprArgs(I)[Lspec/benchmarks/_213_javac/Expression;+23 > j spec.benchmarks._213_javac.Parser.parseMethodExpression(Lspec/benchmarks/_213_javac/Expression;Lspec/benchmarks/_213_javac/Identifier;)Lspec/benchmarks/_213_javac/Expression;+69 > j spec.benchmarks._213_javac.Parser.parseExpression()Lspec/benchmarks/_213_javac/Expression;+426 > > Did anybody see similar problems? I have a similar crash on x86 with javac?s specjvm98 and G1. Thanks for reporting it. I will investigate what?s going on. Roland. From vladimir.x.ivanov at oracle.com Wed Nov 12 11:46:55 2014 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 12 Nov 2014 15:46:55 +0400 Subject: [9] RFR (XXS): 8062258: compiler/debug/TraceIterativeGVN.java segfaults in trace_PhaseIterGVN In-Reply-To: <5452BB80.9030509@oracle.com> References: <5450D639.2050506@oracle.com> <54510129.8080108@oracle.com> <545274D0.3050702@oracle.com> <54529090.1030200@oracle.com> <5452A41D.1060107@oracle.com> <5452BB80.9030509@oracle.com> Message-ID: <546348AF.4040600@oracle.com> Vladimir, What do you think about the following version: http://cr.openjdk.java.net/~vlivanov/8062258/webrev.02 I decided to revert changes in dumping logic. IMO possible duplication of output is a lesser problem than a complication in dumping logic. Best regards, Vladimir Ivanov On 10/31/14, 2:28 AM, Vladimir Kozlov wrote: > Unfortunately not only MemNode has Type::MEMORY type. PhiNode, > SCMemProjNode and others. It looks like you need to fix all places. > Change in memnode.cpp is fine but in node.cpp you need: > > } else if (t == Type::MEMORY && (!is_Mem() || is_LoadStore())) { > st->print(" Memory:"); > MemNode::dump_adr_type(this, adr_type(), st); > > Make sure that other Type::MEMORY nodes don't crash in adr_type(). > Don't do check under NOT_PRODUCT as in your first fix - just return NULL > in all cases. > > Thanks, > Vladimir > > On 10/30/14 1:48 PM, Vladimir Ivanov wrote: >> Vladimir, nice observation! >> >> http://cr.openjdk.java.net/~vlivanov/8062258/webrev.01/ >> >> Best regards, >> Vladimir Ivanov >> >> On 10/30/14, 11:25 PM, Vladimir Kozlov wrote: >>> Hmm, next code in PhaseIterGVN::optimize() worries me. We dump dead >>> nodes before removing them from graph: >>> >>> DEBUG_ONLY(trace_PhaseIterGVN_verbose(n, num_processed++);) >>> if (n->outcnt() != 0) { >>> NOT_PRODUCT(const Type* oldtype = type_or_null(n)); >>> // Do the transformation >>> Node* nn = transform_old(n); >>> NOT_PRODUCT(trace_PhaseIterGVN(n, nn, oldtype);) >>> } else if (!n->is_top()) { >>> remove_dead_node(n); >>> } >>> >>> I am changing my suggestion: fix Node::dump() which calls adr_type() >>> without check. It should not call MemNode::dump_adr_type() because >>> MemNode::dump_spec(), which is called before, already does that and it >>> has NULL check! If you want, you can add " Memory:" output in >>> MemNode::dump_spec() to be similar to Node::dump() output. >>> >>> Thanks, >>> Vladimir >>> >>> On 10/30/14 10:26 AM, Vladimir Ivanov wrote: >>>> Vladimir, do you suggest to simply skip printing info about dead nodes >>>> w/ -XX:+TraceIterativeGVN? >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> On 10/29/14, 7:00 PM, Vladimir Kozlov wrote: >>>>> I would suggest to dead node check in trace_PhaseIterGVN() instead of >>>>> in MemNode::adr_type(). >>>>> So the fix for your JDK-8061995 should fix this problem too. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 10/29/14 4:57 AM, Vladimir Ivanov wrote: >>>>>> http://cr.openjdk.java.net/~vlivanov/8062258/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8062258 >>>>>> >>>>>> Trivial fix to avoid NULL dereference w/ -XX:+TraceIterativeGVN while >>>>>> querying adr_type() on dead MemNode. >>>>>> >>>>>> Testing: failing test. >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov From roland.westrelin at oracle.com Wed Nov 12 13:15:44 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Wed, 12 Nov 2014 14:15:44 +0100 Subject: gc bugs after 8060252: JDK-7173584 compiler changes regress SPECjvm2008 on SPARC In-Reply-To: References: <4295855A5C1DE049A61835A1887419CC2CF26315@DEWDFEMB12A.global.corp.sap> Message-ID: <6A8F58D2-6B64-4FB1-9CD2-AD4437A13F11@oracle.com> I filed the following bug: https://bugs.openjdk.java.net/browse/JDK-8064703 Roland. From evgeniya.stepanova at oracle.com Wed Nov 12 14:49:19 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 12 Nov 2014 18:49:19 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5458B960.8000305@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> Message-ID: <5463736F.7050109@oracle.com> Hi everyone! Since the decision was made to change only tests that fail because of conflict for now (skip "selfish" tests), I post new webrev for hotspot part of the JDK-8019361 : http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ Thanks, Evgeniya Stepanova On 04.11.2014 15:32, Dmitry Fazunenko wrote: > Nice plan! Please feel free to send me any feedback/questions > regarding @requires > > Thanks, > Dima > > > On 04.11.2014 11:40, Bengt Rutisson wrote: >> >> Hi Dima, >> >> Thanks for the answers. I think the currently proposed patch is a >> good start. We will have to evolve the @requires tag in the future, >> but let's have that discussion separate from this review. And we can >> start that discussion later when we have more experience with the >> current version of @requires. >> >> Thanks for doing this! >> Bengt >> >> >> >> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>> Hi Bengt, >>> >>> That's great that we have very closed visions! >>> >>> The general comment: currently, jtreg doesn't support any sort of >>> plugins, so you can't provide a VM specific handler of the @requires >>> or another tag. This is very annoying limitation and we have to live >>> with it. >>> >>> A few more comments inline. >>> >>> >>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>> >>>> >>>> Hi Dima, >>>> >>>> Answers inline. >>>> >>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>> Hi Bengt, >>>>> >>>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>>> >>>>> See comments inline. >>>>> >>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Evgeniya, >>>>>> >>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of >>>>>>> the JDK-8019361 >>>>>>> >>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>> >>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>>> another GC. >>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>> skip test if there is a conflict >>>>>> >>>>>> Thanks for fixing this! It is really great that we finally start >>>>>> sorting this out. >>>>>> >>>>>> First a general comment. The @requires tag has been developed >>>>>> without much cooperation with the GC team. We did have a lot of >>>>>> feedback when it was first presented a year ago, but it does not >>>>>> seem like this feedback was incorporated into the @requires that >>>>>> was eventually built. >>>>> >>>>> We tried to implement as much developer's wishes as possible. But >>>>> not everything is possible, sorry for that. >>>> >>>> Yes, I'm sure you have done your best. It's just that we have been >>>> requesting this feature for 3 years and I was expecting us to be >>>> able to influence the feature much more than was the case now. >>> >>> My personal hope: @requires will address ~90% of existing issues. >>> >>>> >>>>> >>>>>> >>>>>> I think this change that gets proposed now is a big step forward >>>>>> and I won't object to it. But I am pretty convinced that we will >>>>>> soon run in to the limitations of the current @requires >>>>>> implementation and we will have to redo this work. >>>>>> >>>>>> Some of the points I don't really like about the @requires tag are: >>>>>> >>>>>> - the "vm.gc" abstraction is more limiting than helping. It would >>>>>> have been better to just "require" any command line flag. >>>>> "vm.gc" is an alias to a very popular flag. It's also possible to >>>>> use: >>>>> vm.opt.UseG1GC == true instead. >>>>> >>>>> The table with all vars available in jtreg: >>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>> >>>> The problem with having this matching built in to JTreg is that it >>>> makes it very hard to change. When we discussed this a year ago I >>>> think we said that JTreg should only provide a means to test >>>> against the command line and a hook for running some java code in >>>> the @requires tag. That way we could put logic like this in a test >>>> library that is under our control. This would make it easy for us >>>> to change and also enables us to use different logic for different >>>> versions. >>> >>> I would be glad to have own harness... >>> >>>> >>>>> >>>>>> - the requirement should be per @run tag. Right now we have to do >>>>>> what you did in this change and use vm.gc=null even when some >>>>>> tests could actually have been run when a GC was specified. >>>>> it would be great, but it will unlikely happen in jtreg, as well >>>>> as test case support. >>>> >>>> what do you mean with test case support? Hi Evgeniya, >>> >>> Under test case support I mean ability to treat each @run as a >>> separate test. Now >>> >>> @test >>> @run -XX:g1RegSize=1m MyTest >>> @run -XX:g1RegSize=2m MyTest >>> @run -XX:g1RegSize=4m MyTest >>> class MyTest { >>> } >>> >>> is always a single test. You can't exclude, or re-run a part of it. >>> >>> >>>> >>>>> >>>>>> - there are many tests that require more than just a specific GC. >>>>>> Often there are other flags that can't be changed either for the >>>>>> test to work properly. >>>>> >>>>> yes. conflicting GC is just the most popular problem caused by >>>>> conflicting options. >>>>> If we address this issue and we are satisfied with solution, we >>>>> could move further. >>>> >>>> Yes, I agree that taking one step at the time is good. Personally I >>>> would have preferred that the first step was a "just run the >>>> command line as specified in the @run tag" step. >>>> >>>>> >>>>>> >>>>>> Maybe this is not the right place to discuss the current >>>>>> implementation of the @requires tag. I just want to say that I'm >>>>>> not too happy about how the @requires tag turned out. But >>>>>> assuming we have to use it the way it is now I guess the proposed >>>>>> changeset looks good. >>>>> >>>>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>>>> And thanks for reviewing it! >>>> >>>> Agreed. And as I said, I think the patch looks ok. I have not >>>> looked at all tests. But if they now pass with the combinations >>>> that we test with I guess they should be ok. >>> >>> Excellent! Thanks a lot! >>> >>>> >>>>> >>>>>> >>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>> without any GC flag). Tests are being excluded as expected. No >>>>>>> tests failed because of the conflict. >>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>> -XX:+UseConcMarkSweepGC. >>>>> >>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>> >>>> Ok. Thanks. >>>> >>>>> >>>>>> >>>>>> I think some of the test, like >>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run >>>>>> with -XX:+UseParNewGC. Others, like >>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you run >>>>>> with -XX:-UseParNewGC. Could you test these two cases too? >>>>> >>>>> These two tests ignore vm flags. >>>>> Add @requires here is not necessary, but it will allow not execute >>>>> the tests when not needed. >>>>> So, if we run HS tests with 4 GC, we don't need to run these tests >>>>> 4 times, 1 should be enough. >>>> >>>> Do we really want to use the @requires functionality for this >>>> purpose? It seems like a way of misusing @requires. If we just want >>>> the tests to be run once I think Leonid's approach with tests lists >>>> seems more suitable. >>> >>> No, it's not a purpose of course, it's just side effect :) >>> >>> >>>> But are you sure that this is the reason for the @requires in this >>>> case? TestDefNewCMS does sound like a test that is DefNew specific. >>>> I don't see a reason to run it with ParNew. If it doesn't fail >>>> today it should probably be changed so that it does fail if it is >>>> run with the wrong GC. >>> >>> @requires - is not the silver bullet, but it's quite easy way to >>> solve a lot of issues. >>> >>> I hope, @requires will allow to reduce the number of "selfish" >>> tests, which produce a new java process to ignore vm flags coming >>> from outside. No @requires, no other mechanism could 100% protect a >>> test from running with conflicting options, but this is not the goal. >>> >>> If one runs tests with an exotic option, like a new G2 collector, >>> there shouldn't mass failures caused by options conflicts. But a few >>> failures could be handled manually. >>> >>> >>>> >>>>> >>>>>> Similarly it looks to me like there are tests that will fail if >>>>>> you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>>> >>>>>> Just a heads up. These two tests will soon be removed. I'm about >>>>>> to push a changeset that removes them: >>>>>> >>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>> okay, thank for letting us know. >>>>> >>>>>> >>>>>> Is there some way of making sure that all tests are run at one >>>>>> time or another. With this change there is a risk that some tests >>>>>> are never run and always skipped. Will we somehow be tracking >>>>>> what gets skipped and make sure that all tests are at least run >>>>>> once with the correct GC so that it is not skipped all the time? >>>>> >>>>> This is a very good question! >>>>> jtreg now doesn't report skipped tests, hopefully it will do soon, >>>>> after getting fix of: >>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>> >>>>> And yes, tracking tests which are not run is important thing. >>>>> @requires - is not the only to exclude test from execution. >>>>> >>>>> Other examples: >>>>> >>>>> /* >>>>> *@ignore >>>>> *@test >>>>> */ >>>>> ... >>>>> >>>>> /*@bug 4445555 >>>>> *@test >>>>> */ >>>>> ... >>>>> Such tests will never be run, because jtreg treats as test only >>>>> files with @test on the first place... >>>>> >>>>> So, making sure that tests do not disappear is important SQE >>>>> task, we know about that, we're thinking on solution (may be very >>>>> actively). But this subject for another discussion, not within RFR :) >>>> >>>> Right. Glad to hear that you are actively working on this! >>> >>> I was going to say "not very actively", but never mind, we know >>> about this problem. With introducing @requires mechanism it will >>> become more important! >>> >>> >>> Thanks for your comments! >>> >>> -- Dima >>> >>> >>>> >>>> Bengt >>>> >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> >>>>>> >>>>> >>>> >>> >> > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Wed Nov 12 14:23:50 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Wed, 12 Nov 2014 18:23:50 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5463736F.7050109@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> Message-ID: <54636D76.3010905@oracle.com> Hi Evgeniya, The fix looks good to me. I noticed the following minor things: - copyrights need to include the year of last change - test/gc/defnew/HeapChangeLogging.java - is listed among updated files, but doesn't contain any changes - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable 'prohibitedVmOptions' Thanks, Dima On 12.11.2014 18:49, Evgeniya Stepanova wrote: > Hi everyone! > > Since the decision was made to change only tests that fail because of > conflict for now (skip "selfish" tests), I post new webrev for hotspot > part of the JDK-8019361 > : > http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ > > Thanks, > Evgeniya Stepanova > On 04.11.2014 15:32, Dmitry Fazunenko wrote: >> Nice plan! Please feel free to send me any feedback/questions >> regarding @requires >> >> Thanks, >> Dima >> >> >> On 04.11.2014 11:40, Bengt Rutisson wrote: >>> >>> Hi Dima, >>> >>> Thanks for the answers. I think the currently proposed patch is a >>> good start. We will have to evolve the @requires tag in the future, >>> but let's have that discussion separate from this review. And we can >>> start that discussion later when we have more experience with the >>> current version of @requires. >>> >>> Thanks for doing this! >>> Bengt >>> >>> >>> >>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>> Hi Bengt, >>>> >>>> That's great that we have very closed visions! >>>> >>>> The general comment: currently, jtreg doesn't support any sort of >>>> plugins, so you can't provide a VM specific handler of the >>>> @requires or another tag. This is very annoying limitation and we >>>> have to live with it. >>>> >>>> A few more comments inline. >>>> >>>> >>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>> >>>>> >>>>> Hi Dima, >>>>> >>>>> Answers inline. >>>>> >>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>> Hi Bengt, >>>>>> >>>>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>>>> >>>>>> See comments inline. >>>>>> >>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of >>>>>>>> the JDK-8019361 >>>>>>>> >>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>> >>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>>>> another GC. >>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>> skip test if there is a conflict >>>>>>> >>>>>>> Thanks for fixing this! It is really great that we finally start >>>>>>> sorting this out. >>>>>>> >>>>>>> First a general comment. The @requires tag has been developed >>>>>>> without much cooperation with the GC team. We did have a lot of >>>>>>> feedback when it was first presented a year ago, but it does not >>>>>>> seem like this feedback was incorporated into the @requires that >>>>>>> was eventually built. >>>>>> >>>>>> We tried to implement as much developer's wishes as possible. But >>>>>> not everything is possible, sorry for that. >>>>> >>>>> Yes, I'm sure you have done your best. It's just that we have been >>>>> requesting this feature for 3 years and I was expecting us to be >>>>> able to influence the feature much more than was the case now. >>>> >>>> My personal hope: @requires will address ~90% of existing issues. >>>> >>>>> >>>>>> >>>>>>> >>>>>>> I think this change that gets proposed now is a big step forward >>>>>>> and I won't object to it. But I am pretty convinced that we will >>>>>>> soon run in to the limitations of the current @requires >>>>>>> implementation and we will have to redo this work. >>>>>>> >>>>>>> Some of the points I don't really like about the @requires tag are: >>>>>>> >>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>> would have been better to just "require" any command line flag. >>>>>> "vm.gc" is an alias to a very popular flag. It's also possible to >>>>>> use: >>>>>> vm.opt.UseG1GC == true instead. >>>>>> >>>>>> The table with all vars available in jtreg: >>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>> >>>>> The problem with having this matching built in to JTreg is that it >>>>> makes it very hard to change. When we discussed this a year ago I >>>>> think we said that JTreg should only provide a means to test >>>>> against the command line and a hook for running some java code in >>>>> the @requires tag. That way we could put logic like this in a test >>>>> library that is under our control. This would make it easy for us >>>>> to change and also enables us to use different logic for different >>>>> versions. >>>> >>>> I would be glad to have own harness... >>>> >>>>> >>>>>> >>>>>>> - the requirement should be per @run tag. Right now we have to >>>>>>> do what you did in this change and use vm.gc=null even when some >>>>>>> tests could actually have been run when a GC was specified. >>>>>> it would be great, but it will unlikely happen in jtreg, as well >>>>>> as test case support. >>>>> >>>>> what do you mean with test case support? Hi Evgeniya, >>>> >>>> Under test case support I mean ability to treat each @run as a >>>> separate test. Now >>>> >>>> @test >>>> @run -XX:g1RegSize=1m MyTest >>>> @run -XX:g1RegSize=2m MyTest >>>> @run -XX:g1RegSize=4m MyTest >>>> class MyTest { >>>> } >>>> >>>> is always a single test. You can't exclude, or re-run a part of it. >>>> >>>> >>>>> >>>>>> >>>>>>> - there are many tests that require more than just a specific >>>>>>> GC. Often there are other flags that can't be changed either for >>>>>>> the test to work properly. >>>>>> >>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>> conflicting options. >>>>>> If we address this issue and we are satisfied with solution, we >>>>>> could move further. >>>>> >>>>> Yes, I agree that taking one step at the time is good. Personally >>>>> I would have preferred that the first step was a "just run the >>>>> command line as specified in the @run tag" step. >>>>> >>>>>> >>>>>>> >>>>>>> Maybe this is not the right place to discuss the current >>>>>>> implementation of the @requires tag. I just want to say that I'm >>>>>>> not too happy about how the @requires tag turned out. But >>>>>>> assuming we have to use it the way it is now I guess the >>>>>>> proposed changeset looks good. >>>>>> >>>>>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>>>>> And thanks for reviewing it! >>>>> >>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>> looked at all tests. But if they now pass with the combinations >>>>> that we test with I guess they should be ok. >>>> >>>> Excellent! Thanks a lot! >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and >>>>>>>> without any GC flag). Tests are being excluded as expected. No >>>>>>>> tests failed because of the conflict. >>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>> -XX:+UseConcMarkSweepGC. >>>>>> >>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>> >>>>> Ok. Thanks. >>>>> >>>>>> >>>>>>> >>>>>>> I think some of the test, like >>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>> >>>>>> These two tests ignore vm flags. >>>>>> Add @requires here is not necessary, but it will allow not >>>>>> execute the tests when not needed. >>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>> tests 4 times, 1 should be enough. >>>>> >>>>> Do we really want to use the @requires functionality for this >>>>> purpose? It seems like a way of misusing @requires. If we just >>>>> want the tests to be run once I think Leonid's approach with tests >>>>> lists seems more suitable. >>>> >>>> No, it's not a purpose of course, it's just side effect :) >>>> >>>> >>>>> But are you sure that this is the reason for the @requires in this >>>>> case? TestDefNewCMS does sound like a test that is DefNew >>>>> specific. I don't see a reason to run it with ParNew. If it >>>>> doesn't fail today it should probably be changed so that it does >>>>> fail if it is run with the wrong GC. >>>> >>>> @requires - is not the silver bullet, but it's quite easy way to >>>> solve a lot of issues. >>>> >>>> I hope, @requires will allow to reduce the number of "selfish" >>>> tests, which produce a new java process to ignore vm flags coming >>>> from outside. No @requires, no other mechanism could 100% protect a >>>> test from running with conflicting options, but this is not the goal. >>>> >>>> If one runs tests with an exotic option, like a new G2 collector, >>>> there shouldn't mass failures caused by options conflicts. But a >>>> few failures could be handled manually. >>>> >>>> >>>>> >>>>>> >>>>>>> Similarly it looks to me like there are tests that will fail if >>>>>>> you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>>>> >>>>>>> Just a heads up. These two tests will soon be removed. I'm about >>>>>>> to push a changeset that removes them: >>>>>>> >>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>> okay, thank for letting us know. >>>>>> >>>>>>> >>>>>>> Is there some way of making sure that all tests are run at one >>>>>>> time or another. With this change there is a risk that some >>>>>>> tests are never run and always skipped. Will we somehow be >>>>>>> tracking what gets skipped and make sure that all tests are at >>>>>>> least run once with the correct GC so that it is not skipped all >>>>>>> the time? >>>>>> >>>>>> This is a very good question! >>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>> soon, after getting fix of: >>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>> >>>>>> And yes, tracking tests which are not run is important thing. >>>>>> @requires - is not the only to exclude test from execution. >>>>>> >>>>>> Other examples: >>>>>> >>>>>> /* >>>>>> *@ignore >>>>>> *@test >>>>>> */ >>>>>> ... >>>>>> >>>>>> /*@bug 4445555 >>>>>> *@test >>>>>> */ >>>>>> ... >>>>>> Such tests will never be run, because jtreg treats as test only >>>>>> files with @test on the first place... >>>>>> >>>>>> So, making sure that tests do not disappear is important SQE >>>>>> task, we know about that, we're thinking on solution (may be very >>>>>> actively). But this subject for another discussion, not within >>>>>> RFR :) >>>>> >>>>> Right. Glad to hear that you are actively working on this! >>>> >>>> I was going to say "not very actively", but never mind, we know >>>> about this problem. With introducing @requires mechanism it will >>>> become more important! >>>> >>>> >>>> Thanks for your comments! >>>> >>>> -- Dima >>>> >>>> >>>>> >>>>> Bengt >>>>> >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Wed Nov 12 16:28:15 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Wed, 12 Nov 2014 20:28:15 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <54636D76.3010905@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> Message-ID: <54638A9F.2030209@oracle.com> Hi Dmitry, You are right - I've forgotten about copyrights Copyrights and other issues you mentioned fixed. New webrev: http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ Thanks Evgeniya Stepanova On 12.11.2014 18:23, Dmitry Fazunenko wrote: > Hi Evgeniya, > > The fix looks good to me. > > I noticed the following minor things: > - copyrights need to include the year of last change > - test/gc/defnew/HeapChangeLogging.java - is listed among updated > files, but doesn't contain any changes > - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable > 'prohibitedVmOptions' > > Thanks, > Dima > > > On 12.11.2014 18:49, Evgeniya Stepanova wrote: >> Hi everyone! >> >> Since the decision was made to change only tests that fail because of >> conflict for now (skip "selfish" tests), I post new webrev for >> hotspot part of the JDK-8019361 >> : >> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >> >> Thanks, >> Evgeniya Stepanova >> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>> Nice plan! Please feel free to send me any feedback/questions >>> regarding @requires >>> >>> Thanks, >>> Dima >>> >>> >>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>> >>>> Hi Dima, >>>> >>>> Thanks for the answers. I think the currently proposed patch is a >>>> good start. We will have to evolve the @requires tag in the future, >>>> but let's have that discussion separate from this review. And we >>>> can start that discussion later when we have more experience with >>>> the current version of @requires. >>>> >>>> Thanks for doing this! >>>> Bengt >>>> >>>> >>>> >>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>> Hi Bengt, >>>>> >>>>> That's great that we have very closed visions! >>>>> >>>>> The general comment: currently, jtreg doesn't support any sort of >>>>> plugins, so you can't provide a VM specific handler of the >>>>> @requires or another tag. This is very annoying limitation and we >>>>> have to live with it. >>>>> >>>>> A few more comments inline. >>>>> >>>>> >>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>> >>>>>> >>>>>> Hi Dima, >>>>>> >>>>>> Answers inline. >>>>>> >>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>>>>> >>>>>>> See comments inline. >>>>>>> >>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of >>>>>>>>> the JDK-8019361 >>>>>>>>> >>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>> >>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>>>>> another GC. >>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>> skip test if there is a conflict >>>>>>>> >>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>> start sorting this out. >>>>>>>> >>>>>>>> First a general comment. The @requires tag has been developed >>>>>>>> without much cooperation with the GC team. We did have a lot of >>>>>>>> feedback when it was first presented a year ago, but it does >>>>>>>> not seem like this feedback was incorporated into the @requires >>>>>>>> that was eventually built. >>>>>>> >>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>> But not everything is possible, sorry for that. >>>>>> >>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>> been requesting this feature for 3 years and I was expecting us >>>>>> to be able to influence the feature much more than was the case now. >>>>> >>>>> My personal hope: @requires will address ~90% of existing issues. >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>> I think this change that gets proposed now is a big step >>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>> that we will soon run in to the limitations of the current >>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>> >>>>>>>> Some of the points I don't really like about the @requires tag are: >>>>>>>> >>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>> would have been better to just "require" any command line flag. >>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible >>>>>>> to use: >>>>>>> vm.opt.UseG1GC == true instead. >>>>>>> >>>>>>> The table with all vars available in jtreg: >>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>> >>>>>> The problem with having this matching built in to JTreg is that >>>>>> it makes it very hard to change. When we discussed this a year >>>>>> ago I think we said that JTreg should only provide a means to >>>>>> test against the command line and a hook for running some java >>>>>> code in the @requires tag. That way we could put logic like this >>>>>> in a test library that is under our control. This would make it >>>>>> easy for us to change and also enables us to use different logic >>>>>> for different versions. >>>>> >>>>> I would be glad to have own harness... >>>>> >>>>>> >>>>>>> >>>>>>>> - the requirement should be per @run tag. Right now we have to >>>>>>>> do what you did in this change and use vm.gc=null even when >>>>>>>> some tests could actually have been run when a GC was specified. >>>>>>> it would be great, but it will unlikely happen in jtreg, as well >>>>>>> as test case support. >>>>>> >>>>>> what do you mean with test case support? Hi Evgeniya, >>>>> >>>>> Under test case support I mean ability to treat each @run as a >>>>> separate test. Now >>>>> >>>>> @test >>>>> @run -XX:g1RegSize=1m MyTest >>>>> @run -XX:g1RegSize=2m MyTest >>>>> @run -XX:g1RegSize=4m MyTest >>>>> class MyTest { >>>>> } >>>>> >>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>> >>>>> >>>>>> >>>>>>> >>>>>>>> - there are many tests that require more than just a specific >>>>>>>> GC. Often there are other flags that can't be changed either >>>>>>>> for the test to work properly. >>>>>>> >>>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>>> conflicting options. >>>>>>> If we address this issue and we are satisfied with solution, we >>>>>>> could move further. >>>>>> >>>>>> Yes, I agree that taking one step at the time is good. Personally >>>>>> I would have preferred that the first step was a "just run the >>>>>> command line as specified in the @run tag" step. >>>>>> >>>>>>> >>>>>>>> >>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>> I'm not too happy about how the @requires tag turned out. But >>>>>>>> assuming we have to use it the way it is now I guess the >>>>>>>> proposed changeset looks good. >>>>>>> >>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>> jtreg :) >>>>>>> And thanks for reviewing it! >>>>>> >>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>> looked at all tests. But if they now pass with the combinations >>>>>> that we test with I guess they should be ok. >>>>> >>>>> Excellent! Thanks a lot! >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>> >>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>> >>>>>> Ok. Thanks. >>>>>> >>>>>>> >>>>>>>> >>>>>>>> I think some of the test, like >>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>> >>>>>>> These two tests ignore vm flags. >>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>> execute the tests when not needed. >>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>> tests 4 times, 1 should be enough. >>>>>> >>>>>> Do we really want to use the @requires functionality for this >>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>> want the tests to be run once I think Leonid's approach with >>>>>> tests lists seems more suitable. >>>>> >>>>> No, it's not a purpose of course, it's just side effect :) >>>>> >>>>> >>>>>> But are you sure that this is the reason for the @requires in >>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>> doesn't fail today it should probably be changed so that it does >>>>>> fail if it is run with the wrong GC. >>>>> >>>>> @requires - is not the silver bullet, but it's quite easy way to >>>>> solve a lot of issues. >>>>> >>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>> tests, which produce a new java process to ignore vm flags coming >>>>> from outside. No @requires, no other mechanism could 100% protect >>>>> a test from running with conflicting options, but this is not the >>>>> goal. >>>>> >>>>> If one runs tests with an exotic option, like a new G2 collector, >>>>> there shouldn't mass failures caused by options conflicts. But a >>>>> few failures could be handled manually. >>>>> >>>>> >>>>>> >>>>>>> >>>>>>>> Similarly it looks to me like there are tests that will fail if >>>>>>>> you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>>>>> >>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>> about to push a changeset that removes them: >>>>>>>> >>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>> okay, thank for letting us know. >>>>>>> >>>>>>>> >>>>>>>> Is there some way of making sure that all tests are run at one >>>>>>>> time or another. With this change there is a risk that some >>>>>>>> tests are never run and always skipped. Will we somehow be >>>>>>>> tracking what gets skipped and make sure that all tests are at >>>>>>>> least run once with the correct GC so that it is not skipped >>>>>>>> all the time? >>>>>>> >>>>>>> This is a very good question! >>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>> soon, after getting fix of: >>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>> >>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>> @requires - is not the only to exclude test from execution. >>>>>>> >>>>>>> Other examples: >>>>>>> >>>>>>> /* >>>>>>> *@ignore >>>>>>> *@test >>>>>>> */ >>>>>>> ... >>>>>>> >>>>>>> /*@bug 4445555 >>>>>>> *@test >>>>>>> */ >>>>>>> ... >>>>>>> Such tests will never be run, because jtreg treats as test only >>>>>>> files with @test on the first place... >>>>>>> >>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>> very actively). But this subject for another discussion, not >>>>>>> within RFR :) >>>>>> >>>>>> Right. Glad to hear that you are actively working on this! >>>>> >>>>> I was going to say "not very actively", but never mind, we know >>>>> about this problem. With introducing @requires mechanism it will >>>>> become more important! >>>>> >>>>> >>>>> Thanks for your comments! >>>>> >>>>> -- Dima >>>>> >>>>> >>>>>> >>>>>> Bengt >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.r.chase at oracle.com Wed Nov 12 17:03:11 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 12 Nov 2014 12:03:11 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <545F642E.30205@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545F642E.30205@gmail.com> Message-ID: Hello Peter, I was looking at this (thinking it would be a useful thing to benchmark, looking for possible improvements) and noticed that you rely on the hashed objects having a sensible value-dependent hashcode (as opposed to the default Object hashcode). Sadly, this seems not to be the case for MemberNames or for ?Types?. I am sorely tempted to repair this glitch, not sure if it fits in the scope of the original bug, but there?s a lot to be said for future-performance-proofing. David On 2014-11-09, at 7:55 AM, Peter Levart wrote: > Hi David, > > I played a little with the idea of having a hash table instead of packed sorted array for interning. Using ConcurrentHashMap would present quite some memory overhead. A more compact representation is possible in the form of a linear-scan hash table where elements of array are MemberNames themselves: > > http://cr.openjdk.java.net/~plevart/misc/MemberName.intern/jdk.06.diff/ > > This is a drop-in replacement for MemberName on top of your jdk.06 patch. If you have some time, you can run this with your performance tests to see if it presents any difference. If not, then perhaps this interning is not so performance critical after all. > > Regards, Peter -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From david.r.chase at oracle.com Wed Nov 12 18:27:33 2014 From: david.r.chase at oracle.com (David Chase) Date: Wed, 12 Nov 2014 13:27:33 -0500 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: <545F642E.30205@gmail.com> References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545F642E.30205@gmail.com> Message-ID: Hello Peter, > Sadly, this seems not to be the case for MemberNames or for ?Types?. That statement is inoperative. Mistakes were made. It?s compareTo that they lack. David On 2014-11-09, at 7:55 AM, Peter Levart wrote: > Hi David, > > I played a little with the idea of having a hash table instead of packed sorted array for interning. Using ConcurrentHashMap would present quite some memory overhead. A more compact representation is possible in the form of a linear-scan hash table where elements of array are MemberNames themselves: > > http://cr.openjdk.java.net/~plevart/misc/MemberName.intern/jdk.06.diff/ > > This is a drop-in replacement for MemberName on top of your jdk.06 patch. If you have some time, you can run this with your performance tests to see if it presents any difference. If not, then perhaps this interning is not so performance critical after all. > > Regards, Peter -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From paul.j.elschot at gmail.com Wed Nov 12 19:09:29 2014 From: paul.j.elschot at gmail.com (Paul Elschot) Date: Wed, 12 Nov 2014 20:09:29 +0100 Subject: Processor support for select operations In-Reply-To: References: <5461C9E8.1070809@gmail.com> <5462526D.1010302@gmail.com> <535F47A2-C478-4A16-AC57-FE3DF2FC7918@oracle.com> Message-ID: <5463B069.5030102@gmail.com> The difference in performance between Java and C for bit selection from a long is already quite small, so I'll wait for the optimization on power-of-two indexing and avoid Unsafe. Regards, Paul Elschot On 11-11-14 20:00, Vitaly Davidovich wrote: > But in the meantime, just trying Unsafe to see what the perf benefit > would be is still interesting, no? > > Sent from my phone > > On Nov 11, 2014 1:54 PM, "John Rose" > wrote: > > On Nov 11, 2014, at 10:16 AM, Paul Elschot > wrote: > >> A little masking might be enough to enforce the ranges, see the >> code in >> my other reply. > > Power-of-two array indexing is an important use case which can be > optimized without resorting to Unsafe. > We're working on it. > See https://bugs.openjdk.java.net/browse/JDK-8003585 > > ? John > From vladimir.kozlov at oracle.com Thu Nov 13 00:18:42 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Nov 2014 16:18:42 -0800 Subject: [9] RFR (XXS): 8062258: compiler/debug/TraceIterativeGVN.java segfaults in trace_PhaseIterGVN In-Reply-To: <546348AF.4040600@oracle.com> References: <5450D639.2050506@oracle.com> <54510129.8080108@oracle.com> <545274D0.3050702@oracle.com> <54529090.1030200@oracle.com> <5452A41D.1060107@oracle.com> <5452BB80.9030509@oracle.com> <546348AF.4040600@oracle.com> Message-ID: <5463F8E2.1040205@oracle.com> I am fine with this version. Reviewed. Thanks, Vladimir On 11/12/14 3:46 AM, Vladimir Ivanov wrote: > Vladimir, > > What do you think about the following version: > http://cr.openjdk.java.net/~vlivanov/8062258/webrev.02 > > I decided to revert changes in dumping logic. IMO possible duplication > of output is a lesser problem than a complication in dumping logic. > > Best regards, > Vladimir Ivanov > > On 10/31/14, 2:28 AM, Vladimir Kozlov wrote: >> Unfortunately not only MemNode has Type::MEMORY type. PhiNode, >> SCMemProjNode and others. It looks like you need to fix all places. >> Change in memnode.cpp is fine but in node.cpp you need: >> >> } else if (t == Type::MEMORY && (!is_Mem() || is_LoadStore())) { >> st->print(" Memory:"); >> MemNode::dump_adr_type(this, adr_type(), st); >> >> Make sure that other Type::MEMORY nodes don't crash in adr_type(). >> Don't do check under NOT_PRODUCT as in your first fix - just return NULL >> in all cases. >> >> Thanks, >> Vladimir >> >> On 10/30/14 1:48 PM, Vladimir Ivanov wrote: >>> Vladimir, nice observation! >>> >>> http://cr.openjdk.java.net/~vlivanov/8062258/webrev.01/ >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> On 10/30/14, 11:25 PM, Vladimir Kozlov wrote: >>>> Hmm, next code in PhaseIterGVN::optimize() worries me. We dump dead >>>> nodes before removing them from graph: >>>> >>>> DEBUG_ONLY(trace_PhaseIterGVN_verbose(n, num_processed++);) >>>> if (n->outcnt() != 0) { >>>> NOT_PRODUCT(const Type* oldtype = type_or_null(n)); >>>> // Do the transformation >>>> Node* nn = transform_old(n); >>>> NOT_PRODUCT(trace_PhaseIterGVN(n, nn, oldtype);) >>>> } else if (!n->is_top()) { >>>> remove_dead_node(n); >>>> } >>>> >>>> I am changing my suggestion: fix Node::dump() which calls adr_type() >>>> without check. It should not call MemNode::dump_adr_type() because >>>> MemNode::dump_spec(), which is called before, already does that and it >>>> has NULL check! If you want, you can add " Memory:" output in >>>> MemNode::dump_spec() to be similar to Node::dump() output. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 10/30/14 10:26 AM, Vladimir Ivanov wrote: >>>>> Vladimir, do you suggest to simply skip printing info about dead nodes >>>>> w/ -XX:+TraceIterativeGVN? >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>> On 10/29/14, 7:00 PM, Vladimir Kozlov wrote: >>>>>> I would suggest to dead node check in trace_PhaseIterGVN() instead of >>>>>> in MemNode::adr_type(). >>>>>> So the fix for your JDK-8061995 should fix this problem too. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 10/29/14 4:57 AM, Vladimir Ivanov wrote: >>>>>>> http://cr.openjdk.java.net/~vlivanov/8062258/webrev.00/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062258 >>>>>>> >>>>>>> Trivial fix to avoid NULL dereference w/ -XX:+TraceIterativeGVN >>>>>>> while >>>>>>> querying adr_type() on dead MemNode. >>>>>>> >>>>>>> Testing: failing test. >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov From vladimir.kozlov at oracle.com Thu Nov 13 00:39:44 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Nov 2014 16:39:44 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> <545BF25E.7010505@oracle.com> <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> <545D5103.9020309@oracle.com> Message-ID: <5463FDD0.4090102@oracle.com> You don't need leftover changes in opaquenode.* files anymore. jetstream regression may indicate the increase in compilation times. I would suggest to have n->Opcode() == Op_Opaque1 checks in both cases to target only it. Sorry, in my previous mail I incorrectly used Op_CastII in the example code. Thanks, Vladimir On 11/12/14 2:16 AM, Roland Westrelin wrote: > Vladimir, > > New webrev with your suggested change: > > http://cr.openjdk.java.net/~roland/8054478/webrev.02/ > > I also had to make BoolTest::dump_on available in product builds for that code: > > 138 } else { > 139 stringStream ss; > 140 test.dump_on(&ss); > 141 fatal(err_msg_res("unexpected comparison %s", ss.as_string())); > 142 } > > in castnode.cpp > > I did a refworkload run on x64: > > ============================================================================ > tbase: reference_server > Benchmark Samples Mean Stdev > jetstream 10 102.74 0.10 > Write 10 202.90 0.94 > Parse 10 37.10 0.05 > Read 10 23.20 0.13 > rw.runtime 10 10.60 0.05 > Copy 10 641.70 0.10 > scimark 10 1600.34 0.02 > LU 10 3776.61 0.00 > FFT 10 405.76 0.02 > Monte 10 775.88 0.00 > SOR 10 1275.16 0.01 > Sparse 10 1768.28 0.08 > rw.runtime 10 27.20 0.02 > specjbb2000 10 487411.56 0.04 > Last_Warehouse 10 487411.58 0.04 > First_Warehouse 10 95725.01 0.02 > rw.runtime 10 603.90 0.00 > specjbb2005 10 480939.78 0.01 > last 10 480939.79 0.01 > interval_average 10 5937.40 0.01 > peak 10 534158.07 0.02 > overall_average 10 431366.64 0.02 > last_warehouse 10 8.00 0.00 > peak_warehouse 10 2.30 0.21 > first 10 50951.62 0.03 > rw.runtime 10 461.60 0.00 > specjvm98 10 801.92 0.04 > compress 10 604.68 0.10 > javac 10 405.68 0.17 > db 10 449.78 0.00 > jack 10 631.14 0.05 > mtrt 10 2342.41 0.03 > jess 10 957.35 0.02 > rw.runtime 10 41.10 0.04 > mpegaudio 10 1385.76 0.00 > volano25 10 327418.69 0.06 > time 10 2.45 0.05 > connections 10 400.00 0.00 > rw.runtime 10 26.10 0.01 > -------------------------------------------------------------------------- > Weighted Geomean 31199.92 > ============================================================================ > tnew: reference_server > Benchmark Samples Mean Stdev %Diff P Significant > jetstream 10 85.82 0.12 -16.47 0.002 Yes > Write 10 219.60 0.31 -8.23 0.799 * > Parse 10 36.90 0.06 0.54 0.836 * > Read 10 23.60 0.10 -1.72 0.741 * > rw.runtime 10 13.80 0.03 -30.19 0.000 Yes > Copy 10 1058.70 0.26 -64.98 0.001 Yes > scimark 10 1583.22 0.00 -1.07 0.110 * > LU 10 3778.49 0.00 0.05 0.160 * > FFT 10 407.80 0.01 0.50 0.498 * > Monte 10 775.71 0.00 -0.02 0.279 * > SOR 10 1276.75 0.01 0.12 0.742 * > Sparse 10 1677.37 0.01 -5.14 0.086 * > rw.runtime 10 27.50 0.02 -1.10 0.265 * > specjbb2000 10 491672.03 0.04 0.87 0.624 * > Last_Warehouse 10 491672.04 0.04 0.87 0.624 * > First_Warehouse 10 94172.03 0.02 -1.62 0.050 * > rw.runtime 10 604.00 0.00 -0.02 0.585 * > specjbb2005 10 481051.28 0.01 0.02 0.945 * > last 10 481051.29 0.01 0.02 0.945 * > interval_average 10 5938.90 0.01 0.03 0.940 * > peak 10 538706.19 0.03 0.85 0.461 * > overall_average 10 434244.96 0.02 0.67 0.535 * > last_warehouse 10 8.00 0.00 -0.00 0.000 * > peak_warehouse 10 2.00 0.00 13.04 0.081 * > first 10 51039.34 0.03 0.17 0.889 * > rw.runtime 10 462.20 0.00 -0.13 0.120 * > specjvm98 10 806.31 0.04 0.55 0.738 * > compress 10 623.61 0.10 3.13 0.494 * > javac 10 402.37 0.10 -0.81 0.898 * > db 10 450.58 0.00 0.18 0.327 * > jack 10 627.02 0.05 -0.65 0.785 * > mtrt 10 2281.86 0.03 -2.58 0.106 * > jess 10 1000.65 0.10 4.52 0.220 * > rw.runtime 10 40.90 0.03 0.49 0.761 * > mpegaudio 10 1384.19 0.00 -0.11 0.514 * > volano25 10 324906.00 0.08 -0.77 0.799 * > time 10 2.47 0.07 -1.00 0.733 * > connections 10 400.00 0.00 0.00 0.000 * > rw.runtime 10 26.50 0.02 -1.53 0.058 * > -------------------------------------------------------------------------- > Weighted Geomean 30613.61 -1.88 > ============================================================================ > > tbase is current hotspot-comp. tnew is with the change. It?s ok, right? > > Roland. > > >> On Nov 8, 2014, at 12:08 AM, Vladimir Kozlov wrote: >> >> This looks good but I just realized that what you are doing in Opaque1Node::Ideal() we usually do in PhaseIterGVN::add_users_to_worklist(). >> >> For example you can do there (for Phi): >> >> + uint use_op = use->Opcode(); >> if( use->is_Cmp() ) { // Enable CMP/BOOL optimization >> add_users_to_worklist(use); // Put Bool on worklist >> - // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >> - // phi merging either 0 or 1 onto the worklist >> if (use->outcnt() > 0) { >> Node* bol = use->raw_out(0); >> if (bol->outcnt() > 0) { >> Node* iff = bol->raw_out(0); >> if (use_op == Op_CmpI && iff->is_CountedLoopEnd() && >> n->Opcode() == Op_CastII) { >> + // If this opaque node feeds into the limit condition of a >> + // CountedLoop, we need to process the Phi node for the >> + // induction variable: the range of values taken by the Phi is >> + // known now and so its type is also known. >> + CountedLoopEndNode* cle = iff->as_CountedLoopEnd(); >> + if (cle->limit() == this) { >> + _worklist.push(cle->phi()); >> + } >> - if (iff->outcnt() == 2) { >> + } else if (iff->outcnt() == 2) { >> + // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >> + // phi merging either 0 or 1 onto the worklist >> >> >> Thanks, >> Vladimir >> >> >> On 11/7/14 2:17 PM, Roland Westrelin wrote: >>> Vladimir, >>> >>> Thanks for the discussion, suggestions and fixes. Here is an updated webrev: >>> >>> http://cr.openjdk.java.net/~roland/8054478/webrev.01/ >>> >>> Roland. >>> >>>> On Nov 6, 2014, at 11:12 PM, Vladimir Kozlov wrote: >>>> >>>> On 11/6/14 7:39 AM, Roland Westrelin wrote: >>>>> >>>>>> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. >>>>> >>>>> I?ll send an updated webrev. >>>>> >>>>>>>>>> opaquenode.cpp >>>>>>>>>> >>>>>>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>>>>>> >>>>>>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>>>>>> >>>>>>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>>>>>> >>>>>>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >>>>>> >>>>>> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. >>>>> >>>>> (I know you don?t like when I remove the text from previous emails but that was really getting confusing) >>>> >>>> At least leave webrev link so I don't need to search for it in previous mails. >>>> >>>>> >>>>> PhiNode::Value() has this code: >>>>> >>>>> // Check for trip-counted loop. If so, be smarter. >>>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>>> if( l && l->can_be_counted_loop(phase) && >>>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>>> // protect against init_trip() or limit() returning NULL >>>>> const Node *init = l->init_trip(); >>>>> const Node *limit = l->limit(); >>>>> if( init != NULL && limit != NULL && l->stride_is_con() ) { >>>>> const TypeInt *lo = init ->bottom_type()->isa_int(); >>>>> const TypeInt *hi = limit->bottom_type()->isa_int(); >>>>> if( lo && hi ) { // Dying loops might have TOP here >>>>> int stride = l->stride_con(); >>>>> if( stride < 0 ) { // Down-counter loop >>>>> const TypeInt *tmp = lo; lo = hi; hi = tmp; >>>>> stride = -stride; >>>>> } >>>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>>> } >>>>> } >>>>> } >>>>> >>>>> That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. >>>> >>>> My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: >>>> >>>> // Step B2: Build a zero-trip guard for the main-loop. After leaving the >>>> // pre-loop, the main-loop may not execute at all. Later in life this >>>> // zero-trip guard will become the minimum-trip guard when we unroll >>>> // the main-loop. >>>> Node *min_opaq = new Opaque1Node(C, limit); >>>> Node *min_cmp = new CmpINode( pre_incr, min_opaq ); >>>> >>>> >>>> But you are talking about pre-loop exit check: >>>> >>>> // Step B4: Shorten the pre-loop to run only 1 iteration (for now). >>>> // RCE and alignment may change this later. >>>> Node *cmp_end = pre_end->cmp_node(); >>>> assert( cmp_end->in(2) == limit, "" ); >>>> Node *pre_limit = new AddINode( init, stride ); >>>> >>>> // Save the original loop limit in this Opaque1 node for >>>> // use by range check elimination. >>>> Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); >>>> >>>> >>>> But in this case you can add check (cl->limit() == this) to be clear what you are looking for. >>>> Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>>> >>>>> In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. >>>>> >>>>> Roland. >>> > From vladimir.kozlov at oracle.com Thu Nov 13 02:09:32 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 12 Nov 2014 18:09:32 -0800 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <5462616B.3030003@oracle.com> References: <545A960C.7010008@oracle.com> <545AAB7D.3010100@oracle.com> <5462616B.3030003@oracle.com> Message-ID: <546412DC.2020401@oracle.com> I am start worrying about this change. Nested ResourceMark may cause problems. PhaseLive::getfreeset() uses thread local resource area for _free_IndexSet and PhaseLive::compute() resets it on exit: // Init the sparse arrays for delta-sets. ResourceMark rm; // Nuke temp storage on exit But the rest of IndexSet objects uses indexSet_arena which is 'live_arena' in Register_Allocate(). It lives longer then PhaseLive::compute() and PhaseLive::compute() may 'nuke' its data incorrectly if you use thread local resource area for everything. May be you should also separate arenas for BitMap used as IndexSet. I think you need to instrument JVM to get correct data. If you use separate ResourceArea (as in original code 'live_arena') you can print its size at the end of its use scope. I would recommend to run with -Xbatch -XX:-TieredCompilation -XX:CICompilerCount=1 to get serialized data which can be compared. Clean up: Remove commented assert in lrg_union() since it is not applicable anymore. Fix style in the code you are changing. No spaces in if(): if( vec->is_empty() ) return; should be if (vec->is_empty()) { return; } Thanks, Vladimir On 11/11/14 11:20 AM, Aleksey Shipilev wrote: > Thanks for review, Vladimir! > > FTR, here's an updated webrev: > http://cr.openjdk.java.net/~shade/8059461/webrev.03/ > > On 11/06/2014 01:58 AM, Vladimir Kozlov wrote: >> Aleksey, can you compare average memory consumed by IndexSet before and >> after? > > Any ideas how to estimate this reliably? Current code has some > instrumentation, but only about usage, not the concrete footprint. NMT > shows very flaky numbers for Nashorn/Octane: > > $ grep "Compiler" before/* > Compiler (reserved=165KB, committed=165KB) > Compiler (reserved=174KB, committed=174KB) > Compiler (reserved=201KB, committed=201KB) > Compiler (reserved=189KB, committed=189KB) > Compiler (reserved=192KB, committed=192KB) > > $ grep "Compiler" after/* > Compiler (reserved=162KB, committed=162KB) > Compiler (reserved=167KB, committed=167KB) > Compiler (reserved=198KB, committed=198KB) > Compiler (reserved=240KB, committed=240KB) > Compiler (reserved=188KB, committed=188KB) > > $ grep "Arena" before/* > Arena Chunk (reserved=10952KB, committed=10952KB) > Arena Chunk (reserved=11433KB, committed=11433KB) > Arena Chunk (reserved=10506KB, committed=10506KB) > Arena Chunk (reserved=10825KB, committed=10825KB) > Arena Chunk (reserved=7179KB, committed=7179KB) > > $ grep "Arena" after/* > Arena Chunk (reserved=19971KB, committed=19971KB) > Arena Chunk (reserved=20738KB, committed=20738KB) > Arena Chunk (reserved=21090KB, committed=21090KB) > Arena Chunk (reserved=22304KB, committed=22304KB) > Arena Chunk (reserved=14342KB, committed=14342KB) > > It *looks* like a memory leak in Arenas, but it might be as well the > larger peak usage. Or, it might be NMT not telling me the whole story. > > I am running Footprint tests from our performance regression suites now > -- they will show if there is something real. > >> Why you need initialize_in_resource_arena()? by default BitMap() uses >> resource area: >> >> BitMap(idx_t size_in_bits, bool in_resource_area = true); > > Yes, pruned that. > >> Make lrg_union() PhaseConservativeCoalesce class's method. > > Yes, did that. > > Thanks, > -Aleksey. > > From peter.levart at gmail.com Thu Nov 13 08:24:53 2014 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 13 Nov 2014 09:24:53 +0100 Subject: [9] RFR(L) 8013267 : move MemberNameTable from native code to Java heap, use to intern MemberNames In-Reply-To: References: <594FE416-D445-426E-B7A9-C75F012ADE16@oracle.com> <0BDCD5DD-CBFB-4A3D-9BAD-7F9912E9237C@oracle.com> <5453C230.8010709@oracle.com> <9747A3A3-B7F4-407A-8E00-1E647D9DC2D1@oracle.com> <1D195ED3-8BD2-46CB-9D70-29CF809D9F5A@oracle.com> <5456FB59.60905@oracle.com> <632A5C98-B386-4625-BE12-355241581955@oracle.com> <5457AA75.8090103@gmail.com> <5457E0F9.8090004@gmail.com> <5458A57C.4060208@gmail.com> <260D49F5-6380-4FC3-A900-6CD9AB3ED6F7@oracle.com> <5459034E.8070809@gmail.com> <39826508-110B-4FCE-9A58-8C3D1B9FC7DE@oracle.com> <545F642E.30205@gmail.com> Message-ID: <54646AD5.4000404@gmail.com> On 11/12/2014 07:27 PM, David Chase wrote: > Hello Peter, > >> Sadly, this seems not to be the case for MemberNames or for ?Types?. > That statement is inoperative. Mistakes were made. > It?s compareTo that they lack. Yes, I say your quite tricky implementation of MemberName.compareTo, based on hashCode(s), String representations, etc... The hash-table based interning does not need it though. Regards, Peter > David > > > On 2014-11-09, at 7:55 AM, Peter Levart wrote: > >> Hi David, >> >> I played a little with the idea of having a hash table instead of packed sorted array for interning. Using ConcurrentHashMap would present quite some memory overhead. A more compact representation is possible in the form of a linear-scan hash table where elements of array are MemberNames themselves: >> >> http://cr.openjdk.java.net/~plevart/misc/MemberName.intern/jdk.06.diff/ >> >> This is a drop-in replacement for MemberName on top of your jdk.06 patch. If you have some time, you can run this with your performance tests to see if it presents any difference. If not, then perhaps this interning is not so performance critical after all. >> >> Regards, Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From roland.westrelin at oracle.com Thu Nov 13 09:09:38 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Thu, 13 Nov 2014 10:09:38 +0100 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <5463FDD0.4090102@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> <545BF25E.7010505@oracle.com> <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> <545D5103.9020309@oracle.com> <5463FDD0.4090102@oracle.com> Message-ID: > You don't need leftover changes in opaquenode.* files anymore. I?ll remove the changes. > jetstream regression may indicate the increase in compilation times. I would suggest to have n->Opcode() == Op_Opaque1 checks in both cases to target only it. Sorry, in my previous mail I incorrectly used Op_CastII in the example code. Are we guaranteed that the type of the limit won?t improve after the opaque node is removed? Maybe it?s some arithmetic computation that isn?t optimized yet? In that case, we want to re-process the phi even after the opaque node is removed as soon as the limit changes. Can I go with a single reviewer for this change? Roland. > > Thanks, > Vladimir > > On 11/12/14 2:16 AM, Roland Westrelin wrote: >> Vladimir, >> >> New webrev with your suggested change: >> >> http://cr.openjdk.java.net/~roland/8054478/webrev.02/ >> >> I also had to make BoolTest::dump_on available in product builds for that code: >> >> 138 } else { >> 139 stringStream ss; >> 140 test.dump_on(&ss); >> 141 fatal(err_msg_res("unexpected comparison %s", ss.as_string())); >> 142 } >> >> in castnode.cpp >> >> I did a refworkload run on x64: >> >> ============================================================================ >> tbase: reference_server >> Benchmark Samples Mean Stdev >> jetstream 10 102.74 0.10 >> Write 10 202.90 0.94 >> Parse 10 37.10 0.05 >> Read 10 23.20 0.13 >> rw.runtime 10 10.60 0.05 >> Copy 10 641.70 0.10 >> scimark 10 1600.34 0.02 >> LU 10 3776.61 0.00 >> FFT 10 405.76 0.02 >> Monte 10 775.88 0.00 >> SOR 10 1275.16 0.01 >> Sparse 10 1768.28 0.08 >> rw.runtime 10 27.20 0.02 >> specjbb2000 10 487411.56 0.04 >> Last_Warehouse 10 487411.58 0.04 >> First_Warehouse 10 95725.01 0.02 >> rw.runtime 10 603.90 0.00 >> specjbb2005 10 480939.78 0.01 >> last 10 480939.79 0.01 >> interval_average 10 5937.40 0.01 >> peak 10 534158.07 0.02 >> overall_average 10 431366.64 0.02 >> last_warehouse 10 8.00 0.00 >> peak_warehouse 10 2.30 0.21 >> first 10 50951.62 0.03 >> rw.runtime 10 461.60 0.00 >> specjvm98 10 801.92 0.04 >> compress 10 604.68 0.10 >> javac 10 405.68 0.17 >> db 10 449.78 0.00 >> jack 10 631.14 0.05 >> mtrt 10 2342.41 0.03 >> jess 10 957.35 0.02 >> rw.runtime 10 41.10 0.04 >> mpegaudio 10 1385.76 0.00 >> volano25 10 327418.69 0.06 >> time 10 2.45 0.05 >> connections 10 400.00 0.00 >> rw.runtime 10 26.10 0.01 >> -------------------------------------------------------------------------- >> Weighted Geomean 31199.92 >> ============================================================================ >> tnew: reference_server >> Benchmark Samples Mean Stdev %Diff P Significant >> jetstream 10 85.82 0.12 -16.47 0.002 Yes >> Write 10 219.60 0.31 -8.23 0.799 * >> Parse 10 36.90 0.06 0.54 0.836 * >> Read 10 23.60 0.10 -1.72 0.741 * >> rw.runtime 10 13.80 0.03 -30.19 0.000 Yes >> Copy 10 1058.70 0.26 -64.98 0.001 Yes >> scimark 10 1583.22 0.00 -1.07 0.110 * >> LU 10 3778.49 0.00 0.05 0.160 * >> FFT 10 407.80 0.01 0.50 0.498 * >> Monte 10 775.71 0.00 -0.02 0.279 * >> SOR 10 1276.75 0.01 0.12 0.742 * >> Sparse 10 1677.37 0.01 -5.14 0.086 * >> rw.runtime 10 27.50 0.02 -1.10 0.265 * >> specjbb2000 10 491672.03 0.04 0.87 0.624 * >> Last_Warehouse 10 491672.04 0.04 0.87 0.624 * >> First_Warehouse 10 94172.03 0.02 -1.62 0.050 * >> rw.runtime 10 604.00 0.00 -0.02 0.585 * >> specjbb2005 10 481051.28 0.01 0.02 0.945 * >> last 10 481051.29 0.01 0.02 0.945 * >> interval_average 10 5938.90 0.01 0.03 0.940 * >> peak 10 538706.19 0.03 0.85 0.461 * >> overall_average 10 434244.96 0.02 0.67 0.535 * >> last_warehouse 10 8.00 0.00 -0.00 0.000 * >> peak_warehouse 10 2.00 0.00 13.04 0.081 * >> first 10 51039.34 0.03 0.17 0.889 * >> rw.runtime 10 462.20 0.00 -0.13 0.120 * >> specjvm98 10 806.31 0.04 0.55 0.738 * >> compress 10 623.61 0.10 3.13 0.494 * >> javac 10 402.37 0.10 -0.81 0.898 * >> db 10 450.58 0.00 0.18 0.327 * >> jack 10 627.02 0.05 -0.65 0.785 * >> mtrt 10 2281.86 0.03 -2.58 0.106 * >> jess 10 1000.65 0.10 4.52 0.220 * >> rw.runtime 10 40.90 0.03 0.49 0.761 * >> mpegaudio 10 1384.19 0.00 -0.11 0.514 * >> volano25 10 324906.00 0.08 -0.77 0.799 * >> time 10 2.47 0.07 -1.00 0.733 * >> connections 10 400.00 0.00 0.00 0.000 * >> rw.runtime 10 26.50 0.02 -1.53 0.058 * >> -------------------------------------------------------------------------- >> Weighted Geomean 30613.61 -1.88 >> ============================================================================ >> >> tbase is current hotspot-comp. tnew is with the change. It?s ok, right? >> >> Roland. >> >> >>> On Nov 8, 2014, at 12:08 AM, Vladimir Kozlov wrote: >>> >>> This looks good but I just realized that what you are doing in Opaque1Node::Ideal() we usually do in PhaseIterGVN::add_users_to_worklist(). >>> >>> For example you can do there (for Phi): >>> >>> + uint use_op = use->Opcode(); >>> if( use->is_Cmp() ) { // Enable CMP/BOOL optimization >>> add_users_to_worklist(use); // Put Bool on worklist >>> - // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >>> - // phi merging either 0 or 1 onto the worklist >>> if (use->outcnt() > 0) { >>> Node* bol = use->raw_out(0); >>> if (bol->outcnt() > 0) { >>> Node* iff = bol->raw_out(0); >>> if (use_op == Op_CmpI && iff->is_CountedLoopEnd() && >>> n->Opcode() == Op_CastII) { >>> + // If this opaque node feeds into the limit condition of a >>> + // CountedLoop, we need to process the Phi node for the >>> + // induction variable: the range of values taken by the Phi is >>> + // known now and so its type is also known. >>> + CountedLoopEndNode* cle = iff->as_CountedLoopEnd(); >>> + if (cle->limit() == this) { >>> + _worklist.push(cle->phi()); >>> + } >>> - if (iff->outcnt() == 2) { >>> + } else if (iff->outcnt() == 2) { >>> + // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >>> + // phi merging either 0 or 1 onto the worklist >>> >>> >>> Thanks, >>> Vladimir >>> >>> >>> On 11/7/14 2:17 PM, Roland Westrelin wrote: >>>> Vladimir, >>>> >>>> Thanks for the discussion, suggestions and fixes. Here is an updated webrev: >>>> >>>> http://cr.openjdk.java.net/~roland/8054478/webrev.01/ >>>> >>>> Roland. >>>> >>>>> On Nov 6, 2014, at 11:12 PM, Vladimir Kozlov wrote: >>>>> >>>>> On 11/6/14 7:39 AM, Roland Westrelin wrote: >>>>>> >>>>>>> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. >>>>>> >>>>>> I?ll send an updated webrev. >>>>>> >>>>>>>>>>> opaquenode.cpp >>>>>>>>>>> >>>>>>>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>>>>>>> >>>>>>>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>>>>>>> >>>>>>>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>>>>>>> >>>>>>>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >>>>>>> >>>>>>> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. >>>>>> >>>>>> (I know you don?t like when I remove the text from previous emails but that was really getting confusing) >>>>> >>>>> At least leave webrev link so I don't need to search for it in previous mails. >>>>> >>>>>> >>>>>> PhiNode::Value() has this code: >>>>>> >>>>>> // Check for trip-counted loop. If so, be smarter. >>>>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>>>> if( l && l->can_be_counted_loop(phase) && >>>>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>>>> // protect against init_trip() or limit() returning NULL >>>>>> const Node *init = l->init_trip(); >>>>>> const Node *limit = l->limit(); >>>>>> if( init != NULL && limit != NULL && l->stride_is_con() ) { >>>>>> const TypeInt *lo = init ->bottom_type()->isa_int(); >>>>>> const TypeInt *hi = limit->bottom_type()->isa_int(); >>>>>> if( lo && hi ) { // Dying loops might have TOP here >>>>>> int stride = l->stride_con(); >>>>>> if( stride < 0 ) { // Down-counter loop >>>>>> const TypeInt *tmp = lo; lo = hi; hi = tmp; >>>>>> stride = -stride; >>>>>> } >>>>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>>>> } >>>>>> } >>>>>> } >>>>>> >>>>>> That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. >>>>> >>>>> My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: >>>>> >>>>> // Step B2: Build a zero-trip guard for the main-loop. After leaving the >>>>> // pre-loop, the main-loop may not execute at all. Later in life this >>>>> // zero-trip guard will become the minimum-trip guard when we unroll >>>>> // the main-loop. >>>>> Node *min_opaq = new Opaque1Node(C, limit); >>>>> Node *min_cmp = new CmpINode( pre_incr, min_opaq ); >>>>> >>>>> >>>>> But you are talking about pre-loop exit check: >>>>> >>>>> // Step B4: Shorten the pre-loop to run only 1 iteration (for now). >>>>> // RCE and alignment may change this later. >>>>> Node *cmp_end = pre_end->cmp_node(); >>>>> assert( cmp_end->in(2) == limit, "" ); >>>>> Node *pre_limit = new AddINode( init, stride ); >>>>> >>>>> // Save the original loop limit in this Opaque1 node for >>>>> // use by range check elimination. >>>>> Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); >>>>> >>>>> >>>>> But in this case you can add check (cl->limit() == this) to be clear what you are looking for. >>>>> Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>>> >>>>>> In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. >>>>>> >>>>>> Roland. >>>> >> From goetz.lindenmaier at sap.com Thu Nov 13 09:36:52 2014 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 13 Nov 2014 09:36:52 +0000 Subject: gc bugs after 8060252: JDK-7173584 compiler changes regress SPECjvm2008 on SPARC In-Reply-To: <6A8F58D2-6B64-4FB1-9CD2-AD4437A13F11@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF26315@DEWDFEMB12A.global.corp.sap> <6A8F58D2-6B64-4FB1-9CD2-AD4437A13F11@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF270EE@DEWDFEMB12A.global.corp.sap> Hi Roland, thanks for the description in the bug! I guess that's just the reason why tightly_coupled_allocation() was only called after emitting all the tests: it analyzes whether these tests could be optimized away. If they go to some uncommon trap, the initialization is needed. Best regards, Goetz. -----Original Message----- From: Roland Westrelin [mailto:roland.westrelin at oracle.com] Sent: Mittwoch, 12. November 2014 14:16 To: Lindenmaier, Goetz Cc: hotspot-compiler-dev at openjdk.java.net Subject: Re: gc bugs after 8060252: JDK-7173584 compiler changes regress SPECjvm2008 on SPARC I filed the following bug: https://bugs.openjdk.java.net/browse/JDK-8064703 Roland. From zoltan.majo at oracle.com Thu Nov 13 13:32:43 2014 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Thu, 13 Nov 2014 14:32:43 +0100 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups Message-ID: <5464B2FB.1050606@oracle.com> Hi, please review the following patch. Bug: https://bugs.openjdk.java.net/browse/JDK-8062854 Problem: Currently, we list all short-running JTREG tests that we execute in JPRT in the hotspot/test/TEST.groups file (using the test groups hotspot_compiler_[1-3] and hotspot_compiler_closed). The problem with this approach is that if we want to execute a new (short-running) JTREG test in JPRT, the TEST.groups file must be updated to execute the newly added JTREG test. Moreover, the current selection of tests is based on a per-group time limit of 10 minutes (i.e., each test group executes 10 minutes on the slowest JPRT platform available), but it seems that a 15-minute limit is also tolerable. Solution: Move all test directories of the form hotspot/test/ to "functionality-based" directories. Each functionality-based directory contains tests related to a specific functionality of the VM, for example: hotspot/test/compiler/c1, hotspot/test/compiler/c2, and hotspot/test/compiler/codecache. In the TEST.groups file we list only functionality-based directories, and not individual tests. We also exclude long-running tests. Webrev: http://cr.openjdk.java.net/~zmajo/8062854/webrev.00/ The webrev is large because a large number of directories was moved. The test/TEST.groups file summarizes changes well. Testing: JPRT Before changes (group / #tests / execution time on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848): hotspot_compiler_1 / 73 / 9m16s hotspot_compiler_2 / 60 / 9m50s hotspot_compiler_3 / 69 / 13m32s After changes (group / #tests / execution time on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848): hotspot_compiler_1 / 107 / 16m55s hotspot_compiler_2 / 76 / 14m49s hotspot_compiler_3 / 59 / 13m44s All tests (also excluded ones) were executed on all platforms to check if any configuration error has appeared due to reorganizing the directory structure. All tests pass. Thank you and best regards, Zoltan From bengt.rutisson at oracle.com Thu Nov 13 13:32:22 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 14:32:22 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <54638A9F.2030209@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> Message-ID: <5464B2E6.9070802@oracle.com> Hi Evgeniya, On 2014-11-12 17:28, Evgeniya Stepanova wrote: > Hi Dmitry, > > You are right - I've forgotten about copyrights > Copyrights and other issues you mentioned fixed. New webrev: > http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be good to add -XX:+UseG1GC to the @run tags and then use @requires vm.gc=="G1" | vm.gc == null. The change to test/gc/defnew/HeapChangeLogging.java is unrelated to the conflicting GC combinations. Should that really be part of this changeset? The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we really need @requires for them? Otherwise it look ok to me. Bengt > > Thanks > Evgeniya Stepanova > On 12.11.2014 18:23, Dmitry Fazunenko wrote: >> Hi Evgeniya, >> >> The fix looks good to me. >> >> I noticed the following minor things: >> - copyrights need to include the year of last change >> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >> files, but doesn't contain any changes >> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >> 'prohibitedVmOptions' >> >> Thanks, >> Dima >> >> >> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>> Hi everyone! >>> >>> Since the decision was made to change only tests that fail because >>> of conflict for now (skip "selfish" tests), I post new webrev for >>> hotspot part of the JDK-8019361 >>> : >>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>> >>> Thanks, >>> Evgeniya Stepanova >>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>> Nice plan! Please feel free to send me any feedback/questions >>>> regarding @requires >>>> >>>> Thanks, >>>> Dima >>>> >>>> >>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>> >>>>> Hi Dima, >>>>> >>>>> Thanks for the answers. I think the currently proposed patch is a >>>>> good start. We will have to evolve the @requires tag in the >>>>> future, but let's have that discussion separate from this review. >>>>> And we can start that discussion later when we have more >>>>> experience with the current version of @requires. >>>>> >>>>> Thanks for doing this! >>>>> Bengt >>>>> >>>>> >>>>> >>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>> Hi Bengt, >>>>>> >>>>>> That's great that we have very closed visions! >>>>>> >>>>>> The general comment: currently, jtreg doesn't support any sort of >>>>>> plugins, so you can't provide a VM specific handler of the >>>>>> @requires or another tag. This is very annoying limitation and we >>>>>> have to live with it. >>>>>> >>>>>> A few more comments inline. >>>>>> >>>>>> >>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>> >>>>>>> >>>>>>> Hi Dima, >>>>>>> >>>>>>> Answers inline. >>>>>>> >>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>> Hi Bengt, >>>>>>>> >>>>>>>> Thanks a lot for your detailed feedback, we appreciate it very >>>>>>>> much! >>>>>>>> >>>>>>>> See comments inline. >>>>>>>> >>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi Evgeniya, >>>>>>>>> >>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part >>>>>>>>>> of the JDK-8019361 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>> >>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set >>>>>>>>>> another GC. >>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>> skip test if there is a conflict >>>>>>>>> >>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>> start sorting this out. >>>>>>>>> >>>>>>>>> First a general comment. The @requires tag has been developed >>>>>>>>> without much cooperation with the GC team. We did have a lot >>>>>>>>> of feedback when it was first presented a year ago, but it >>>>>>>>> does not seem like this feedback was incorporated into the >>>>>>>>> @requires that was eventually built. >>>>>>>> >>>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>>> But not everything is possible, sorry for that. >>>>>>> >>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>> been requesting this feature for 3 years and I was expecting us >>>>>>> to be able to influence the feature much more than was the case now. >>>>>> >>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>> >>>>>>>>> Some of the points I don't really like about the @requires tag >>>>>>>>> are: >>>>>>>>> >>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible >>>>>>>> to use: >>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>> >>>>>>>> The table with all vars available in jtreg: >>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>> >>>>>>> The problem with having this matching built in to JTreg is that >>>>>>> it makes it very hard to change. When we discussed this a year >>>>>>> ago I think we said that JTreg should only provide a means to >>>>>>> test against the command line and a hook for running some java >>>>>>> code in the @requires tag. That way we could put logic like this >>>>>>> in a test library that is under our control. This would make it >>>>>>> easy for us to change and also enables us to use different logic >>>>>>> for different versions. >>>>>> >>>>>> I would be glad to have own harness... >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> - the requirement should be per @run tag. Right now we have to >>>>>>>>> do what you did in this change and use vm.gc=null even when >>>>>>>>> some tests could actually have been run when a GC was specified. >>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>> well as test case support. >>>>>>> >>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>> >>>>>> Under test case support I mean ability to treat each @run as a >>>>>> separate test. Now >>>>>> >>>>>> @test >>>>>> @run -XX:g1RegSize=1m MyTest >>>>>> @run -XX:g1RegSize=2m MyTest >>>>>> @run -XX:g1RegSize=4m MyTest >>>>>> class MyTest { >>>>>> } >>>>>> >>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> - there are many tests that require more than just a specific >>>>>>>>> GC. Often there are other flags that can't be changed either >>>>>>>>> for the test to work properly. >>>>>>>> >>>>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>>>> conflicting options. >>>>>>>> If we address this issue and we are satisfied with solution, we >>>>>>>> could move further. >>>>>>> >>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>> Personally I would have preferred that the first step was a >>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>>> I'm not too happy about how the @requires tag turned out. But >>>>>>>>> assuming we have to use it the way it is now I guess the >>>>>>>>> proposed changeset looks good. >>>>>>>> >>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>> jtreg :) >>>>>>>> And thanks for reviewing it! >>>>>>> >>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>> looked at all tests. But if they now pass with the combinations >>>>>>> that we test with I guess they should be ok. >>>>>> >>>>>> Excellent! Thanks a lot! >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>> >>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>> >>>>>>> Ok. Thanks. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> I think some of the test, like >>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>>> >>>>>>>> These two tests ignore vm flags. >>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>> execute the tests when not needed. >>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>> tests 4 times, 1 should be enough. >>>>>>> >>>>>>> Do we really want to use the @requires functionality for this >>>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>>> want the tests to be run once I think Leonid's approach with >>>>>>> tests lists seems more suitable. >>>>>> >>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>> >>>>>> >>>>>>> But are you sure that this is the reason for the @requires in >>>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>>> doesn't fail today it should probably be changed so that it does >>>>>>> fail if it is run with the wrong GC. >>>>>> >>>>>> @requires - is not the silver bullet, but it's quite easy way to >>>>>> solve a lot of issues. >>>>>> >>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>> tests, which produce a new java process to ignore vm flags coming >>>>>> from outside. No @requires, no other mechanism could 100% protect >>>>>> a test from running with conflicting options, but this is not the >>>>>> goal. >>>>>> >>>>>> If one runs tests with an exotic option, like a new G2 collector, >>>>>> there shouldn't mass failures caused by options conflicts. But a >>>>>> few failures could be handled manually. >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> Similarly it looks to me like there are tests that will fail >>>>>>>>> if you run them with -XX:-UseParallelOldGC or >>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>> >>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>> about to push a changeset that removes them: >>>>>>>>> >>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>> okay, thank for letting us know. >>>>>>>> >>>>>>>>> >>>>>>>>> Is there some way of making sure that all tests are run at one >>>>>>>>> time or another. With this change there is a risk that some >>>>>>>>> tests are never run and always skipped. Will we somehow be >>>>>>>>> tracking what gets skipped and make sure that all tests are at >>>>>>>>> least run once with the correct GC so that it is not skipped >>>>>>>>> all the time? >>>>>>>> >>>>>>>> This is a very good question! >>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>> soon, after getting fix of: >>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>> >>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>> >>>>>>>> Other examples: >>>>>>>> >>>>>>>> /* >>>>>>>> *@ignore >>>>>>>> *@test >>>>>>>> */ >>>>>>>> ... >>>>>>>> >>>>>>>> /*@bug 4445555 >>>>>>>> *@test >>>>>>>> */ >>>>>>>> ... >>>>>>>> Such tests will never be run, because jtreg treats as test only >>>>>>>> files with @test on the first place... >>>>>>>> >>>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>>> very actively). But this subject for another discussion, not >>>>>>>> within RFR :) >>>>>>> >>>>>>> Right. Glad to hear that you are actively working on this! >>>>>> >>>>>> I was going to say "not very actively", but never mind, we know >>>>>> about this problem. With introducing @requires mechanism it will >>>>>> become more important! >>>>>> >>>>>> >>>>>> Thanks for your comments! >>>>>> >>>>>> -- Dima >>>>>> >>>>>> >>>>>>> >>>>>>> Bengt >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dima >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> -- >>> /Evgeniya Stepanova/ >> > > -- > /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Thu Nov 13 12:49:49 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 13 Nov 2014 16:49:49 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B2E6.9070802@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> Message-ID: <5464A8ED.4070708@oracle.com> On 13.11.2014 17:32, Bengt Rutisson wrote: > > Hi Evgeniya, > > On 2014-11-12 17:28, Evgeniya Stepanova wrote: >> Hi Dmitry, >> >> You are right - I've forgotten about copyrights >> Copyrights and other issues you mentioned fixed. New webrev: >> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ > > > For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be > good to add -XX:+UseG1GC to the @run tags and then use @requires > vm.gc=="G1" | vm.gc == null. > > > The change to test/gc/defnew/HeapChangeLogging.java is unrelated to > the conflicting GC combinations. Should that really be part of this > changeset? > > > The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we > really need @requires for them? Yes, we do. These tests use TestShrinkAuxiliaryData class which implements its own mechanism to analyze VM options an skip if not applicable collector is given. @requires - allows to rely on jtreg. Driver mode is a kind of indicator, that the test will spawn its own java process. Thanks Dima > > > Otherwise it look ok to me. > > Bengt > > >> >> Thanks >> Evgeniya Stepanova >> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>> Hi Evgeniya, >>> >>> The fix looks good to me. >>> >>> I noticed the following minor things: >>> - copyrights need to include the year of last change >>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>> files, but doesn't contain any changes >>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >>> 'prohibitedVmOptions' >>> >>> Thanks, >>> Dima >>> >>> >>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>> Hi everyone! >>>> >>>> Since the decision was made to change only tests that fail because >>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>> hotspot part of the JDK-8019361 >>>> : >>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>> Nice plan! Please feel free to send me any feedback/questions >>>>> regarding @requires >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Dima, >>>>>> >>>>>> Thanks for the answers. I think the currently proposed patch is a >>>>>> good start. We will have to evolve the @requires tag in the >>>>>> future, but let's have that discussion separate from this review. >>>>>> And we can start that discussion later when we have more >>>>>> experience with the current version of @requires. >>>>>> >>>>>> Thanks for doing this! >>>>>> Bengt >>>>>> >>>>>> >>>>>> >>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> That's great that we have very closed visions! >>>>>>> >>>>>>> The general comment: currently, jtreg doesn't support any sort >>>>>>> of plugins, so you can't provide a VM specific handler of the >>>>>>> @requires or another tag. This is very annoying limitation and >>>>>>> we have to live with it. >>>>>>> >>>>>>> A few more comments inline. >>>>>>> >>>>>>> >>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi Dima, >>>>>>>> >>>>>>>> Answers inline. >>>>>>>> >>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>> Hi Bengt, >>>>>>>>> >>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it very >>>>>>>>> much! >>>>>>>>> >>>>>>>>> See comments inline. >>>>>>>>> >>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part >>>>>>>>>>> of the JDK-8019361 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>> set another GC. >>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>> skip test if there is a conflict >>>>>>>>>> >>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>> start sorting this out. >>>>>>>>>> >>>>>>>>>> First a general comment. The @requires tag has been developed >>>>>>>>>> without much cooperation with the GC team. We did have a lot >>>>>>>>>> of feedback when it was first presented a year ago, but it >>>>>>>>>> does not seem like this feedback was incorporated into the >>>>>>>>>> @requires that was eventually built. >>>>>>>>> >>>>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>>>> But not everything is possible, sorry for that. >>>>>>>> >>>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>>> been requesting this feature for 3 years and I was expecting us >>>>>>>> to be able to influence the feature much more than was the case >>>>>>>> now. >>>>>>> >>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>>> >>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>> tag are: >>>>>>>>>> >>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible >>>>>>>>> to use: >>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>> >>>>>>>>> The table with all vars available in jtreg: >>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>> >>>>>>>> The problem with having this matching built in to JTreg is that >>>>>>>> it makes it very hard to change. When we discussed this a year >>>>>>>> ago I think we said that JTreg should only provide a means to >>>>>>>> test against the command line and a hook for running some java >>>>>>>> code in the @requires tag. That way we could put logic like >>>>>>>> this in a test library that is under our control. This would >>>>>>>> make it easy for us to change and also enables us to use >>>>>>>> different logic for different versions. >>>>>>> >>>>>>> I would be glad to have own harness... >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> - the requirement should be per @run tag. Right now we have >>>>>>>>>> to do what you did in this change and use vm.gc=null even >>>>>>>>>> when some tests could actually have been run when a GC was >>>>>>>>>> specified. >>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>> well as test case support. >>>>>>>> >>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>> >>>>>>> Under test case support I mean ability to treat each @run as a >>>>>>> separate test. Now >>>>>>> >>>>>>> @test >>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>> class MyTest { >>>>>>> } >>>>>>> >>>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> - there are many tests that require more than just a specific >>>>>>>>>> GC. Often there are other flags that can't be changed either >>>>>>>>>> for the test to work properly. >>>>>>>>> >>>>>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>>>>> conflicting options. >>>>>>>>> If we address this issue and we are satisfied with solution, >>>>>>>>> we could move further. >>>>>>>> >>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>> Personally I would have preferred that the first step was a >>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>>>> I'm not too happy about how the @requires tag turned out. But >>>>>>>>>> assuming we have to use it the way it is now I guess the >>>>>>>>>> proposed changeset looks good. >>>>>>>>> >>>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>>> jtreg :) >>>>>>>>> And thanks for reviewing it! >>>>>>>> >>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>>> looked at all tests. But if they now pass with the combinations >>>>>>>> that we test with I guess they should be ok. >>>>>>> >>>>>>> Excellent! Thanks a lot! >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>> >>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>> >>>>>>>> Ok. Thanks. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think some of the test, like >>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>>>> >>>>>>>>> These two tests ignore vm flags. >>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>> execute the tests when not needed. >>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>>> tests 4 times, 1 should be enough. >>>>>>>> >>>>>>>> Do we really want to use the @requires functionality for this >>>>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>>>> want the tests to be run once I think Leonid's approach with >>>>>>>> tests lists seems more suitable. >>>>>>> >>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>> >>>>>>> >>>>>>>> But are you sure that this is the reason for the @requires in >>>>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>>>> doesn't fail today it should probably be changed so that it >>>>>>>> does fail if it is run with the wrong GC. >>>>>>> >>>>>>> @requires - is not the silver bullet, but it's quite easy way to >>>>>>> solve a lot of issues. >>>>>>> >>>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>>> tests, which produce a new java process to ignore vm flags >>>>>>> coming from outside. No @requires, no other mechanism could 100% >>>>>>> protect a test from running with conflicting options, but this >>>>>>> is not the goal. >>>>>>> >>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>> collector, there shouldn't mass failures caused by options >>>>>>> conflicts. But a few failures could be handled manually. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> Similarly it looks to me like there are tests that will fail >>>>>>>>>> if you run them with -XX:-UseParallelOldGC or >>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>> >>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>> >>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>> okay, thank for letting us know. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>> some tests are never run and always skipped. Will we somehow >>>>>>>>>> be tracking what gets skipped and make sure that all tests >>>>>>>>>> are at least run once with the correct GC so that it is not >>>>>>>>>> skipped all the time? >>>>>>>>> >>>>>>>>> This is a very good question! >>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>>> soon, after getting fix of: >>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>> >>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>> >>>>>>>>> Other examples: >>>>>>>>> >>>>>>>>> /* >>>>>>>>> *@ignore >>>>>>>>> *@test >>>>>>>>> */ >>>>>>>>> ... >>>>>>>>> >>>>>>>>> /*@bug 4445555 >>>>>>>>> *@test >>>>>>>>> */ >>>>>>>>> ... >>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>> only files with @test on the first place... >>>>>>>>> >>>>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>>>> very actively). But this subject for another discussion, not >>>>>>>>> within RFR :) >>>>>>>> >>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>> >>>>>>> I was going to say "not very actively", but never mind, we know >>>>>>> about this problem. With introducing @requires mechanism it will >>>>>>> become more important! >>>>>>> >>>>>>> >>>>>>> Thanks for your comments! >>>>>>> >>>>>>> -- Dima >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Bengt >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Thu Nov 13 13:42:11 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 14:42:11 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464A8ED.4070708@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> Message-ID: <5464B533.9000100@oracle.com> On 2014-11-13 13:49, Dmitry Fazunenko wrote: > > On 13.11.2014 17:32, Bengt Rutisson wrote: >> >> Hi Evgeniya, >> >> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>> Hi Dmitry, >>> >>> You are right - I've forgotten about copyrights >>> Copyrights and other issues you mentioned fixed. New webrev: >>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >> >> >> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be >> good to add -XX:+UseG1GC to the @run tags and then use @requires >> vm.gc=="G1" | vm.gc == null. >> >> >> The change to test/gc/defnew/HeapChangeLogging.java is unrelated to >> the conflicting GC combinations. Should that really be part of this >> changeset? >> >> >> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >> really need @requires for them? > > Yes, we do. > These tests use TestShrinkAuxiliaryData class which implements its own > mechanism to analyze VM options an skip if not applicable collector is > given. @requires - allows to rely on jtreg. > > Driver mode is a kind of indicator, that the test will spawn its own > java process. I thought the point of @driver was that no external vmoptions were passed to such a test. Is that not the case? Bengt > > Thanks > Dima > >> >> >> Otherwise it look ok to me. >> >> Bengt >> >> >>> >>> Thanks >>> Evgeniya Stepanova >>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>> Hi Evgeniya, >>>> >>>> The fix looks good to me. >>>> >>>> I noticed the following minor things: >>>> - copyrights need to include the year of last change >>>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>>> files, but doesn't contain any changes >>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >>>> 'prohibitedVmOptions' >>>> >>>> Thanks, >>>> Dima >>>> >>>> >>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>> Hi everyone! >>>>> >>>>> Since the decision was made to change only tests that fail because >>>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>>> hotspot part of the JDK-8019361 >>>>> : >>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>> >>>>> Thanks, >>>>> Evgeniya Stepanova >>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>> regarding @requires >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Dima, >>>>>>> >>>>>>> Thanks for the answers. I think the currently proposed patch is >>>>>>> a good start. We will have to evolve the @requires tag in the >>>>>>> future, but let's have that discussion separate from this >>>>>>> review. And we can start that discussion later when we have more >>>>>>> experience with the current version of @requires. >>>>>>> >>>>>>> Thanks for doing this! >>>>>>> Bengt >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>> Hi Bengt, >>>>>>>> >>>>>>>> That's great that we have very closed visions! >>>>>>>> >>>>>>>> The general comment: currently, jtreg doesn't support any sort >>>>>>>> of plugins, so you can't provide a VM specific handler of the >>>>>>>> @requires or another tag. This is very annoying limitation and >>>>>>>> we have to live with it. >>>>>>>> >>>>>>>> A few more comments inline. >>>>>>>> >>>>>>>> >>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Dima, >>>>>>>>> >>>>>>>>> Answers inline. >>>>>>>>> >>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>> Hi Bengt, >>>>>>>>>> >>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>> very much! >>>>>>>>>> >>>>>>>>>> See comments inline. >>>>>>>>>> >>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part >>>>>>>>>>>> of the JDK-8019361 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>>> set another GC. >>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" >>>>>>>>>>>> to skip test if there is a conflict >>>>>>>>>>> >>>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>>> start sorting this out. >>>>>>>>>>> >>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>> developed without much cooperation with the GC team. We did >>>>>>>>>>> have a lot of feedback when it was first presented a year >>>>>>>>>>> ago, but it does not seem like this feedback was >>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>> >>>>>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>>>>> But not everything is possible, sorry for that. >>>>>>>>> >>>>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>>>> been requesting this feature for 3 years and I was expecting >>>>>>>>> us to be able to influence the feature much more than was the >>>>>>>>> case now. >>>>>>>> >>>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>>>> >>>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>>> tag are: >>>>>>>>>>> >>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>> possible to use: >>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>> >>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>> >>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>> that it makes it very hard to change. When we discussed this a >>>>>>>>> year ago I think we said that JTreg should only provide a >>>>>>>>> means to test against the command line and a hook for running >>>>>>>>> some java code in the @requires tag. That way we could put >>>>>>>>> logic like this in a test library that is under our control. >>>>>>>>> This would make it easy for us to change and also enables us >>>>>>>>> to use different logic for different versions. >>>>>>>> >>>>>>>> I would be glad to have own harness... >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> - the requirement should be per @run tag. Right now we have >>>>>>>>>>> to do what you did in this change and use vm.gc=null even >>>>>>>>>>> when some tests could actually have been run when a GC was >>>>>>>>>>> specified. >>>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>>> well as test case support. >>>>>>>>> >>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>> >>>>>>>> Under test case support I mean ability to treat each @run as a >>>>>>>> separate test. Now >>>>>>>> >>>>>>>> @test >>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>> class MyTest { >>>>>>>> } >>>>>>>> >>>>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>> >>>>>>>>>> yes. conflicting GC is just the most popular problem caused >>>>>>>>>> by conflicting options. >>>>>>>>>> If we address this issue and we are satisfied with solution, >>>>>>>>>> we could move further. >>>>>>>>> >>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>> Personally I would have preferred that the first step was a >>>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>>>>> I'm not too happy about how the @requires tag turned out. >>>>>>>>>>> But assuming we have to use it the way it is now I guess the >>>>>>>>>>> proposed changeset looks good. >>>>>>>>>> >>>>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>>>> jtreg :) >>>>>>>>>> And thanks for reviewing it! >>>>>>>>> >>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>>>> looked at all tests. But if they now pass with the >>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>> >>>>>>>> Excellent! Thanks a lot! >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>> >>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>> >>>>>>>>> Ok. Thanks. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think some of the test, like >>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if >>>>>>>>>>> you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if >>>>>>>>>>> you run with -XX:-UseParNewGC. Could you test these two >>>>>>>>>>> cases too? >>>>>>>>>> >>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>>> execute the tests when not needed. >>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>>>> tests 4 times, 1 should be enough. >>>>>>>>> >>>>>>>>> Do we really want to use the @requires functionality for this >>>>>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>>>>> want the tests to be run once I think Leonid's approach with >>>>>>>>> tests lists seems more suitable. >>>>>>>> >>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>> >>>>>>>> >>>>>>>>> But are you sure that this is the reason for the @requires in >>>>>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>>>>> doesn't fail today it should probably be changed so that it >>>>>>>>> does fail if it is run with the wrong GC. >>>>>>>> >>>>>>>> @requires - is not the silver bullet, but it's quite easy way >>>>>>>> to solve a lot of issues. >>>>>>>> >>>>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>>>> tests, which produce a new java process to ignore vm flags >>>>>>>> coming from outside. No @requires, no other mechanism could >>>>>>>> 100% protect a test from running with conflicting options, but >>>>>>>> this is not the goal. >>>>>>>> >>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Similarly it looks to me like there are tests that will fail >>>>>>>>>>> if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>> >>>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>>> >>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>> okay, thank for letting us know. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>>> some tests are never run and always skipped. Will we somehow >>>>>>>>>>> be tracking what gets skipped and make sure that all tests >>>>>>>>>>> are at least run once with the correct GC so that it is not >>>>>>>>>>> skipped all the time? >>>>>>>>>> >>>>>>>>>> This is a very good question! >>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>>>> soon, after getting fix of: >>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>> >>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>> >>>>>>>>>> Other examples: >>>>>>>>>> >>>>>>>>>> /* >>>>>>>>>> *@ignore >>>>>>>>>> *@test >>>>>>>>>> */ >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>>> /*@bug 4445555 >>>>>>>>>> *@test >>>>>>>>>> */ >>>>>>>>>> ... >>>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>>> only files with @test on the first place... >>>>>>>>>> >>>>>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>>>>> very actively). But this subject for another discussion, not >>>>>>>>>> within RFR :) >>>>>>>>> >>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>> >>>>>>>> I was going to say "not very actively", but never mind, we know >>>>>>>> about this problem. With introducing @requires mechanism it >>>>>>>> will become more important! >>>>>>>> >>>>>>>> >>>>>>>> Thanks for your comments! >>>>>>>> >>>>>>>> -- Dima >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>> >>> -- >>> /Evgeniya Stepanova/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Thu Nov 13 12:56:35 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 13 Nov 2014 16:56:35 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B533.9000100@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> Message-ID: <5464AA83.4030306@oracle.com> On 13.11.2014 17:42, Bengt Rutisson wrote: > > On 2014-11-13 13:49, Dmitry Fazunenko wrote: >> >> On 13.11.2014 17:32, Bengt Rutisson wrote: >>> >>> Hi Evgeniya, >>> >>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>> Hi Dmitry, >>>> >>>> You are right - I've forgotten about copyrights >>>> Copyrights and other issues you mentioned fixed. New webrev: >>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>> >>> >>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be >>> good to add -XX:+UseG1GC to the @run tags and then use @requires >>> vm.gc=="G1" | vm.gc == null. >>> >>> >>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated to >>> the conflicting GC combinations. Should that really be part of this >>> changeset? >>> >>> >>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >>> really need @requires for them? >> >> Yes, we do. >> These tests use TestShrinkAuxiliaryData class which implements its >> own mechanism to analyze VM options an skip if not applicable >> collector is given. @requires - allows to rely on jtreg. >> >> Driver mode is a kind of indicator, that the test will spawn its own >> java process. > > I thought the point of @driver was that no external vmoptions were > passed to such a test. Is that not the case? In the driver mode VM is started without external VM flags. Those flags are passed to the tests via system property. The driver mode is a sort of shell to start something else. -- Dima > > Bengt > >> >> Thanks >> Dima >> >>> >>> >>> Otherwise it look ok to me. >>> >>> Bengt >>> >>> >>>> >>>> Thanks >>>> Evgeniya Stepanova >>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>> Hi Evgeniya, >>>>> >>>>> The fix looks good to me. >>>>> >>>>> I noticed the following minor things: >>>>> - copyrights need to include the year of last change >>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>>>> files, but doesn't contain any changes >>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >>>>> 'prohibitedVmOptions' >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>> Hi everyone! >>>>>> >>>>>> Since the decision was made to change only tests that fail >>>>>> because of conflict for now (skip "selfish" tests), I post new >>>>>> webrev for hotspot part of the JDK-8019361 >>>>>> : >>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>> >>>>>> Thanks, >>>>>> Evgeniya Stepanova >>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>>> regarding @requires >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Dima, >>>>>>>> >>>>>>>> Thanks for the answers. I think the currently proposed patch is >>>>>>>> a good start. We will have to evolve the @requires tag in the >>>>>>>> future, but let's have that discussion separate from this >>>>>>>> review. And we can start that discussion later when we have >>>>>>>> more experience with the current version of @requires. >>>>>>>> >>>>>>>> Thanks for doing this! >>>>>>>> Bengt >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>> Hi Bengt, >>>>>>>>> >>>>>>>>> That's great that we have very closed visions! >>>>>>>>> >>>>>>>>> The general comment: currently, jtreg doesn't support any sort >>>>>>>>> of plugins, so you can't provide a VM specific handler of the >>>>>>>>> @requires or another tag. This is very annoying limitation and >>>>>>>>> we have to live with it. >>>>>>>>> >>>>>>>>> A few more comments inline. >>>>>>>>> >>>>>>>>> >>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Dima, >>>>>>>>>> >>>>>>>>>> Answers inline. >>>>>>>>>> >>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>>> very much! >>>>>>>>>>> >>>>>>>>>>> See comments inline. >>>>>>>>>>> >>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>> >>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>>>> set another GC. >>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" >>>>>>>>>>>>> to skip test if there is a conflict >>>>>>>>>>>> >>>>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>>>> start sorting this out. >>>>>>>>>>>> >>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>> developed without much cooperation with the GC team. We did >>>>>>>>>>>> have a lot of feedback when it was first presented a year >>>>>>>>>>>> ago, but it does not seem like this feedback was >>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>> >>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>> >>>>>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>>>>> been requesting this feature for 3 years and I was expecting >>>>>>>>>> us to be able to influence the feature much more than was the >>>>>>>>>> case now. >>>>>>>>> >>>>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>>>>> >>>>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>>>> tag are: >>>>>>>>>>>> >>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>> possible to use: >>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>> >>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>> >>>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>>> that it makes it very hard to change. When we discussed this >>>>>>>>>> a year ago I think we said that JTreg should only provide a >>>>>>>>>> means to test against the command line and a hook for running >>>>>>>>>> some java code in the @requires tag. That way we could put >>>>>>>>>> logic like this in a test library that is under our control. >>>>>>>>>> This would make it easy for us to change and also enables us >>>>>>>>>> to use different logic for different versions. >>>>>>>>> >>>>>>>>> I would be glad to have own harness... >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> - the requirement should be per @run tag. Right now we have >>>>>>>>>>>> to do what you did in this change and use vm.gc=null even >>>>>>>>>>>> when some tests could actually have been run when a GC was >>>>>>>>>>>> specified. >>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>>>> well as test case support. >>>>>>>>>> >>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>> >>>>>>>>> Under test case support I mean ability to treat each @run as a >>>>>>>>> separate test. Now >>>>>>>>> >>>>>>>>> @test >>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>> class MyTest { >>>>>>>>> } >>>>>>>>> >>>>>>>>> is always a single test. You can't exclude, or re-run a part >>>>>>>>> of it. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>> >>>>>>>>>>> yes. conflicting GC is just the most popular problem caused >>>>>>>>>>> by conflicting options. >>>>>>>>>>> If we address this issue and we are satisfied with solution, >>>>>>>>>>> we could move further. >>>>>>>>>> >>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>> Personally I would have preferred that the first step was a >>>>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>>> implementation of the @requires tag. I just want to say >>>>>>>>>>>> that I'm not too happy about how the @requires tag turned >>>>>>>>>>>> out. But assuming we have to use it the way it is now I >>>>>>>>>>>> guess the proposed changeset looks good. >>>>>>>>>>> >>>>>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>>>>> jtreg :) >>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>> >>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>>>>> looked at all tests. But if they now pass with the >>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>> >>>>>>>>> Excellent! Thanks a lot! >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests are >>>>>>>>>>>>> being excluded as expected. No tests failed because of the >>>>>>>>>>>>> conflict. >>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>> >>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>>> >>>>>>>>>> Ok. Thanks. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if >>>>>>>>>>>> you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if >>>>>>>>>>>> you run with -XX:-UseParNewGC. Could you test these two >>>>>>>>>>>> cases too? >>>>>>>>>>> >>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>>>> execute the tests when not needed. >>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>>>>> tests 4 times, 1 should be enough. >>>>>>>>>> >>>>>>>>>> Do we really want to use the @requires functionality for this >>>>>>>>>> purpose? It seems like a way of misusing @requires. If we >>>>>>>>>> just want the tests to be run once I think Leonid's approach >>>>>>>>>> with tests lists seems more suitable. >>>>>>>>> >>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>> >>>>>>>>> >>>>>>>>>> But are you sure that this is the reason for the @requires in >>>>>>>>>> this case? TestDefNewCMS does sound like a test that is >>>>>>>>>> DefNew specific. I don't see a reason to run it with ParNew. >>>>>>>>>> If it doesn't fail today it should probably be changed so >>>>>>>>>> that it does fail if it is run with the wrong GC. >>>>>>>>> >>>>>>>>> @requires - is not the silver bullet, but it's quite easy way >>>>>>>>> to solve a lot of issues. >>>>>>>>> >>>>>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>>>>> tests, which produce a new java process to ignore vm flags >>>>>>>>> coming from outside. No @requires, no other mechanism could >>>>>>>>> 100% protect a test from running with conflicting options, but >>>>>>>>> this is not the goal. >>>>>>>>> >>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>> >>>>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>>>> >>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>>>> some tests are never run and always skipped. Will we >>>>>>>>>>>> somehow be tracking what gets skipped and make sure that >>>>>>>>>>>> all tests are at least run once with the correct GC so that >>>>>>>>>>>> it is not skipped all the time? >>>>>>>>>>> >>>>>>>>>>> This is a very good question! >>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>>>>> soon, after getting fix of: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>> >>>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>> >>>>>>>>>>> Other examples: >>>>>>>>>>> >>>>>>>>>>> /* >>>>>>>>>>> *@ignore >>>>>>>>>>> *@test >>>>>>>>>>> */ >>>>>>>>>>> ... >>>>>>>>>>> >>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>> *@test >>>>>>>>>>> */ >>>>>>>>>>> ... >>>>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>>>> only files with @test on the first place... >>>>>>>>>>> >>>>>>>>>>> So, making sure that tests do not disappear is important >>>>>>>>>>> SQE task, we know about that, we're thinking on solution >>>>>>>>>>> (may be very actively). But this subject for another >>>>>>>>>>> discussion, not within RFR :) >>>>>>>>>> >>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>> >>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>> know about this problem. With introducing @requires mechanism >>>>>>>>> it will become more important! >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks for your comments! >>>>>>>>> >>>>>>>>> -- Dima >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.r.chase at oracle.com Thu Nov 13 13:57:12 2014 From: david.r.chase at oracle.com (David Chase) Date: Thu, 13 Nov 2014 08:57:12 -0500 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups In-Reply-To: <5464B2FB.1050606@oracle.com> References: <5464B2FB.1050606@oracle.com> Message-ID: On 2014-11-13, at 8:32 AM, Zolt?n Maj? wrote: > The problem with this approach is that if we want to execute a new (short-running) JTREG test in JPRT, the TEST.groups file must be updated to execute the newly added JTREG test. Moreover, the current selection of tests is based on a per-group time limit of 10 minutes (i.e., each test group executes 10 minutes on the slowest JPRT platform available), but it seems that a 15-minute limit is also tolerable. > > Solution: Move all test directories of the form hotspot/test/ to "functionality-based" directories. Each functionality-based directory contains tests related to a specific functionality of the VM, for example: hotspot/test/compiler/c1, hotspot/test/compiler/c2, and hotspot/test/compiler/codecache. In the TEST.groups file we list only functionality-based directories, and not individual tests. We also exclude long-running tests. > After changes (group / #tests / execution time on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848): > > hotspot_compiler_1 / 107 / 16m55s > hotspot_compiler_2 / 76 / 14m49s > hotspot_compiler_3 / 59 / 13m44s (Not a Reviewer) If 15 minutes is a hard limit, I note that 16m 55s is larger than that, and 14m49s is pretty darn close. Don?t we run a risk of test-time creep as new tests are added and by-default are run? But those two issues are small compared to the goodness of this test re-org. I think I can live with time-creep risk, though we might need to kick a few tests out of the first group. David -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bengt.rutisson at oracle.com Thu Nov 13 13:59:34 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 14:59:34 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464AA83.4030306@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> Message-ID: <5464B946.90806@oracle.com> On 2014-11-13 13:56, Dmitry Fazunenko wrote: > > On 13.11.2014 17:42, Bengt Rutisson wrote: >> >> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>> >>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>> >>>> Hi Evgeniya, >>>> >>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>> Hi Dmitry, >>>>> >>>>> You are right - I've forgotten about copyrights >>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>> >>>> >>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would >>>> be good to add -XX:+UseG1GC to the @run tags and then use >>>> @requires vm.gc=="G1" | vm.gc == null. >>>> >>>> >>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated to >>>> the conflicting GC combinations. Should that really be part of this >>>> changeset? >>>> >>>> >>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >>>> really need @requires for them? >>> >>> Yes, we do. >>> These tests use TestShrinkAuxiliaryData class which implements its >>> own mechanism to analyze VM options an skip if not applicable >>> collector is given. @requires - allows to rely on jtreg. >>> >>> Driver mode is a kind of indicator, that the test will spawn its own >>> java process. >> >> I thought the point of @driver was that no external vmoptions were >> passed to such a test. Is that not the case? > > In the driver mode VM is started without external VM flags. Those > flags are passed to the tests via system property. > The driver mode is a sort of shell to start something else. Right. So, why would you need @requires on the TestShrinkAuxiliaryDataXX tests because the utility TestShrinkAuxiliaryData picks up the vm flags through Utils.getTestJavaOpts(). What's the point in running this in a driver mode when they anyway pick up the vm flags? I'm asking because adding @requires to these tests means that we will run them less often than we do now. So, I'm a bit worried that we reduce the amount of testing we do. Bengt > > -- Dima > > >> >> Bengt >> >>> >>> Thanks >>> Dima >>> >>>> >>>> >>>> Otherwise it look ok to me. >>>> >>>> Bengt >>>> >>>> >>>>> >>>>> Thanks >>>>> Evgeniya Stepanova >>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>> Hi Evgeniya, >>>>>> >>>>>> The fix looks good to me. >>>>>> >>>>>> I noticed the following minor things: >>>>>> - copyrights need to include the year of last change >>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>>>>> files, but doesn't contain any changes >>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>> variable 'prohibitedVmOptions' >>>>>> >>>>>> Thanks, >>>>>> Dima >>>>>> >>>>>> >>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>> Hi everyone! >>>>>>> >>>>>>> Since the decision was made to change only tests that fail >>>>>>> because of conflict for now (skip "selfish" tests), I post new >>>>>>> webrev for hotspot part of the JDK-8019361 >>>>>>> : >>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Evgeniya Stepanova >>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>>>> regarding @requires >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dima >>>>>>>> >>>>>>>> >>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi Dima, >>>>>>>>> >>>>>>>>> Thanks for the answers. I think the currently proposed patch >>>>>>>>> is a good start. We will have to evolve the @requires tag in >>>>>>>>> the future, but let's have that discussion separate from this >>>>>>>>> review. And we can start that discussion later when we have >>>>>>>>> more experience with the current version of @requires. >>>>>>>>> >>>>>>>>> Thanks for doing this! >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>> Hi Bengt, >>>>>>>>>> >>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>> >>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>> sort of plugins, so you can't provide a VM specific handler >>>>>>>>>> of the @requires or another tag. This is very annoying >>>>>>>>>> limitation and we have to live with it. >>>>>>>>>> >>>>>>>>>> A few more comments inline. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Dima, >>>>>>>>>>> >>>>>>>>>>> Answers inline. >>>>>>>>>>> >>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>> >>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>>>> very much! >>>>>>>>>>>> >>>>>>>>>>>> See comments inline. >>>>>>>>>>>> >>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>> >>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>>>>> set another GC. >>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" >>>>>>>>>>>>>> to skip test if there is a conflict >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>>>>> start sorting this out. >>>>>>>>>>>>> >>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>> developed without much cooperation with the GC team. We >>>>>>>>>>>>> did have a lot of feedback when it was first presented a >>>>>>>>>>>>> year ago, but it does not seem like this feedback was >>>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>>> >>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>> >>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>> expecting us to be able to influence the feature much more >>>>>>>>>>> than was the case now. >>>>>>>>>> >>>>>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>>>> forward and I won't object to it. But I am pretty >>>>>>>>>>>>> convinced that we will soon run in to the limitations of >>>>>>>>>>>>> the current @requires implementation and we will have to >>>>>>>>>>>>> redo this work. >>>>>>>>>>>>> >>>>>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>>>>> tag are: >>>>>>>>>>>>> >>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. >>>>>>>>>>>>> It would have been better to just "require" any command >>>>>>>>>>>>> line flag. >>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>> possible to use: >>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>> >>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>> >>>>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>>>> that it makes it very hard to change. When we discussed this >>>>>>>>>>> a year ago I think we said that JTreg should only provide a >>>>>>>>>>> means to test against the command line and a hook for >>>>>>>>>>> running some java code in the @requires tag. That way we >>>>>>>>>>> could put logic like this in a test library that is under >>>>>>>>>>> our control. This would make it easy for us to change and >>>>>>>>>>> also enables us to use different logic for different versions. >>>>>>>>>> >>>>>>>>>> I would be glad to have own harness... >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>> have to do what you did in this change and use vm.gc=null >>>>>>>>>>>>> even when some tests could actually have been run when a >>>>>>>>>>>>> GC was specified. >>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>>>>> well as test case support. >>>>>>>>>>> >>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> Under test case support I mean ability to treat each @run as >>>>>>>>>> a separate test. Now >>>>>>>>>> >>>>>>>>>> @test >>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>> class MyTest { >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> is always a single test. You can't exclude, or re-run a part >>>>>>>>>> of it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>> >>>>>>>>>>>> yes. conflicting GC is just the most popular problem caused >>>>>>>>>>>> by conflicting options. >>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>> solution, we could move further. >>>>>>>>>>> >>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>> Personally I would have preferred that the first step was a >>>>>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>>>> implementation of the @requires tag. I just want to say >>>>>>>>>>>>> that I'm not too happy about how the @requires tag turned >>>>>>>>>>>>> out. But assuming we have to use it the way it is now I >>>>>>>>>>>>> guess the proposed changeset looks good. >>>>>>>>>>>> >>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>> about jtreg :) >>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>> >>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have >>>>>>>>>>> not looked at all tests. But if they now pass with the >>>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>>> >>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests are >>>>>>>>>>>>>> being excluded as expected. No tests failed because of >>>>>>>>>>>>>> the conflict. >>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>> >>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>>>> >>>>>>>>>>> Ok. Thanks. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if >>>>>>>>>>>>> you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if >>>>>>>>>>>>> you run with -XX:-UseParNewGC. Could you test these two >>>>>>>>>>>>> cases too? >>>>>>>>>>>> >>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>>>>> execute the tests when not needed. >>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>> >>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>> this purpose? It seems like a way of misusing @requires. If >>>>>>>>>>> we just want the tests to be run once I think Leonid's >>>>>>>>>>> approach with tests lists seems more suitable. >>>>>>>>>> >>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> But are you sure that this is the reason for the @requires >>>>>>>>>>> in this case? TestDefNewCMS does sound like a test that is >>>>>>>>>>> DefNew specific. I don't see a reason to run it with ParNew. >>>>>>>>>>> If it doesn't fail today it should probably be changed so >>>>>>>>>>> that it does fail if it is run with the wrong GC. >>>>>>>>>> >>>>>>>>>> @requires - is not the silver bullet, but it's quite easy way >>>>>>>>>> to solve a lot of issues. >>>>>>>>>> >>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>> "selfish" tests, which produce a new java process to ignore >>>>>>>>>> vm flags coming from outside. No @requires, no other >>>>>>>>>> mechanism could 100% protect a test from running with >>>>>>>>>> conflicting options, but this is not the goal. >>>>>>>>>> >>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>>> >>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>>>>> >>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>>>>> some tests are never run and always skipped. Will we >>>>>>>>>>>>> somehow be tracking what gets skipped and make sure that >>>>>>>>>>>>> all tests are at least run once with the correct GC so >>>>>>>>>>>>> that it is not skipped all the time? >>>>>>>>>>>> >>>>>>>>>>>> This is a very good question! >>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will >>>>>>>>>>>> do soon, after getting fix of: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>> >>>>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>> >>>>>>>>>>>> Other examples: >>>>>>>>>>>> >>>>>>>>>>>> /* >>>>>>>>>>>> *@ignore >>>>>>>>>>>> *@test >>>>>>>>>>>> */ >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>> *@test >>>>>>>>>>>> */ >>>>>>>>>>>> ... >>>>>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>>>>> only files with @test on the first place... >>>>>>>>>>>> >>>>>>>>>>>> So, making sure that tests do not disappear is important >>>>>>>>>>>> SQE task, we know about that, we're thinking on solution >>>>>>>>>>>> (may be very actively). But this subject for another >>>>>>>>>>>> discussion, not within RFR :) >>>>>>>>>>> >>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>> >>>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>>> know about this problem. With introducing @requires mechanism >>>>>>>>>> it will become more important! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks for your comments! >>>>>>>>>> >>>>>>>>>> -- Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Dima >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Bengt >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>> >>>>> -- >>>>> /Evgeniya Stepanova/ >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nils.eliasson at oracle.com Thu Nov 13 14:11:54 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Thu, 13 Nov 2014 15:11:54 +0100 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out Message-ID: <5464BC2A.4020206@oracle.com> Hi, Please review this small change. 1) Fixing a deadlock in diagnostic command dcmd print_compile_queues - between the CompileQueue_lock and safepointing. The CompileQueue_lock requires that we are not at a safepoint when taking it. Otherwise a java-thread that already holds the lock will block when trying to get another lock. In this specific case the compiler threads are Java threads, they take the CompileQueue_lock frequently and some time takes the CompileTaskAlloc_lock after that. Fixing this by making the diagnostic command not request a safepoint, 2) Refactoring away the lock() accessor in CompileQueue and stating the lock explicitly. This removes an element of surprise and makes it easier understanding the code. webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ bug: https://bugs.openjdk.java.net/browse/JDK-8061256 Thanks, Nils Eliasson From zoltan.majo at oracle.com Thu Nov 13 14:12:39 2014 From: zoltan.majo at oracle.com (=?windows-1252?Q?Zolt=E1n_Maj=F3?=) Date: Thu, 13 Nov 2014 15:12:39 +0100 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups In-Reply-To: References: <5464B2FB.1050606@oracle.com> Message-ID: <5464BC57.80200@oracle.com> Hi David, thank you for the feedback! On 11/13/2014 02:57 PM, David Chase wrote: > On 2014-11-13, at 8:32 AM, Zolt?n Maj? wrote: >> The problem with this approach is that if we want to execute a new (short-running) JTREG test in JPRT, the TEST.groups file must be updated to execute the newly added JTREG test. Moreover, the current selection of tests is based on a per-group time limit of 10 minutes (i.e., each test group executes 10 minutes on the slowest JPRT platform available), but it seems that a 15-minute limit is also tolerable. >> >> Solution: Move all test directories of the form hotspot/test/ to "functionality-based" directories. Each functionality-based directory contains tests related to a specific functionality of the VM, for example: hotspot/test/compiler/c1, hotspot/test/compiler/c2, and hotspot/test/compiler/codecache. In the TEST.groups file we list only functionality-based directories, and not individual tests. We also exclude long-running tests. >> After changes (group / #tests / execution time on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848): >> >> hotspot_compiler_1 / 107 / 16m55s >> hotspot_compiler_2 / 76 / 14m49s >> hotspot_compiler_3 / 59 / 13m44s > (Not a Reviewer) > > If 15 minutes is a hard limit, I note that 16m 55s is larger than that, and 14m49s is pretty darn close. The limit is not that "hard", I think. And by accident, the numbers I reported before include initialization and cleanup times as well, and these are not easily controllable. Here is only the "work" time for the same runs as before: hotspot_compiler_1 / work=15m6s hotspot_compiler_2 / work=13m55s hotspot_compiler_3 / work=12m43s > Don?t we run a risk of test-time creep as new tests are added and by-default are run? Yes, we do, but I think the limit is not that hard, and we can re-evaluate test times periodically (when the number of tests significantly changes or when we recycle some of our slowest machines). > But those two issues are small compared to the goodness of this test re-org. > I think I can live with time-creep risk, though we might need to kick a few tests out of the first group. I'll look into that. Thanks and best regards, Zoltan From bengt.rutisson at oracle.com Thu Nov 13 14:29:00 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 13 Nov 2014 15:29:00 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B946.90806@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> Message-ID: > 13 nov 2014 kl. 14:59 skrev Bengt Rutisson : > > >> On 2014-11-13 13:56, Dmitry Fazunenko wrote: >> >>> On 13.11.2014 17:42, Bengt Rutisson wrote: >>> >>>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>> >>>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>> >>>>> Hi Evgeniya, >>>>> >>>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>> Hi Dmitry, >>>>>> >>>>>> You are right - I've forgotten about copyrights >>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>> >>>>> >>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be good to add -XX:+UseG1GC to the @run tags and then use @requires vm.gc=="G1" | vm.gc == null. >>>>> >>>>> >>>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated to the conflicting GC combinations. Should that really be part of this changeset? >>>>> >>>>> >>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we really need @requires for them? >>>> >>>> Yes, we do. >>>> These tests use TestShrinkAuxiliaryData class which implements its own mechanism to analyze VM options an skip if not applicable collector is given. @requires - allows to rely on jtreg. >>>> >>>> Driver mode is a kind of indicator, that the test will spawn its own java process. >>> >>> I thought the point of @driver was that no external vmoptions were passed to such a test. Is that not the case? >> >> In the driver mode VM is started without external VM flags. Those flags are passed to the tests via system property. >> The driver mode is a sort of shell to start something else. > > Right. So, why would you need @requires on the TestShrinkAuxiliaryDataXX tests because the utility TestShrinkAuxiliaryData picks up the vm flags through Utils.getTestJavaOpts(). What's the point in running this in a driver mode when they anyway pick up the vm flags? The first sentence in the above paragraph makes more sense if you remove the words "why would". Bengt > > I'm asking because adding @requires to these tests means that we will run them less often than we do now. So, I'm a bit worried that we reduce the amount of testing we do. > > Bengt > >> >> -- Dima >> >> >>> >>> Bengt >>> >>>> >>>> Thanks >>>> Dima >>>> >>>>> >>>>> >>>>> Otherwise it look ok to me. >>>>> >>>>> Bengt >>>>> >>>>> >>>>>> >>>>>> Thanks >>>>>> Evgeniya Stepanova >>>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> The fix looks good to me. >>>>>>> >>>>>>> I noticed the following minor things: >>>>>>> - copyrights need to include the year of last change >>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated files, but doesn't contain any changes >>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable 'prohibitedVmOptions' >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>> Hi everyone! >>>>>>>> >>>>>>>> Since the decision was made to change only tests that fail because of conflict for now (skip "selfish" tests), I post new webrev for hotspot part of the JDK-8019361: >>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>> Nice plan! Please feel free to send me any feedback/questions regarding @requires >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Dima, >>>>>>>>>> >>>>>>>>>> Thanks for the answers. I think the currently proposed patch is a good start. We will have to evolve the @requires tag in the future, but let's have that discussion separate from this review. And we can start that discussion later when we have more experience with the current version of @requires. >>>>>>>>>> >>>>>>>>>> Thanks for doing this! >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>> >>>>>>>>>>> The general comment: currently, jtreg doesn't support any sort of plugins, so you can't provide a VM specific handler of the @requires or another tag. This is very annoying limitation and we have to live with it. >>>>>>>>>>> >>>>>>>>>>> A few more comments inline. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Dima, >>>>>>>>>>>> >>>>>>>>>>>> Answers inline. >>>>>>>>>>>> >>>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it very much! >>>>>>>>>>>>> >>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>> >>>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part of the JDK-8019361 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg set another GC. >>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to skip test if there is a conflict >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for fixing this! It is really great that we finally start sorting this out. >>>>>>>>>>>>>> >>>>>>>>>>>>>> First a general comment. The @requires tag has been developed without much cooperation with the GC team. We did have a lot of feedback when it was first presented a year ago, but it does not seem like this feedback was incorporated into the @requires that was eventually built. >>>>>>>>>>>>> >>>>>>>>>>>>> We tried to implement as much developer's wishes as possible. But not everything is possible, sorry for that. >>>>>>>>>>>> >>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we have been requesting this feature for 3 years and I was expecting us to be able to influence the feature much more than was the case now. >>>>>>>>>>> >>>>>>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think this change that gets proposed now is a big step forward and I won't object to it. But I am pretty convinced that we will soon run in to the limitations of the current @requires implementation and we will have to redo this work. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Some of the points I don't really like about the @requires tag are: >>>>>>>>>>>>>> >>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It would have been better to just "require" any command line flag. >>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible to use: >>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>> >>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>> >>>>>>>>>>>> The problem with having this matching built in to JTreg is that it makes it very hard to change. When we discussed this a year ago I think we said that JTreg should only provide a means to test against the command line and a hook for running some java code in the @requires tag. That way we could put logic like this in a test library that is under our control. This would make it easy for us to change and also enables us to use different logic for different versions. >>>>>>>>>>> >>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we have to do what you did in this change and use vm.gc=null even when some tests could actually have been run when a GC was specified. >>>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, as well as test case support. >>>>>>>>>>>> >>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> Under test case support I mean ability to treat each @run as a separate test. Now >>>>>>>>>>> >>>>>>>>>>> @test >>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>> class MyTest { >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - there are many tests that require more than just a specific GC. Often there are other flags that can't be changed either for the test to work properly. >>>>>>>>>>>>> >>>>>>>>>>>>> yes. conflicting GC is just the most popular problem caused by conflicting options. >>>>>>>>>>>>> If we address this issue and we are satisfied with solution, we could move further. >>>>>>>>>>>> >>>>>>>>>>>> Yes, I agree that taking one step at the time is good. Personally I would have preferred that the first step was a "just run the command line as specified in the @run tag" step. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Maybe this is not the right place to discuss the current implementation of the @requires tag. I just want to say that I'm not too happy about how the @requires tag turned out. But assuming we have to use it the way it is now I guess the proposed changeset looks good. >>>>>>>>>>>>> >>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not about jtreg :) >>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>> >>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not looked at all tests. But if they now pass with the combinations that we test with I guess they should be ok. >>>>>>>>>>> >>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep and without any GC flag). Tests are being excluded as expected. No tests failed because of the conflict. >>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>> >>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>>>>> >>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think some of the test, like test/gc/startup_warnings/TestDefNewCMS.java, will fail if you run with -XX:+UseParNewGC. Others, like test/gc/startup_warnings/TestParNewCMS.java, will fail if you run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>>>>>>>> >>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>> Add @requires here is not necessary, but it will allow not execute the tests when not needed. >>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these tests 4 times, 1 should be enough. >>>>>>>>>>>> >>>>>>>>>>>> Do we really want to use the @requires functionality for this purpose? It seems like a way of misusing @requires. If we just want the tests to be run once I think Leonid's approach with tests lists seems more suitable. >>>>>>>>>>> >>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> But are you sure that this is the reason for the @requires in this case? TestDefNewCMS does sound like a test that is DefNew specific. I don't see a reason to run it with ParNew. If it doesn't fail today it should probably be changed so that it does fail if it is run with the wrong GC. >>>>>>>>>>> >>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy way to solve a lot of issues. >>>>>>>>>>> >>>>>>>>>>> I hope, @requires will allow to reduce the number of "selfish" tests, which produce a new java process to ignore vm flags coming from outside. No @requires, no other mechanism could 100% protect a test from running with conflicting options, but this is not the goal. >>>>>>>>>>> >>>>>>>>>>> If one runs tests with an exotic option, like a new G2 collector, there shouldn't mass failures caused by options conflicts. But a few failures could be handled manually. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Similarly it looks to me like there are tests that will fail if you run them with -XX:-UseParallelOldGC or -XX:+UseParallelOldGC. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm about to push a changeset that removes them: >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Is there some way of making sure that all tests are run at one time or another. With this change there is a risk that some tests are never run and always skipped. Will we somehow be tracking what gets skipped and make sure that all tests are at least run once with the correct GC so that it is not skipped all the time? >>>>>>>>>>>>> >>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do soon, after getting fix of: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>> >>>>>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>> >>>>>>>>>>>>> Other examples: >>>>>>>>>>>>> >>>>>>>>>>>>> /* >>>>>>>>>>>>> *@ignore >>>>>>>>>>>>> *@test >>>>>>>>>>>>> */ >>>>>>>>>>>>> ... >>>>>>>>>>>>> >>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>> *@test >>>>>>>>>>>>> */ >>>>>>>>>>>>> ... >>>>>>>>>>>>> Such tests will never be run, because jtreg treats as test only files with @test on the first place... >>>>>>>>>>>>> >>>>>>>>>>>>> So, making sure that tests do not disappear is important SQE task, we know about that, we're thinking on solution (may be very actively). But this subject for another discussion, not within RFR :) >>>>>>>>>>>> >>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>> >>>>>>>>>>> I was going to say "not very actively", but never mind, we know about this problem. With introducing @requires mechanism it will become more important! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks for your comments! >>>>>>>>>>> >>>>>>>>>>> -- Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Dima >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>> >>>>>>>> -- >>>>>>>> Evgeniya Stepanova >>>>>> >>>>>> -- >>>>>> Evgeniya Stepanova > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeniya.stepanova at oracle.com Thu Nov 13 14:33:10 2014 From: evgeniya.stepanova at oracle.com (Evgeniya Stepanova) Date: Thu, 13 Nov 2014 18:33:10 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B2E6.9070802@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> Message-ID: <5464C126.3020003@oracle.com> Hi Bengt, On 13.11.2014 17:32, Bengt Rutisson wrote: > > Hi Evgeniya, > > On 2014-11-12 17:28, Evgeniya Stepanova wrote: >> Hi Dmitry, >> >> You are right - I've forgotten about copyrights >> Copyrights and other issues you mentioned fixed. New webrev: >> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ > > > For /test/gc/arguments/TestG1HeapRegionSize.java I think it would be > good to add -XX:+UseG1GC to the @run tags and then use @requires > vm.gc=="G1" | vm.gc == null. > It is a good idea, I'll change the test > The change to test/gc/defnew/HeapChangeLogging.java is unrelated to > the conflicting GC combinations. Should that really be part of this > changeset? > This change is from previous webrev. Test is "selfish" and I've removed @requires from it, but left comments change that Jon pointed to me. So this change is really not related to the @requires. I'll remove them if it is confuse you. > > The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we > really need @requires for them? > > > Otherwise it look ok to me. > > Bengt > > >> >> Thanks >> Evgeniya Stepanova >> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>> Hi Evgeniya, >>> >>> The fix looks good to me. >>> >>> I noticed the following minor things: >>> - copyrights need to include the year of last change >>> - test/gc/defnew/HeapChangeLogging.java - is listed among updated >>> files, but doesn't contain any changes >>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed variable >>> 'prohibitedVmOptions' >>> >>> Thanks, >>> Dima >>> >>> >>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>> Hi everyone! >>>> >>>> Since the decision was made to change only tests that fail because >>>> of conflict for now (skip "selfish" tests), I post new webrev for >>>> hotspot part of the JDK-8019361 >>>> : >>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>> >>>> Thanks, >>>> Evgeniya Stepanova >>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>> Nice plan! Please feel free to send me any feedback/questions >>>>> regarding @requires >>>>> >>>>> Thanks, >>>>> Dima >>>>> >>>>> >>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Dima, >>>>>> >>>>>> Thanks for the answers. I think the currently proposed patch is a >>>>>> good start. We will have to evolve the @requires tag in the >>>>>> future, but let's have that discussion separate from this review. >>>>>> And we can start that discussion later when we have more >>>>>> experience with the current version of @requires. >>>>>> >>>>>> Thanks for doing this! >>>>>> Bengt >>>>>> >>>>>> >>>>>> >>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> That's great that we have very closed visions! >>>>>>> >>>>>>> The general comment: currently, jtreg doesn't support any sort >>>>>>> of plugins, so you can't provide a VM specific handler of the >>>>>>> @requires or another tag. This is very annoying limitation and >>>>>>> we have to live with it. >>>>>>> >>>>>>> A few more comments inline. >>>>>>> >>>>>>> >>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi Dima, >>>>>>>> >>>>>>>> Answers inline. >>>>>>>> >>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>> Hi Bengt, >>>>>>>>> >>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it very >>>>>>>>> much! >>>>>>>>> >>>>>>>>> See comments inline. >>>>>>>>> >>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot part >>>>>>>>>>> of the JDK-8019361 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Problem: Some tests explicitly set GC and fail when jtreg >>>>>>>>>>> set another GC. >>>>>>>>>>> Solution: Such tests marked with the jtreg tag "requires" to >>>>>>>>>>> skip test if there is a conflict >>>>>>>>>> >>>>>>>>>> Thanks for fixing this! It is really great that we finally >>>>>>>>>> start sorting this out. >>>>>>>>>> >>>>>>>>>> First a general comment. The @requires tag has been developed >>>>>>>>>> without much cooperation with the GC team. We did have a lot >>>>>>>>>> of feedback when it was first presented a year ago, but it >>>>>>>>>> does not seem like this feedback was incorporated into the >>>>>>>>>> @requires that was eventually built. >>>>>>>>> >>>>>>>>> We tried to implement as much developer's wishes as possible. >>>>>>>>> But not everything is possible, sorry for that. >>>>>>>> >>>>>>>> Yes, I'm sure you have done your best. It's just that we have >>>>>>>> been requesting this feature for 3 years and I was expecting us >>>>>>>> to be able to influence the feature much more than was the case >>>>>>>> now. >>>>>>> >>>>>>> My personal hope: @requires will address ~90% of existing issues. >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>> forward and I won't object to it. But I am pretty convinced >>>>>>>>>> that we will soon run in to the limitations of the current >>>>>>>>>> @requires implementation and we will have to redo this work. >>>>>>>>>> >>>>>>>>>> Some of the points I don't really like about the @requires >>>>>>>>>> tag are: >>>>>>>>>> >>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. It >>>>>>>>>> would have been better to just "require" any command line flag. >>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also possible >>>>>>>>> to use: >>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>> >>>>>>>>> The table with all vars available in jtreg: >>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>> >>>>>>>> The problem with having this matching built in to JTreg is that >>>>>>>> it makes it very hard to change. When we discussed this a year >>>>>>>> ago I think we said that JTreg should only provide a means to >>>>>>>> test against the command line and a hook for running some java >>>>>>>> code in the @requires tag. That way we could put logic like >>>>>>>> this in a test library that is under our control. This would >>>>>>>> make it easy for us to change and also enables us to use >>>>>>>> different logic for different versions. >>>>>>> >>>>>>> I would be glad to have own harness... >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> - the requirement should be per @run tag. Right now we have >>>>>>>>>> to do what you did in this change and use vm.gc=null even >>>>>>>>>> when some tests could actually have been run when a GC was >>>>>>>>>> specified. >>>>>>>>> it would be great, but it will unlikely happen in jtreg, as >>>>>>>>> well as test case support. >>>>>>>> >>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>> >>>>>>> Under test case support I mean ability to treat each @run as a >>>>>>> separate test. Now >>>>>>> >>>>>>> @test >>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>> class MyTest { >>>>>>> } >>>>>>> >>>>>>> is always a single test. You can't exclude, or re-run a part of it. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> - there are many tests that require more than just a specific >>>>>>>>>> GC. Often there are other flags that can't be changed either >>>>>>>>>> for the test to work properly. >>>>>>>>> >>>>>>>>> yes. conflicting GC is just the most popular problem caused by >>>>>>>>> conflicting options. >>>>>>>>> If we address this issue and we are satisfied with solution, >>>>>>>>> we could move further. >>>>>>>> >>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>> Personally I would have preferred that the first step was a >>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>> implementation of the @requires tag. I just want to say that >>>>>>>>>> I'm not too happy about how the @requires tag turned out. But >>>>>>>>>> assuming we have to use it the way it is now I guess the >>>>>>>>>> proposed changeset looks good. >>>>>>>>> >>>>>>>>> yes, this thread is about change made by Evgeniya, not about >>>>>>>>> jtreg :) >>>>>>>>> And thanks for reviewing it! >>>>>>>> >>>>>>>> Agreed. And as I said, I think the patch looks ok. I have not >>>>>>>> looked at all tests. But if they now pass with the combinations >>>>>>>> that we test with I guess they should be ok. >>>>>>> >>>>>>> Excellent! Thanks a lot! >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, -XX:+UseConcMarkSweep >>>>>>>>>>> and without any GC flag). Tests are being excluded as >>>>>>>>>>> expected. No tests failed because of the conflict. >>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>> >>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>> >>>>>>>> Ok. Thanks. >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think some of the test, like >>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if you >>>>>>>>>> run with -XX:+UseParNewGC. Others, like >>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if you >>>>>>>>>> run with -XX:-UseParNewGC. Could you test these two cases too? >>>>>>>>> >>>>>>>>> These two tests ignore vm flags. >>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>> execute the tests when not needed. >>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run these >>>>>>>>> tests 4 times, 1 should be enough. >>>>>>>> >>>>>>>> Do we really want to use the @requires functionality for this >>>>>>>> purpose? It seems like a way of misusing @requires. If we just >>>>>>>> want the tests to be run once I think Leonid's approach with >>>>>>>> tests lists seems more suitable. >>>>>>> >>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>> >>>>>>> >>>>>>>> But are you sure that this is the reason for the @requires in >>>>>>>> this case? TestDefNewCMS does sound like a test that is DefNew >>>>>>>> specific. I don't see a reason to run it with ParNew. If it >>>>>>>> doesn't fail today it should probably be changed so that it >>>>>>>> does fail if it is run with the wrong GC. >>>>>>> >>>>>>> @requires - is not the silver bullet, but it's quite easy way to >>>>>>> solve a lot of issues. >>>>>>> >>>>>>> I hope, @requires will allow to reduce the number of "selfish" >>>>>>> tests, which produce a new java process to ignore vm flags >>>>>>> coming from outside. No @requires, no other mechanism could 100% >>>>>>> protect a test from running with conflicting options, but this >>>>>>> is not the goal. >>>>>>> >>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>> collector, there shouldn't mass failures caused by options >>>>>>> conflicts. But a few failures could be handled manually. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> Similarly it looks to me like there are tests that will fail >>>>>>>>>> if you run them with -XX:-UseParallelOldGC or >>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>> >>>>>>>>>> Just a heads up. These two tests will soon be removed. I'm >>>>>>>>>> about to push a changeset that removes them: >>>>>>>>>> >>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>> okay, thank for letting us know. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Is there some way of making sure that all tests are run at >>>>>>>>>> one time or another. With this change there is a risk that >>>>>>>>>> some tests are never run and always skipped. Will we somehow >>>>>>>>>> be tracking what gets skipped and make sure that all tests >>>>>>>>>> are at least run once with the correct GC so that it is not >>>>>>>>>> skipped all the time? >>>>>>>>> >>>>>>>>> This is a very good question! >>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will do >>>>>>>>> soon, after getting fix of: >>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>> >>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>> >>>>>>>>> Other examples: >>>>>>>>> >>>>>>>>> /* >>>>>>>>> *@ignore >>>>>>>>> *@test >>>>>>>>> */ >>>>>>>>> ... >>>>>>>>> >>>>>>>>> /*@bug 4445555 >>>>>>>>> *@test >>>>>>>>> */ >>>>>>>>> ... >>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>> only files with @test on the first place... >>>>>>>>> >>>>>>>>> So, making sure that tests do not disappear is important SQE >>>>>>>>> task, we know about that, we're thinking on solution (may be >>>>>>>>> very actively). But this subject for another discussion, not >>>>>>>>> within RFR :) >>>>>>>> >>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>> >>>>>>> I was going to say "not very actively", but never mind, we know >>>>>>> about this problem. With introducing @requires mechanism it will >>>>>>> become more important! >>>>>>> >>>>>>> >>>>>>> Thanks for your comments! >>>>>>> >>>>>>> -- Dima >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Bengt >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> /Evgeniya Stepanova/ >>> >> >> -- >> /Evgeniya Stepanova/ > -- /Evgeniya Stepanova/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.hartmann at oracle.com Thu Nov 13 14:51:34 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 13 Nov 2014 15:51:34 +0100 Subject: [9] RFR(S): 8050079: crash while compiling java.lang.ref.Finalizer::runFinalizer Message-ID: <5464C576.7040705@oracle.com> Hi, please review the following patch. Bug: https://bugs.openjdk.java.net/browse/JDK-8050079 Webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.00/ Problem (1): C1 compiles a call to Object.finalize() in 'java.lang.ref.Finalizer::runFinalizer'. To determine if the call is monomorphic, the compiler searches through the subclasses of Object until it finds a class that overrides finalize(). This check is performed in 'ClassHierarchyWalker::find_witness_anywhere' by calling 'is_witness(Klass* k)' on the subclasses. After casting with 'InstanceKlass::cast(k)' the (Sub-)Klass k is treated as an InstanceKlass and 'InstanceKlass::find_method' is called. The crash happens in 'binary_search' when a subclass k is not an InstanceKlass but an ObjectArrayKlass and therefore '_methods' is NULL. Why didn't the bug show up earlier? The method Object.finalize() is usually overridden by the subclasses 'java.lang.ClassLoader$NativeLibrary' and 'java.lang.Enum' which are loaded during startup. That means a call to Object.finalize() is never treated as monomorphic and 'ClassHierarchyWalker::find_witness_anywhere' returns before encountering an ObjectArrayKlass in the subclass hierarchy of Object. The failure only occurs in an internal test environment for JRebel [1]. According to the core file of the crash, the usual class hierarchy of the Java Runtime Environment was changed by the framework. For example, java.lang.String is no longer a subclass of java.lang.Object but of another class called 'JaveleonObject' [2]. Especially the two overriding classes 'ClassLoader$NativeLibrary' and 'Enum' are not subclasses of Object. This leads to the crash because C1 encounters an ObjectArrayKlass in the subclass hierarchy. Of course, this bug could also show up if the sequence of subclasses in the 'Klass::subklass()' list changes. Problem (2): In case of a worklist overflow we recursively call 'find_witness_anywhere' and create a new worklist (see line 1178 of dependencies.cpp). The problem is that we now execute the interface related checks that would otherwise not be executed (line 1132). For interface subtypes of Object (for example java/lang/Readable) we then bail out if 'nof_impls > 1'. Solution: (1) A check for 'oop_is_instance' is added to ignore non-instance Klasses in the subclass hierarchy. (2) I added a check to only execute the interface related checks if we come from a top level call. I was finally able to write a small jtreg regression test. Although the test is a little bit hacky, it is stable and deterministically reproduces the bug. The test works by overriding the default implementation of java.lang.Object, renaming the method 'finalize' to 'finalizeObject' and thus guaranteeing that 'finalizeObject' is never overridden and all calls to it are monomorphic. By triggering a C1 compilation of 'Object.finalizeObject()' we trigger the bug because there are no overriding instance Klasses in the call hierarchy. The '-XX:-VerifyDependencies' flag is necessary to not bail out too early because of problem (2). Testing: JPRT with new test Thanks, Tobias [1] http://zeroturnaround.com/software/jrebel/ [2] https://bugs.openjdk.java.net/secure/attachment/23360/replay.log From dmitry.fazunenko at oracle.com Thu Nov 13 13:57:20 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Thu, 13 Nov 2014 17:57:20 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B946.90806@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> Message-ID: <5464B8C0.6050300@oracle.com> On 13.11.2014 17:59, Bengt Rutisson wrote: > > On 2014-11-13 13:56, Dmitry Fazunenko wrote: >> >> On 13.11.2014 17:42, Bengt Rutisson wrote: >>> >>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>> >>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>> >>>>> Hi Evgeniya, >>>>> >>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>> Hi Dmitry, >>>>>> >>>>>> You are right - I've forgotten about copyrights >>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>> >>>>> >>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would >>>>> be good to add -XX:+UseG1GC to the @run tags and then use >>>>> @requires vm.gc=="G1" | vm.gc == null. >>>>> >>>>> >>>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated >>>>> to the conflicting GC combinations. Should that really be part of >>>>> this changeset? >>>>> >>>>> >>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >>>>> really need @requires for them? >>>> >>>> Yes, we do. >>>> These tests use TestShrinkAuxiliaryData class which implements its >>>> own mechanism to analyze VM options an skip if not applicable >>>> collector is given. @requires - allows to rely on jtreg. >>>> >>>> Driver mode is a kind of indicator, that the test will spawn its >>>> own java process. >>> >>> I thought the point of @driver was that no external vmoptions were >>> passed to such a test. Is that not the case? >> >> In the driver mode VM is started without external VM flags. Those >> flags are passed to the tests via system property. >> The driver mode is a sort of shell to start something else. > > Right. So, why would you need @requires on the > TestShrinkAuxiliaryDataXX tests because the utility > TestShrinkAuxiliaryData picks up the vm flags through > Utils.getTestJavaOpts(). What's the point in running this in a driver > mode when they anyway pick up the vm flags? TestShrinkAuxiliaryData implemented a workaround awaiting for @requires to appear in jtreg. Frankly speaking, the driver mode doesn't bring a lot of value, it's rather confusing and obligate developers to be more careful. If a class just spawns another java process with a real test, it's a big deal to run this class with or without external options. But there is no guarantee, that people will not start run real tests in driver mode... > > I'm asking because adding @requires to these tests means that we will > run them less often than we do now. So, I'm a bit worried that we > reduce the amount of testing we do. Don't worry about it. We want to run more tests, believe me. -- Dima > > Bengt > >> >> -- Dima >> >> >>> >>> Bengt >>> >>>> >>>> Thanks >>>> Dima >>>> >>>>> >>>>> >>>>> Otherwise it look ok to me. >>>>> >>>>> Bengt >>>>> >>>>> >>>>>> >>>>>> Thanks >>>>>> Evgeniya Stepanova >>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> The fix looks good to me. >>>>>>> >>>>>>> I noticed the following minor things: >>>>>>> - copyrights need to include the year of last change >>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among >>>>>>> updated files, but doesn't contain any changes >>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>>> variable 'prohibitedVmOptions' >>>>>>> >>>>>>> Thanks, >>>>>>> Dima >>>>>>> >>>>>>> >>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>> Hi everyone! >>>>>>>> >>>>>>>> Since the decision was made to change only tests that fail >>>>>>>> because of conflict for now (skip "selfish" tests), I post new >>>>>>>> webrev for hotspot part of the JDK-8019361 >>>>>>>> : >>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Evgeniya Stepanova >>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>>>>> regarding @requires >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Dima, >>>>>>>>>> >>>>>>>>>> Thanks for the answers. I think the currently proposed patch >>>>>>>>>> is a good start. We will have to evolve the @requires tag in >>>>>>>>>> the future, but let's have that discussion separate from this >>>>>>>>>> review. And we can start that discussion later when we have >>>>>>>>>> more experience with the current version of @requires. >>>>>>>>>> >>>>>>>>>> Thanks for doing this! >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>> >>>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>>> sort of plugins, so you can't provide a VM specific handler >>>>>>>>>>> of the @requires or another tag. This is very annoying >>>>>>>>>>> limitation and we have to live with it. >>>>>>>>>>> >>>>>>>>>>> A few more comments inline. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Dima, >>>>>>>>>>>> >>>>>>>>>>>> Answers inline. >>>>>>>>>>>> >>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>>>>> very much! >>>>>>>>>>>>> >>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>> >>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>> fix: http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>> jtreg set another GC. >>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>> "requires" to skip test if there is a conflict >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for fixing this! It is really great that we >>>>>>>>>>>>>> finally start sorting this out. >>>>>>>>>>>>>> >>>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>>> developed without much cooperation with the GC team. We >>>>>>>>>>>>>> did have a lot of feedback when it was first presented a >>>>>>>>>>>>>> year ago, but it does not seem like this feedback was >>>>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>>>> >>>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>>> >>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>>> expecting us to be able to influence the feature much more >>>>>>>>>>>> than was the case now. >>>>>>>>>>> >>>>>>>>>>> My personal hope: @requires will address ~90% of existing >>>>>>>>>>> issues. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>>>>> forward and I won't object to it. But I am pretty >>>>>>>>>>>>>> convinced that we will soon run in to the limitations of >>>>>>>>>>>>>> the current @requires implementation and we will have to >>>>>>>>>>>>>> redo this work. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Some of the points I don't really like about the >>>>>>>>>>>>>> @requires tag are: >>>>>>>>>>>>>> >>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. >>>>>>>>>>>>>> It would have been better to just "require" any command >>>>>>>>>>>>>> line flag. >>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>>> possible to use: >>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>> >>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>> >>>>>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>>>>> that it makes it very hard to change. When we discussed >>>>>>>>>>>> this a year ago I think we said that JTreg should only >>>>>>>>>>>> provide a means to test against the command line and a hook >>>>>>>>>>>> for running some java code in the @requires tag. That way >>>>>>>>>>>> we could put logic like this in a test library that is >>>>>>>>>>>> under our control. This would make it easy for us to change >>>>>>>>>>>> and also enables us to use different logic for different >>>>>>>>>>>> versions. >>>>>>>>>>> >>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>>> have to do what you did in this change and use vm.gc=null >>>>>>>>>>>>>> even when some tests could actually have been run when a >>>>>>>>>>>>>> GC was specified. >>>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, >>>>>>>>>>>>> as well as test case support. >>>>>>>>>>>> >>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>> >>>>>>>>>>> Under test case support I mean ability to treat each @run as >>>>>>>>>>> a separate test. Now >>>>>>>>>>> >>>>>>>>>>> @test >>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>> class MyTest { >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> is always a single test. You can't exclude, or re-run a part >>>>>>>>>>> of it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>>> >>>>>>>>>>>>> yes. conflicting GC is just the most popular problem >>>>>>>>>>>>> caused by conflicting options. >>>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>>> solution, we could move further. >>>>>>>>>>>> >>>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>>> Personally I would have preferred that the first step was a >>>>>>>>>>>> "just run the command line as specified in the @run tag" step. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>>>>> implementation of the @requires tag. I just want to say >>>>>>>>>>>>>> that I'm not too happy about how the @requires tag turned >>>>>>>>>>>>>> out. But assuming we have to use it the way it is now I >>>>>>>>>>>>>> guess the proposed changeset looks good. >>>>>>>>>>>>> >>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>>> about jtreg :) >>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>> >>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have >>>>>>>>>>>> not looked at all tests. But if they now pass with the >>>>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>>>> >>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests >>>>>>>>>>>>>>> are being excluded as expected. No tests failed because >>>>>>>>>>>>>>> of the conflict. >>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>> >>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will submit) >>>>>>>>>>>> >>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail if >>>>>>>>>>>>>> you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail if >>>>>>>>>>>>>> you run with -XX:-UseParNewGC. Could you test these two >>>>>>>>>>>>>> cases too? >>>>>>>>>>>>> >>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>> Add @requires here is not necessary, but it will allow not >>>>>>>>>>>>> execute the tests when not needed. >>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>>> >>>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>>> this purpose? It seems like a way of misusing @requires. If >>>>>>>>>>>> we just want the tests to be run once I think Leonid's >>>>>>>>>>>> approach with tests lists seems more suitable. >>>>>>>>>>> >>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> But are you sure that this is the reason for the @requires >>>>>>>>>>>> in this case? TestDefNewCMS does sound like a test that is >>>>>>>>>>>> DefNew specific. I don't see a reason to run it with >>>>>>>>>>>> ParNew. If it doesn't fail today it should probably be >>>>>>>>>>>> changed so that it does fail if it is run with the wrong GC. >>>>>>>>>>> >>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy >>>>>>>>>>> way to solve a lot of issues. >>>>>>>>>>> >>>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>>> "selfish" tests, which produce a new java process to ignore >>>>>>>>>>> vm flags coming from outside. No @requires, no other >>>>>>>>>>> mechanism could 100% protect a test from running with >>>>>>>>>>> conflicting options, but this is not the goal. >>>>>>>>>>> >>>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. >>>>>>>>>>>>>> I'm about to push a changeset that removes them: >>>>>>>>>>>>>> >>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Is there some way of making sure that all tests are run >>>>>>>>>>>>>> at one time or another. With this change there is a risk >>>>>>>>>>>>>> that some tests are never run and always skipped. Will we >>>>>>>>>>>>>> somehow be tracking what gets skipped and make sure that >>>>>>>>>>>>>> all tests are at least run once with the correct GC so >>>>>>>>>>>>>> that it is not skipped all the time? >>>>>>>>>>>>> >>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will >>>>>>>>>>>>> do soon, after getting fix of: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>> >>>>>>>>>>>>> And yes, tracking tests which are not run is important thing. >>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>> >>>>>>>>>>>>> Other examples: >>>>>>>>>>>>> >>>>>>>>>>>>> /* >>>>>>>>>>>>> *@ignore >>>>>>>>>>>>> *@test >>>>>>>>>>>>> */ >>>>>>>>>>>>> ... >>>>>>>>>>>>> >>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>> *@test >>>>>>>>>>>>> */ >>>>>>>>>>>>> ... >>>>>>>>>>>>> Such tests will never be run, because jtreg treats as test >>>>>>>>>>>>> only files with @test on the first place... >>>>>>>>>>>>> >>>>>>>>>>>>> So, making sure that tests do not disappear is important >>>>>>>>>>>>> SQE task, we know about that, we're thinking on solution >>>>>>>>>>>>> (may be very actively). But this subject for another >>>>>>>>>>>>> discussion, not within RFR :) >>>>>>>>>>>> >>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>> >>>>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>>>> know about this problem. With introducing @requires >>>>>>>>>>> mechanism it will become more important! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks for your comments! >>>>>>>>>>> >>>>>>>>>>> -- Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Dima >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>> >>>>>> -- >>>>>> /Evgeniya Stepanova/ >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.noll at oracle.com Thu Nov 13 14:58:57 2014 From: albert.noll at oracle.com (Albert Noll) Date: Thu, 13 Nov 2014 15:58:57 +0100 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out In-Reply-To: <5464BC2A.4020206@oracle.com> References: <5464BC2A.4020206@oracle.com> Message-ID: <5464C731.4090109@oracle.com> Hi Nils, CompileQueue::lock() always returns MethodCompileQueue_lock ( see init_compiler_sweeper_threads() ). It seems that your changes don't change existing behavior, or am I missing something? Note that we have 2 compilation queues, but only 1 lock. On 11/13/2014 03:11 PM, Nils Eliasson wrote: > Hi, > > Please review this small change. > > 1) Fixing a deadlock in diagnostic command dcmd print_compile_queues - > between the CompileQueue_lock and safepointing. The CompileQueue_lock > requires that we are not at a safepoint when taking it. Otherwise a > java-thread that already holds the lock will block when trying to get > another lock. In this specific case the compiler threads are Java > threads, they take the CompileQueue_lock frequently and some time > takes the CompileTaskAlloc_lock after that. > > Fixing this by making the diagnostic command not request a safepoint, > > 2) Refactoring away the lock() accessor in CompileQueue and stating > the lock explicitly. This removes an element of surprise and makes it > easier understanding the code. What element of surprise are you talking about? Personally, I prefer the current design, i.e., getting the lock by calling a function instead of referencing a global variable directly. If we ever decide to go for a 2 compile queue locks (I think that makes sense since we have two compilation queues), we would have to introduce the functions again. Best, Albert > webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ > bug: https://bugs.openjdk.java.net/browse/JDK-8061256 > > Thanks, > Nils Eliasson > From albert.noll at oracle.com Thu Nov 13 15:36:35 2014 From: albert.noll at oracle.com (Albert Noll) Date: Thu, 13 Nov 2014 16:36:35 +0100 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out In-Reply-To: <5464C731.4090109@oracle.com> References: <5464BC2A.4020206@oracle.com> <5464C731.4090109@oracle.com> Message-ID: <5464D003.8050907@oracle.com> Hi Nils, On 11/13/2014 03:58 PM, Albert Noll wrote: > Hi Nils, > > CompileQueue::lock() always returns MethodCompileQueue_lock ( see > init_compiler_sweeper_threads() ). It seems that your changes don't > change existing behavior, or am I missing something? Note that we have > 2 compilation queues, but only 1 lock. > It seems that I missed the most important parts of you change indeed. Is it right that Mode evaluation_mode() const { return _no_safepoint; } makes the VM operation not execute at a safepoint? I have a question nevertheless. Why did you choose to return '_no_safepoint' and not '_concurrent'? Best, Albert > On 11/13/2014 03:11 PM, Nils Eliasson wrote: >> Hi, >> >> Please review this small change. >> >> 1) Fixing a deadlock in diagnostic command dcmd print_compile_queues >> - between the CompileQueue_lock and safepointing. The >> CompileQueue_lock requires that we are not at a safepoint when taking >> it. Otherwise a java-thread that already holds the lock will block >> when trying to get another lock. In this specific case the compiler >> threads are Java threads, they take the CompileQueue_lock frequently >> and some time takes the CompileTaskAlloc_lock after that. >> >> Fixing this by making the diagnostic command not request a safepoint, >> >> 2) Refactoring away the lock() accessor in CompileQueue and stating >> the lock explicitly. This removes an element of surprise and makes it >> easier understanding the code. > What element of surprise are you talking about? Personally, I prefer > the current design, i.e., getting the lock by calling a function > instead of referencing a global variable directly. If we ever decide > to go for a 2 compile queue locks (I think that makes sense since we > have two compilation queues), we would have to introduce the functions > again. > > Best, > Albert > > >> webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8061256 >> >> Thanks, >> Nils Eliasson >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nils.eliasson at oracle.com Thu Nov 13 15:39:15 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Thu, 13 Nov 2014 16:39:15 +0100 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out In-Reply-To: <5464C731.4090109@oracle.com> References: <5464BC2A.4020206@oracle.com> <5464C731.4090109@oracle.com> Message-ID: <5464D0A3.2020908@oracle.com> Hi Albert, Yes exactly. When there is only one lock, there is no good reason for adding it to both queues. (That only make sense with separate locks.) It is much easier to find all usages of MethodCompileQueue_lock with the patch. Regards, //Nils On 2014-11-13 15:58, Albert Noll wrote: > Hi Nils, > > CompileQueue::lock() always returns MethodCompileQueue_lock ( see > init_compiler_sweeper_threads() ). It seems that your changes don't > change existing behavior, or am I missing something? Note that we have > 2 compilation queues, but only 1 lock. > > On 11/13/2014 03:11 PM, Nils Eliasson wrote: >> Hi, >> >> Please review this small change. >> >> 1) Fixing a deadlock in diagnostic command dcmd print_compile_queues >> - between the CompileQueue_lock and safepointing. The >> CompileQueue_lock requires that we are not at a safepoint when taking >> it. Otherwise a java-thread that already holds the lock will block >> when trying to get another lock. In this specific case the compiler >> threads are Java threads, they take the CompileQueue_lock frequently >> and some time takes the CompileTaskAlloc_lock after that. >> >> Fixing this by making the diagnostic command not request a safepoint, >> >> 2) Refactoring away the lock() accessor in CompileQueue and stating >> the lock explicitly. This removes an element of surprise and makes it >> easier understanding the code. > What element of surprise are you talking about? Personally, I prefer > the current design, i.e., getting the lock by calling a function > instead of referencing a global variable directly. If we ever decide > to go for a 2 compile queue locks (I think that makes sense since we > have two compilation queues), we would have to introduce the functions > again. > > Best, > Albert > > >> webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8061256 >> >> Thanks, >> Nils Eliasson >> > From albert.noll at oracle.com Thu Nov 13 15:50:09 2014 From: albert.noll at oracle.com (Albert Noll) Date: Thu, 13 Nov 2014 16:50:09 +0100 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out In-Reply-To: <5464D0A3.2020908@oracle.com> References: <5464BC2A.4020206@oracle.com> <5464C731.4090109@oracle.com> <5464D0A3.2020908@oracle.com> Message-ID: <5464D331.7000805@oracle.com> Hi Nils, On 11/13/2014 04:39 PM, Nils Eliasson wrote: > Hi Albert, > > Yes exactly. When there is only one lock, there is no good reason for > adding it to both queues. (That only make sense with separate locks.) > It is much easier to find all usages of MethodCompileQueue_lock with > the patch. It is a property of the compile queue to have a lock. As a result, it makes sense for the compile queue to keep a reference to that lock. I would argue that this keeps the code more maintainable compared to referencing a global variable directly. However, I don't have a strong opinion here. In any case, if you decide to go for this refactoring, you should also adapt the constructor of CompileQueue(const char* name, Monitor* lock). Best, Albert > > Regards, > //Nils > > On 2014-11-13 15:58, Albert Noll wrote: >> Hi Nils, >> >> CompileQueue::lock() always returns MethodCompileQueue_lock ( see >> init_compiler_sweeper_threads() ). It seems that your changes don't >> change existing behavior, or am I missing something? Note that we >> have 2 compilation queues, but only 1 lock. >> >> On 11/13/2014 03:11 PM, Nils Eliasson wrote: >>> Hi, >>> >>> Please review this small change. >>> >>> 1) Fixing a deadlock in diagnostic command dcmd print_compile_queues >>> - between the CompileQueue_lock and safepointing. The >>> CompileQueue_lock requires that we are not at a safepoint when >>> taking it. Otherwise a java-thread that already holds the lock will >>> block when trying to get another lock. In this specific case the >>> compiler threads are Java threads, they take the CompileQueue_lock >>> frequently and some time takes the CompileTaskAlloc_lock after that. >>> >>> Fixing this by making the diagnostic command not request a safepoint, >>> >>> 2) Refactoring away the lock() accessor in CompileQueue and stating >>> the lock explicitly. This removes an element of surprise and makes >>> it easier understanding the code. >> What element of surprise are you talking about? Personally, I prefer >> the current design, i.e., getting the lock by calling a function >> instead of referencing a global variable directly. If we ever decide >> to go for a 2 compile queue locks (I think that makes sense since we >> have two compilation queues), we would have to introduce the >> functions again. >> >> Best, >> Albert >> >> >>> webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8061256 >>> >>> Thanks, >>> Nils Eliasson >>> >> > From vladimir.kozlov at oracle.com Thu Nov 13 16:26:22 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Nov 2014 08:26:22 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> <545BF25E.7010505@oracle.com> <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> <545D5103.9020309@oracle.com> <5463FDD0.4090102@oracle.com> Message-ID: <5464DBAE.2000607@oracle.com> On 11/13/14 1:09 AM, Roland Westrelin wrote: > >> You don't need leftover changes in opaquenode.* files anymore. > > I?ll remove the changes. > >> jetstream regression may indicate the increase in compilation times. I would suggest to have n->Opcode() == Op_Opaque1 checks in both cases to target only it. Sorry, in my previous mail I incorrectly used Op_CastII in the example code. > > > Are we guaranteed that the type of the limit won?t improve after the opaque node is removed? Maybe it?s some arithmetic computation that isn?t optimized yet? In that case, we want to re-process the phi even after the opaque node is removed as soon as the limit changes. My main concern was regressions in your run. Which I think is increase of compilation time. If you can show that time (-XX:+CITime) does not increase much I am fine with current change. And I still want you to run this in aurora.se - I don't want new performance bugs filed 2 months from now. > > Can I go with a single reviewer for this change? Ask Igor for second review. Thanks, Vladimir > > Roland. > >> >> Thanks, >> Vladimir >> >> On 11/12/14 2:16 AM, Roland Westrelin wrote: >>> Vladimir, >>> >>> New webrev with your suggested change: >>> >>> http://cr.openjdk.java.net/~roland/8054478/webrev.02/ >>> >>> I also had to make BoolTest::dump_on available in product builds for that code: >>> >>> 138 } else { >>> 139 stringStream ss; >>> 140 test.dump_on(&ss); >>> 141 fatal(err_msg_res("unexpected comparison %s", ss.as_string())); >>> 142 } >>> >>> in castnode.cpp >>> >>> I did a refworkload run on x64: >>> >>> ============================================================================ >>> tbase: reference_server >>> Benchmark Samples Mean Stdev >>> jetstream 10 102.74 0.10 >>> Write 10 202.90 0.94 >>> Parse 10 37.10 0.05 >>> Read 10 23.20 0.13 >>> rw.runtime 10 10.60 0.05 >>> Copy 10 641.70 0.10 >>> scimark 10 1600.34 0.02 >>> LU 10 3776.61 0.00 >>> FFT 10 405.76 0.02 >>> Monte 10 775.88 0.00 >>> SOR 10 1275.16 0.01 >>> Sparse 10 1768.28 0.08 >>> rw.runtime 10 27.20 0.02 >>> specjbb2000 10 487411.56 0.04 >>> Last_Warehouse 10 487411.58 0.04 >>> First_Warehouse 10 95725.01 0.02 >>> rw.runtime 10 603.90 0.00 >>> specjbb2005 10 480939.78 0.01 >>> last 10 480939.79 0.01 >>> interval_average 10 5937.40 0.01 >>> peak 10 534158.07 0.02 >>> overall_average 10 431366.64 0.02 >>> last_warehouse 10 8.00 0.00 >>> peak_warehouse 10 2.30 0.21 >>> first 10 50951.62 0.03 >>> rw.runtime 10 461.60 0.00 >>> specjvm98 10 801.92 0.04 >>> compress 10 604.68 0.10 >>> javac 10 405.68 0.17 >>> db 10 449.78 0.00 >>> jack 10 631.14 0.05 >>> mtrt 10 2342.41 0.03 >>> jess 10 957.35 0.02 >>> rw.runtime 10 41.10 0.04 >>> mpegaudio 10 1385.76 0.00 >>> volano25 10 327418.69 0.06 >>> time 10 2.45 0.05 >>> connections 10 400.00 0.00 >>> rw.runtime 10 26.10 0.01 >>> -------------------------------------------------------------------------- >>> Weighted Geomean 31199.92 >>> ============================================================================ >>> tnew: reference_server >>> Benchmark Samples Mean Stdev %Diff P Significant >>> jetstream 10 85.82 0.12 -16.47 0.002 Yes >>> Write 10 219.60 0.31 -8.23 0.799 * >>> Parse 10 36.90 0.06 0.54 0.836 * >>> Read 10 23.60 0.10 -1.72 0.741 * >>> rw.runtime 10 13.80 0.03 -30.19 0.000 Yes >>> Copy 10 1058.70 0.26 -64.98 0.001 Yes >>> scimark 10 1583.22 0.00 -1.07 0.110 * >>> LU 10 3778.49 0.00 0.05 0.160 * >>> FFT 10 407.80 0.01 0.50 0.498 * >>> Monte 10 775.71 0.00 -0.02 0.279 * >>> SOR 10 1276.75 0.01 0.12 0.742 * >>> Sparse 10 1677.37 0.01 -5.14 0.086 * >>> rw.runtime 10 27.50 0.02 -1.10 0.265 * >>> specjbb2000 10 491672.03 0.04 0.87 0.624 * >>> Last_Warehouse 10 491672.04 0.04 0.87 0.624 * >>> First_Warehouse 10 94172.03 0.02 -1.62 0.050 * >>> rw.runtime 10 604.00 0.00 -0.02 0.585 * >>> specjbb2005 10 481051.28 0.01 0.02 0.945 * >>> last 10 481051.29 0.01 0.02 0.945 * >>> interval_average 10 5938.90 0.01 0.03 0.940 * >>> peak 10 538706.19 0.03 0.85 0.461 * >>> overall_average 10 434244.96 0.02 0.67 0.535 * >>> last_warehouse 10 8.00 0.00 -0.00 0.000 * >>> peak_warehouse 10 2.00 0.00 13.04 0.081 * >>> first 10 51039.34 0.03 0.17 0.889 * >>> rw.runtime 10 462.20 0.00 -0.13 0.120 * >>> specjvm98 10 806.31 0.04 0.55 0.738 * >>> compress 10 623.61 0.10 3.13 0.494 * >>> javac 10 402.37 0.10 -0.81 0.898 * >>> db 10 450.58 0.00 0.18 0.327 * >>> jack 10 627.02 0.05 -0.65 0.785 * >>> mtrt 10 2281.86 0.03 -2.58 0.106 * >>> jess 10 1000.65 0.10 4.52 0.220 * >>> rw.runtime 10 40.90 0.03 0.49 0.761 * >>> mpegaudio 10 1384.19 0.00 -0.11 0.514 * >>> volano25 10 324906.00 0.08 -0.77 0.799 * >>> time 10 2.47 0.07 -1.00 0.733 * >>> connections 10 400.00 0.00 0.00 0.000 * >>> rw.runtime 10 26.50 0.02 -1.53 0.058 * >>> -------------------------------------------------------------------------- >>> Weighted Geomean 30613.61 -1.88 >>> ============================================================================ >>> >>> tbase is current hotspot-comp. tnew is with the change. It?s ok, right? >>> >>> Roland. >>> >>> >>>> On Nov 8, 2014, at 12:08 AM, Vladimir Kozlov wrote: >>>> >>>> This looks good but I just realized that what you are doing in Opaque1Node::Ideal() we usually do in PhaseIterGVN::add_users_to_worklist(). >>>> >>>> For example you can do there (for Phi): >>>> >>>> + uint use_op = use->Opcode(); >>>> if( use->is_Cmp() ) { // Enable CMP/BOOL optimization >>>> add_users_to_worklist(use); // Put Bool on worklist >>>> - // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >>>> - // phi merging either 0 or 1 onto the worklist >>>> if (use->outcnt() > 0) { >>>> Node* bol = use->raw_out(0); >>>> if (bol->outcnt() > 0) { >>>> Node* iff = bol->raw_out(0); >>>> if (use_op == Op_CmpI && iff->is_CountedLoopEnd() && >>>> n->Opcode() == Op_CastII) { >>>> + // If this opaque node feeds into the limit condition of a >>>> + // CountedLoop, we need to process the Phi node for the >>>> + // induction variable: the range of values taken by the Phi is >>>> + // known now and so its type is also known. >>>> + CountedLoopEndNode* cle = iff->as_CountedLoopEnd(); >>>> + if (cle->limit() == this) { >>>> + _worklist.push(cle->phi()); >>>> + } >>>> - if (iff->outcnt() == 2) { >>>> + } else if (iff->outcnt() == 2) { >>>> + // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >>>> + // phi merging either 0 or 1 onto the worklist >>>> >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> >>>> On 11/7/14 2:17 PM, Roland Westrelin wrote: >>>>> Vladimir, >>>>> >>>>> Thanks for the discussion, suggestions and fixes. Here is an updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~roland/8054478/webrev.01/ >>>>> >>>>> Roland. >>>>> >>>>>> On Nov 6, 2014, at 11:12 PM, Vladimir Kozlov wrote: >>>>>> >>>>>> On 11/6/14 7:39 AM, Roland Westrelin wrote: >>>>>>> >>>>>>>> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. >>>>>>> >>>>>>> I?ll send an updated webrev. >>>>>>> >>>>>>>>>>>> opaquenode.cpp >>>>>>>>>>>> >>>>>>>>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>>>>>>>> >>>>>>>>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>>>>>>>> >>>>>>>>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>>>>>>>> >>>>>>>>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >>>>>>>> >>>>>>>> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. >>>>>>> >>>>>>> (I know you don?t like when I remove the text from previous emails but that was really getting confusing) >>>>>> >>>>>> At least leave webrev link so I don't need to search for it in previous mails. >>>>>> >>>>>>> >>>>>>> PhiNode::Value() has this code: >>>>>>> >>>>>>> // Check for trip-counted loop. If so, be smarter. >>>>>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>>>>> if( l && l->can_be_counted_loop(phase) && >>>>>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>>>>> // protect against init_trip() or limit() returning NULL >>>>>>> const Node *init = l->init_trip(); >>>>>>> const Node *limit = l->limit(); >>>>>>> if( init != NULL && limit != NULL && l->stride_is_con() ) { >>>>>>> const TypeInt *lo = init ->bottom_type()->isa_int(); >>>>>>> const TypeInt *hi = limit->bottom_type()->isa_int(); >>>>>>> if( lo && hi ) { // Dying loops might have TOP here >>>>>>> int stride = l->stride_con(); >>>>>>> if( stride < 0 ) { // Down-counter loop >>>>>>> const TypeInt *tmp = lo; lo = hi; hi = tmp; >>>>>>> stride = -stride; >>>>>>> } >>>>>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>>>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>>>>> } >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. >>>>>> >>>>>> My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: >>>>>> >>>>>> // Step B2: Build a zero-trip guard for the main-loop. After leaving the >>>>>> // pre-loop, the main-loop may not execute at all. Later in life this >>>>>> // zero-trip guard will become the minimum-trip guard when we unroll >>>>>> // the main-loop. >>>>>> Node *min_opaq = new Opaque1Node(C, limit); >>>>>> Node *min_cmp = new CmpINode( pre_incr, min_opaq ); >>>>>> >>>>>> >>>>>> But you are talking about pre-loop exit check: >>>>>> >>>>>> // Step B4: Shorten the pre-loop to run only 1 iteration (for now). >>>>>> // RCE and alignment may change this later. >>>>>> Node *cmp_end = pre_end->cmp_node(); >>>>>> assert( cmp_end->in(2) == limit, "" ); >>>>>> Node *pre_limit = new AddINode( init, stride ); >>>>>> >>>>>> // Save the original loop limit in this Opaque1 node for >>>>>> // use by range check elimination. >>>>>> Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); >>>>>> >>>>>> >>>>>> But in this case you can add check (cl->limit() == this) to be clear what you are looking for. >>>>>> Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>>> >>>>>>> In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. >>>>>>> >>>>>>> Roland. >>>>> >>> > From zoltan.majo at oracle.com Thu Nov 13 16:29:59 2014 From: zoltan.majo at oracle.com (=?windows-1252?Q?Zolt=E1n_Maj=F3?=) Date: Thu, 13 Nov 2014 17:29:59 +0100 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups In-Reply-To: <5464BC57.80200@oracle.com> References: <5464B2FB.1050606@oracle.com> <5464BC57.80200@oracle.com> Message-ID: <5464DC87.1070301@oracle.com> Hi, On 11/13/2014 03:12 PM, Zolt?n Maj? wrote: > Here is only the "work" time for the same runs as before: > > hotspot_compiler_1 / work=15m6s > hotspot_compiler_2 / work=13m55s > hotspot_compiler_3 / work=12m43s I updated the test list. Here are the results: group / #tests / "work" time on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848 hotspot_compiler_1 / 107 / work=15m hotspot_compiler_2 / 79 / work=15m2s hotspot_compiler_3 / 62 / work=14m14s Here is the updated webrev: http://cr.openjdk.java.net/~zmajo/8062854/webrev.01/ Thank you and best regards, Zoltan From vladimir.kozlov at oracle.com Thu Nov 13 16:38:50 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Nov 2014 08:38:50 -0800 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups In-Reply-To: <5464DC87.1070301@oracle.com> References: <5464B2FB.1050606@oracle.com> <5464BC57.80200@oracle.com> <5464DC87.1070301@oracle.com> Message-ID: <5464DE9A.8090105@oracle.com> This is good approach. I see you removed all 5091921 tests when before you ran most of them. What is the reason? Can you still run some? Thanks, Vladimir On 11/13/14 8:29 AM, Zolt?n Maj? wrote: > Hi, > > > On 11/13/2014 03:12 PM, Zolt?n Maj? wrote: >> Here is only the "work" time for the same runs as before: >> >> hotspot_compiler_1 / work=15m6s >> hotspot_compiler_2 / work=13m55s >> hotspot_compiler_3 / work=12m43s > > I updated the test list. Here are the results: > > group / #tests / "work" time on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848 > > hotspot_compiler_1 / 107 / work=15m > hotspot_compiler_2 / 79 / work=15m2s > hotspot_compiler_3 / 62 / work=14m14s > > Here is the updated webrev: http://cr.openjdk.java.net/~zmajo/8062854/webrev.01/ > > Thank you and best regards, > > > Zoltan > From zoltan.majo at oracle.com Thu Nov 13 16:45:34 2014 From: zoltan.majo at oracle.com (=?windows-1252?Q?Zolt=E1n_Maj=F3?=) Date: Thu, 13 Nov 2014 17:45:34 +0100 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups In-Reply-To: <5464DE9A.8090105@oracle.com> References: <5464B2FB.1050606@oracle.com> <5464BC57.80200@oracle.com> <5464DC87.1070301@oracle.com> <5464DE9A.8090105@oracle.com> Message-ID: <5464E02E.7060406@oracle.com> Hi, On 11/13/2014 05:38 PM, Vladimir Kozlov wrote: > This is good approach. I see you removed all 5091921 tests when before > you ran most of them. What is the reason? Can you still run some? I removed 5091921 test because - 3 of them have a high execution time and - I wanted to keep the exclude list simple (not exclude many individual tests). But if you like, we can include short-running 5091921 tests again (for example into hotspot_compiler_1). That would result in around 1 min extra time and 3 tests excluded. Thank you and best regards, Zoltan > Thanks, > Vladimir > > On 11/13/14 8:29 AM, Zolt?n Maj? wrote: >> Hi, >> >> >> On 11/13/2014 03:12 PM, Zolt?n Maj? wrote: >>> Here is only the "work" time for the same runs as before: >>> >>> hotspot_compiler_1 / work=15m6s >>> hotspot_compiler_2 / work=13m55s >>> hotspot_compiler_3 / work=12m43s >> >> I updated the test list. Here are the results: >> >> group / #tests / "work" time on solaris_sparcv9 cpus=6 >> parallelcount=6 cpufreqmhz=2848 >> >> hotspot_compiler_1 / 107 / work=15m >> hotspot_compiler_2 / 79 / work=15m2s >> hotspot_compiler_3 / 62 / work=14m14s >> >> Here is the updated webrev: >> http://cr.openjdk.java.net/~zmajo/8062854/webrev.01/ >> >> Thank you and best regards, >> >> >> Zoltan >> From vladimir.kozlov at oracle.com Thu Nov 13 16:52:15 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Nov 2014 08:52:15 -0800 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups In-Reply-To: <5464E02E.7060406@oracle.com> References: <5464B2FB.1050606@oracle.com> <5464BC57.80200@oracle.com> <5464DC87.1070301@oracle.com> <5464DE9A.8090105@oracle.com> <5464E02E.7060406@oracle.com> Message-ID: <5464E1BF.5050208@oracle.com> On 11/13/14 8:45 AM, Zolt?n Maj? wrote: > Hi, > > > On 11/13/2014 05:38 PM, Vladimir Kozlov wrote: >> This is good approach. I see you removed all 5091921 tests when before you ran most of them. What is the reason? Can >> you still run some? > > I removed 5091921 test because > > - 3 of them have a high execution time and > - I wanted to keep the exclude list simple (not exclude many individual tests). > > But if you like, we can include short-running 5091921 tests again (for example into hotspot_compiler_1). That would > result in around 1 min extra time and 3 tests excluded. Yes, please do that. Thanks, Vladimir > > Thank you and best regards, > > > Zoltan > > >> Thanks, >> Vladimir >> >> On 11/13/14 8:29 AM, Zolt?n Maj? wrote: >>> Hi, >>> >>> >>> On 11/13/2014 03:12 PM, Zolt?n Maj? wrote: >>>> Here is only the "work" time for the same runs as before: >>>> >>>> hotspot_compiler_1 / work=15m6s >>>> hotspot_compiler_2 / work=13m55s >>>> hotspot_compiler_3 / work=12m43s >>> >>> I updated the test list. Here are the results: >>> >>> group / #tests / "work" time on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848 >>> >>> hotspot_compiler_1 / 107 / work=15m >>> hotspot_compiler_2 / 79 / work=15m2s >>> hotspot_compiler_3 / 62 / work=14m14s >>> >>> Here is the updated webrev: http://cr.openjdk.java.net/~zmajo/8062854/webrev.01/ >>> >>> Thank you and best regards, >>> >>> >>> Zoltan >>> > From vladimir.kozlov at oracle.com Thu Nov 13 19:48:39 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Nov 2014 11:48:39 -0800 Subject: [9] RFR(S): 8050079: crash while compiling java.lang.ref.Finalizer::runFinalizer In-Reply-To: <5464C576.7040705@oracle.com> References: <5464C576.7040705@oracle.com> Message-ID: <54650B17.1030401@oracle.com> Should you fix problem (2) other way around by adding check in the following loop? If you don't bailout on 'nof_impls > 1' on interface what result you get? I think test should go into new 'dependencies' subdir not 'inlining'. Why you fork separate JVM process? Thanks, Vladimir On 11/13/14 6:51 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8050079 > Webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.00/ > > Problem (1): > C1 compiles a call to Object.finalize() in > 'java.lang.ref.Finalizer::runFinalizer'. To determine if the call is > monomorphic, the compiler searches through the subclasses of Object until it > finds a class that overrides finalize(). This check is performed in > 'ClassHierarchyWalker::find_witness_anywhere' by calling 'is_witness(Klass* k)' > on the subclasses. After casting with 'InstanceKlass::cast(k)' the (Sub-)Klass k > is treated as an InstanceKlass and 'InstanceKlass::find_method' is called. The > crash happens in 'binary_search' when a subclass k is not an InstanceKlass but > an ObjectArrayKlass and therefore '_methods' is NULL. > > Why didn't the bug show up earlier? > The method Object.finalize() is usually overridden by the subclasses > 'java.lang.ClassLoader$NativeLibrary' and 'java.lang.Enum' which are loaded > during startup. That means a call to Object.finalize() is never treated as > monomorphic and 'ClassHierarchyWalker::find_witness_anywhere' returns before > encountering an ObjectArrayKlass in the subclass hierarchy of Object. The > failure only occurs in an internal test environment for JRebel [1]. According to > the core file of the crash, the usual class hierarchy of the Java Runtime > Environment was changed by the framework. For example, java.lang.String is no > longer a subclass of java.lang.Object but of another class called > 'JaveleonObject' [2]. Especially the two overriding classes > 'ClassLoader$NativeLibrary' and 'Enum' are not subclasses of Object. This leads > to the crash because C1 encounters an ObjectArrayKlass in the subclass hierarchy. > Of course, this bug could also show up if the sequence of subclasses in the > 'Klass::subklass()' list changes. > > Problem (2): > In case of a worklist overflow we recursively call 'find_witness_anywhere' and > create a new worklist (see line 1178 of dependencies.cpp). The problem is that > we now execute the interface related checks that would otherwise not be executed > (line 1132). For interface subtypes of Object (for example java/lang/Readable) > we then bail out if 'nof_impls > 1'. > > Solution: > (1) A check for 'oop_is_instance' is added to ignore non-instance Klasses in the > subclass hierarchy. > (2) I added a check to only execute the interface related checks if we come from > a top level call. > > I was finally able to write a small jtreg regression test. Although the test is > a little bit hacky, it is stable and deterministically reproduces the bug. The > test works by overriding the default implementation of java.lang.Object, > renaming the method 'finalize' to 'finalizeObject' and thus guaranteeing that > 'finalizeObject' is never overridden and all calls to it are monomorphic. By > triggering a C1 compilation of 'Object.finalizeObject()' we trigger the bug > because there are no overriding instance Klasses in the call hierarchy. > > The '-XX:-VerifyDependencies' flag is necessary to not bail out too early > because of problem (2). > > Testing: > JPRT with new test > > Thanks, > Tobias > > > [1] http://zeroturnaround.com/software/jrebel/ > [2] https://bugs.openjdk.java.net/secure/attachment/23360/replay.log > From zoltan.majo at oracle.com Fri Nov 14 07:44:07 2014 From: zoltan.majo at oracle.com (=?windows-1252?Q?Zolt=E1n_Maj=F3?=) Date: Fri, 14 Nov 2014 08:44:07 +0100 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups In-Reply-To: <5464E1BF.5050208@oracle.com> References: <5464B2FB.1050606@oracle.com> <5464BC57.80200@oracle.com> <5464DC87.1070301@oracle.com> <5464DE9A.8090105@oracle.com> <5464E02E.7060406@oracle.com> <5464E1BF.5050208@oracle.com> Message-ID: <5465B2C7.1010205@oracle.com> Hi Vladimir, thank you for the feedback. On 11/13/2014 05:52 PM, Vladimir Kozlov wrote: > On 11/13/14 8:45 AM, Zolt?n Maj? wrote: >> Hi, >> >> >> On 11/13/2014 05:38 PM, Vladimir Kozlov wrote: >>> This is good approach. I see you removed all 5091921 tests when >>> before you ran most of them. What is the reason? Can >>> you still run some? >> >> I removed 5091921 test because >> >> - 3 of them have a high execution time and >> - I wanted to keep the exclude list simple (not exclude many >> individual tests). >> >> But if you like, we can include short-running 5091921 tests again >> (for example into hotspot_compiler_1). That would >> result in around 1 min extra time and 3 tests excluded. > > Yes, please do that. I added the short-running 5091921 tests to hotspot_compiler_1. Here are the results (on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848): before: 107 tests / work=15m after: 121 tests / work=16m14s Here is the updated webrev: http://cr.openjdk.java.net/~zmajo/8062854/webrev.02/ Thank you and best regards, Zoltan > > Thanks, > Vladimir > >> >> Thank you and best regards, >> >> >> Zoltan >> >> >>> Thanks, >>> Vladimir >>> >>> On 11/13/14 8:29 AM, Zolt?n Maj? wrote: >>>> Hi, >>>> >>>> >>>> On 11/13/2014 03:12 PM, Zolt?n Maj? wrote: >>>>> Here is only the "work" time for the same runs as before: >>>>> >>>>> hotspot_compiler_1 / work=15m6s >>>>> hotspot_compiler_2 / work=13m55s >>>>> hotspot_compiler_3 / work=12m43s >>>> >>>> I updated the test list. Here are the results: >>>> >>>> group / #tests / "work" time on solaris_sparcv9 cpus=6 >>>> parallelcount=6 cpufreqmhz=2848 >>>> >>>> hotspot_compiler_1 / 107 / work=15m >>>> hotspot_compiler_2 / 79 / work=15m2s >>>> hotspot_compiler_3 / 62 / work=14m14s >>>> >>>> Here is the updated webrev: >>>> http://cr.openjdk.java.net/~zmajo/8062854/webrev.01/ >>>> >>>> Thank you and best regards, >>>> >>>> >>>> Zoltan >>>> >> From bengt.rutisson at oracle.com Fri Nov 14 09:16:36 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 10:16:36 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5464B8C0.6050300@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> <5464B8C0.6050300@oracle.com> Message-ID: <5465C874.10504@oracle.com> On 2014-11-13 14:57, Dmitry Fazunenko wrote: > > On 13.11.2014 17:59, Bengt Rutisson wrote: >> >> On 2014-11-13 13:56, Dmitry Fazunenko wrote: >>> >>> On 13.11.2014 17:42, Bengt Rutisson wrote: >>>> >>>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>>> >>>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Evgeniya, >>>>>> >>>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>>> Hi Dmitry, >>>>>>> >>>>>>> You are right - I've forgotten about copyrights >>>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>>> >>>>>> >>>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it would >>>>>> be good to add -XX:+UseG1GC to the @run tags and then use >>>>>> @requires vm.gc=="G1" | vm.gc == null. >>>>>> >>>>>> >>>>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated >>>>>> to the conflicting GC combinations. Should that really be part of >>>>>> this changeset? >>>>>> >>>>>> >>>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do we >>>>>> really need @requires for them? >>>>> >>>>> Yes, we do. >>>>> These tests use TestShrinkAuxiliaryData class which implements its >>>>> own mechanism to analyze VM options an skip if not applicable >>>>> collector is given. @requires - allows to rely on jtreg. >>>>> >>>>> Driver mode is a kind of indicator, that the test will spawn its >>>>> own java process. >>>> >>>> I thought the point of @driver was that no external vmoptions were >>>> passed to such a test. Is that not the case? >>> >>> In the driver mode VM is started without external VM flags. Those >>> flags are passed to the tests via system property. >>> The driver mode is a sort of shell to start something else. >> >> Right. So, why would you need @requires on the >> TestShrinkAuxiliaryDataXX tests because the utility >> TestShrinkAuxiliaryData picks up the vm flags through >> Utils.getTestJavaOpts(). What's the point in running this in a driver >> mode when they anyway pick up the vm flags? > > TestShrinkAuxiliaryData implemented a workaround awaiting for > @requires to appear in jtreg. > > Frankly speaking, the driver mode doesn't bring a lot of value, it's > rather confusing and obligate developers to be more careful. If a > class just spawns another java process with a real test, it's a big > deal to run this class with or without external options. But there is > no guarantee, that people will not start run real tests in driver mode... Ok. So, do we want to keep "driver" for this test or not? > >> >> I'm asking because adding @requires to these tests means that we will >> run them less often than we do now. So, I'm a bit worried that we >> reduce the amount of testing we do. > > Don't worry about it. > We want to run more tests, believe me. Sure, but adding @requires means that we run the test less often. The TestShrinkAuxiliaryData tests were for example run every week in PIT testing but with the @requires tag they will only be run every 4th week since PIT testing is rotating which GC it runs tests with. Bengt > > -- Dima > > >> >> Bengt >> >>> >>> -- Dima >>> >>> >>>> >>>> Bengt >>>> >>>>> >>>>> Thanks >>>>> Dima >>>>> >>>>>> >>>>>> >>>>>> Otherwise it look ok to me. >>>>>> >>>>>> Bengt >>>>>> >>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> Evgeniya Stepanova >>>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> The fix looks good to me. >>>>>>>> >>>>>>>> I noticed the following minor things: >>>>>>>> - copyrights need to include the year of last change >>>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among >>>>>>>> updated files, but doesn't contain any changes >>>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>>>> variable 'prohibitedVmOptions' >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Dima >>>>>>>> >>>>>>>> >>>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>>> Hi everyone! >>>>>>>>> >>>>>>>>> Since the decision was made to change only tests that fail >>>>>>>>> because of conflict for now (skip "selfish" tests), I post new >>>>>>>>> webrev for hotspot part of the JDK-8019361 >>>>>>>>> : >>>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Evgeniya Stepanova >>>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>>> Nice plan! Please feel free to send me any feedback/questions >>>>>>>>>> regarding @requires >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Dima, >>>>>>>>>>> >>>>>>>>>>> Thanks for the answers. I think the currently proposed patch >>>>>>>>>>> is a good start. We will have to evolve the @requires tag in >>>>>>>>>>> the future, but let's have that discussion separate from >>>>>>>>>>> this review. And we can start that discussion later when we >>>>>>>>>>> have more experience with the current version of @requires. >>>>>>>>>>> >>>>>>>>>>> Thanks for doing this! >>>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>> >>>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>>> >>>>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>>>> sort of plugins, so you can't provide a VM specific handler >>>>>>>>>>>> of the @requires or another tag. This is very annoying >>>>>>>>>>>> limitation and we have to live with it. >>>>>>>>>>>> >>>>>>>>>>>> A few more comments inline. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>> >>>>>>>>>>>>> Answers inline. >>>>>>>>>>>>> >>>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate it >>>>>>>>>>>>>> very much! >>>>>>>>>>>>>> >>>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>>> fix: >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>>> jtreg set another GC. >>>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>>> "requires" to skip test if there is a conflict >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for fixing this! It is really great that we >>>>>>>>>>>>>>> finally start sorting this out. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>>>> developed without much cooperation with the GC team. We >>>>>>>>>>>>>>> did have a lot of feedback when it was first presented a >>>>>>>>>>>>>>> year ago, but it does not seem like this feedback was >>>>>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>>>> >>>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>>>> expecting us to be able to influence the feature much more >>>>>>>>>>>>> than was the case now. >>>>>>>>>>>> >>>>>>>>>>>> My personal hope: @requires will address ~90% of existing >>>>>>>>>>>> issues. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think this change that gets proposed now is a big step >>>>>>>>>>>>>>> forward and I won't object to it. But I am pretty >>>>>>>>>>>>>>> convinced that we will soon run in to the limitations of >>>>>>>>>>>>>>> the current @requires implementation and we will have to >>>>>>>>>>>>>>> redo this work. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Some of the points I don't really like about the >>>>>>>>>>>>>>> @requires tag are: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than helping. >>>>>>>>>>>>>>> It would have been better to just "require" any command >>>>>>>>>>>>>>> line flag. >>>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>>>> possible to use: >>>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>>> >>>>>>>>>>>>> The problem with having this matching built in to JTreg is >>>>>>>>>>>>> that it makes it very hard to change. When we discussed >>>>>>>>>>>>> this a year ago I think we said that JTreg should only >>>>>>>>>>>>> provide a means to test against the command line and a >>>>>>>>>>>>> hook for running some java code in the @requires tag. That >>>>>>>>>>>>> way we could put logic like this in a test library that is >>>>>>>>>>>>> under our control. This would make it easy for us to >>>>>>>>>>>>> change and also enables us to use different logic for >>>>>>>>>>>>> different versions. >>>>>>>>>>>> >>>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>>>> have to do what you did in this change and use >>>>>>>>>>>>>>> vm.gc=null even when some tests could actually have been >>>>>>>>>>>>>>> run when a GC was specified. >>>>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, >>>>>>>>>>>>>> as well as test case support. >>>>>>>>>>>>> >>>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>>> >>>>>>>>>>>> Under test case support I mean ability to treat each @run >>>>>>>>>>>> as a separate test. Now >>>>>>>>>>>> >>>>>>>>>>>> @test >>>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>>> class MyTest { >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> is always a single test. You can't exclude, or re-run a >>>>>>>>>>>> part of it. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>>>> >>>>>>>>>>>>>> yes. conflicting GC is just the most popular problem >>>>>>>>>>>>>> caused by conflicting options. >>>>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>>>> solution, we could move further. >>>>>>>>>>>>> >>>>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>>>> Personally I would have preferred that the first step was >>>>>>>>>>>>> a "just run the command line as specified in the @run tag" >>>>>>>>>>>>> step. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Maybe this is not the right place to discuss the current >>>>>>>>>>>>>>> implementation of the @requires tag. I just want to say >>>>>>>>>>>>>>> that I'm not too happy about how the @requires tag >>>>>>>>>>>>>>> turned out. But assuming we have to use it the way it is >>>>>>>>>>>>>>> now I guess the proposed changeset looks good. >>>>>>>>>>>>>> >>>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>>>> about jtreg :) >>>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>>> >>>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have >>>>>>>>>>>>> not looked at all tests. But if they now pass with the >>>>>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>>>>> >>>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests >>>>>>>>>>>>>>>> are being excluded as expected. No tests failed because >>>>>>>>>>>>>>>> of the conflict. >>>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>>> >>>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will >>>>>>>>>>>>>> submit) >>>>>>>>>>>>> >>>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail >>>>>>>>>>>>>>> if you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail >>>>>>>>>>>>>>> if you run with -XX:-UseParNewGC. Could you test these >>>>>>>>>>>>>>> two cases too? >>>>>>>>>>>>>> >>>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>>> Add @requires here is not necessary, but it will allow >>>>>>>>>>>>>> not execute the tests when not needed. >>>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>>>> >>>>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>>>> this purpose? It seems like a way of misusing @requires. >>>>>>>>>>>>> If we just want the tests to be run once I think Leonid's >>>>>>>>>>>>> approach with tests lists seems more suitable. >>>>>>>>>>>> >>>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> But are you sure that this is the reason for the @requires >>>>>>>>>>>>> in this case? TestDefNewCMS does sound like a test that is >>>>>>>>>>>>> DefNew specific. I don't see a reason to run it with >>>>>>>>>>>>> ParNew. If it doesn't fail today it should probably be >>>>>>>>>>>>> changed so that it does fail if it is run with the wrong GC. >>>>>>>>>>>> >>>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy >>>>>>>>>>>> way to solve a lot of issues. >>>>>>>>>>>> >>>>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>>>> "selfish" tests, which produce a new java process to ignore >>>>>>>>>>>> vm flags coming from outside. No @requires, no other >>>>>>>>>>>> mechanism could 100% protect a test from running with >>>>>>>>>>>> conflicting options, but this is not the goal. >>>>>>>>>>>> >>>>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. >>>>>>>>>>>>>>> I'm about to push a changeset that removes them: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is there some way of making sure that all tests are run >>>>>>>>>>>>>>> at one time or another. With this change there is a risk >>>>>>>>>>>>>>> that some tests are never run and always skipped. Will >>>>>>>>>>>>>>> we somehow be tracking what gets skipped and make sure >>>>>>>>>>>>>>> that all tests are at least run once with the correct GC >>>>>>>>>>>>>>> so that it is not skipped all the time? >>>>>>>>>>>>>> >>>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it will >>>>>>>>>>>>>> do soon, after getting fix of: >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>>> >>>>>>>>>>>>>> And yes, tracking tests which are not run is important >>>>>>>>>>>>>> thing. >>>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Other examples: >>>>>>>>>>>>>> >>>>>>>>>>>>>> /* >>>>>>>>>>>>>> *@ignore >>>>>>>>>>>>>> *@test >>>>>>>>>>>>>> */ >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> >>>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>>> *@test >>>>>>>>>>>>>> */ >>>>>>>>>>>>>> ... >>>>>>>>>>>>>> Such tests will never be run, because jtreg treats as >>>>>>>>>>>>>> test only files with @test on the first place... >>>>>>>>>>>>>> >>>>>>>>>>>>>> So, making sure that tests do not disappear is important >>>>>>>>>>>>>> SQE task, we know about that, we're thinking on solution >>>>>>>>>>>>>> (may be very actively). But this subject for another >>>>>>>>>>>>>> discussion, not within RFR :) >>>>>>>>>>>>> >>>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>>> >>>>>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>>>>> know about this problem. With introducing @requires >>>>>>>>>>>> mechanism it will become more important! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your comments! >>>>>>>>>>>> >>>>>>>>>>>> -- Dima >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Bengt >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Dima >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /Evgeniya Stepanova/ >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /Evgeniya Stepanova/ >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nils.eliasson at oracle.com Fri Nov 14 09:43:28 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 14 Nov 2014 10:43:28 +0100 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out In-Reply-To: <5464D003.8050907@oracle.com> References: <5464BC2A.4020206@oracle.com> <5464C731.4090109@oracle.com> <5464D003.8050907@oracle.com> Message-ID: <5465CEC0.3040707@oracle.com> On 2014-11-13 16:36, Albert Noll wrote: > Hi Nils, > > On 11/13/2014 03:58 PM, Albert Noll wrote: >> Hi Nils, >> >> CompileQueue::lock() always returns MethodCompileQueue_lock ( see >> init_compiler_sweeper_threads() ). It seems that your changes don't >> change existing behavior, or am I missing something? Note that we >> have 2 compilation queues, but only 1 lock. >> > It seems that I missed the most important parts of you change indeed. > Is it right that > Mode evaluation_mode() const { return _no_safepoint; } > > makes the VM operation not execute at a safepoint? I have a question > nevertheless. Why did you choose to return '_no_safepoint' and not > '_concurrent'? The diagnostic commands run in their own thread, and that thread still has to wait for the output. Running the command cuncurrently would require some care with how the operation is allocated so that it can be passed to the VM_thread and be enqueued there. Regards, Nils > > Best, > Albert > >> On 11/13/2014 03:11 PM, Nils Eliasson wrote: >>> Hi, >>> >>> Please review this small change. >>> >>> 1) Fixing a deadlock in diagnostic command dcmd print_compile_queues >>> - between the CompileQueue_lock and safepointing. The >>> CompileQueue_lock requires that we are not at a safepoint when >>> taking it. Otherwise a java-thread that already holds the lock will >>> block when trying to get another lock. In this specific case the >>> compiler threads are Java threads, they take the CompileQueue_lock >>> frequently and some time takes the CompileTaskAlloc_lock after that. >>> >>> Fixing this by making the diagnostic command not request a safepoint, >>> >>> 2) Refactoring away the lock() accessor in CompileQueue and stating >>> the lock explicitly. This removes an element of surprise and makes >>> it easier understanding the code. >> What element of surprise are you talking about? Personally, I prefer >> the current design, i.e., getting the lock by calling a function >> instead of referencing a global variable directly. If we ever decide >> to go for a 2 compile queue locks (I think that makes sense since we >> have two compilation queues), we would have to introduce the >> functions again. >> >> Best, >> Albert >> >> >>> webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8061256 >>> >>> Thanks, >>> Nils Eliasson >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From albert.noll at oracle.com Fri Nov 14 09:59:32 2014 From: albert.noll at oracle.com (Albert Noll) Date: Fri, 14 Nov 2014 10:59:32 +0100 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out In-Reply-To: <5465CEC0.3040707@oracle.com> References: <5464BC2A.4020206@oracle.com> <5464C731.4090109@oracle.com> <5464D003.8050907@oracle.com> <5465CEC0.3040707@oracle.com> Message-ID: <5465D284.2010506@oracle.com> Hi Nils, On 11/14/2014 10:43 AM, Nils Eliasson wrote: > > On 2014-11-13 16:36, Albert Noll wrote: >> Hi Nils, >> >> On 11/13/2014 03:58 PM, Albert Noll wrote: >>> Hi Nils, >>> >>> CompileQueue::lock() always returns MethodCompileQueue_lock ( see >>> init_compiler_sweeper_threads() ). It seems that your changes don't >>> change existing behavior, or am I missing something? Note that we >>> have 2 compilation queues, but only 1 lock. >>> >> It seems that I missed the most important parts of you change indeed. >> Is it right that >> Mode evaluation_mode() const { return _no_safepoint; } >> >> makes the VM operation not execute at a safepoint? I have a question >> nevertheless. Why did you choose to return '_no_safepoint' and not >> '_concurrent'? > > The diagnostic commands run in their own thread, and that thread still > has to wait for the output. Running the command cuncurrently would > require some care with how the operation is allocated so that it can > be passed to the VM_thread and be enqueued there. > Thanks for the explanation. Best, Albert > Regards, > Nils > >> >> Best, >> Albert >> >>> On 11/13/2014 03:11 PM, Nils Eliasson wrote: >>>> Hi, >>>> >>>> Please review this small change. >>>> >>>> 1) Fixing a deadlock in diagnostic command dcmd >>>> print_compile_queues - between the CompileQueue_lock and >>>> safepointing. The CompileQueue_lock requires that we are not at a >>>> safepoint when taking it. Otherwise a java-thread that already >>>> holds the lock will block when trying to get another lock. In this >>>> specific case the compiler threads are Java threads, they take the >>>> CompileQueue_lock frequently and some time takes the >>>> CompileTaskAlloc_lock after that. >>>> >>>> Fixing this by making the diagnostic command not request a safepoint, >>>> >>>> 2) Refactoring away the lock() accessor in CompileQueue and stating >>>> the lock explicitly. This removes an element of surprise and makes >>>> it easier understanding the code. >>> What element of surprise are you talking about? Personally, I prefer >>> the current design, i.e., getting the lock by calling a function >>> instead of referencing a global variable directly. If we ever decide >>> to go for a 2 compile queue locks (I think that makes sense since we >>> have two compilation queues), we would have to introduce the >>> functions again. >>> >>> Best, >>> Albert >>> >>> >>>> webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8061256 >>>> >>>> Thanks, >>>> Nils Eliasson >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.fazunenko at oracle.com Fri Nov 14 10:02:07 2014 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Fri, 14 Nov 2014 14:02:07 +0400 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5465C874.10504@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> <5464B8C0.6050300@oracle.com> <5465C874.10504@oracle.com> Message-ID: <5465D31F.2040002@oracle.com> Hi Bengt, On 14.11.2014 13:16, Bengt Rutisson wrote: > > On 2014-11-13 14:57, Dmitry Fazunenko wrote: >> >> On 13.11.2014 17:59, Bengt Rutisson wrote: >>> >>> On 2014-11-13 13:56, Dmitry Fazunenko wrote: >>>> >>>> On 13.11.2014 17:42, Bengt Rutisson wrote: >>>>> >>>>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>>>> >>>>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Evgeniya, >>>>>>> >>>>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>>>> Hi Dmitry, >>>>>>>> >>>>>>>> You are right - I've forgotten about copyrights >>>>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>>>> >>>>>>> >>>>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it >>>>>>> would be good to add -XX:+UseG1GC to the @run tags and then use >>>>>>> @requires vm.gc=="G1" | vm.gc == null. >>>>>>> >>>>>>> >>>>>>> The change to test/gc/defnew/HeapChangeLogging.java is unrelated >>>>>>> to the conflicting GC combinations. Should that really be part >>>>>>> of this changeset? >>>>>>> >>>>>>> >>>>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do >>>>>>> we really need @requires for them? >>>>>> >>>>>> Yes, we do. >>>>>> These tests use TestShrinkAuxiliaryData class which implements >>>>>> its own mechanism to analyze VM options an skip if not applicable >>>>>> collector is given. @requires - allows to rely on jtreg. >>>>>> >>>>>> Driver mode is a kind of indicator, that the test will spawn its >>>>>> own java process. >>>>> >>>>> I thought the point of @driver was that no external vmoptions were >>>>> passed to such a test. Is that not the case? >>>> >>>> In the driver mode VM is started without external VM flags. Those >>>> flags are passed to the tests via system property. >>>> The driver mode is a sort of shell to start something else. >>> >>> Right. So, why would you need @requires on the >>> TestShrinkAuxiliaryDataXX tests because the utility >>> TestShrinkAuxiliaryData picks up the vm flags through >>> Utils.getTestJavaOpts(). What's the point in running this in a >>> driver mode when they anyway pick up the vm flags? >> >> TestShrinkAuxiliaryData implemented a workaround awaiting for >> @requires to appear in jtreg. >> >> Frankly speaking, the driver mode doesn't bring a lot of value, it's >> rather confusing and obligate developers to be more careful. If a >> class just spawns another java process with a real test, it's a big >> deal to run this class with or without external options. But there is >> no guarantee, that people will not start run real tests in driver mode... > > Ok. So, do we want to keep "driver" for this test or not? I believe yes: TestShrinkAuxiliaryData - is a real test TestShrinkAuxiliaryDataXX - are drivers that run TestShrinkAuxiliaryData with various options So, this is a good example of usage the 'driver' concept. > >> >>> >>> I'm asking because adding @requires to these tests means that we >>> will run them less often than we do now. So, I'm a bit worried that >>> we reduce the amount of testing we do. >> >> Don't worry about it. >> We want to run more tests, believe me. > > Sure, but adding @requires means that we run the test less often. The > TestShrinkAuxiliaryData tests were for example run every week in PIT > testing but with the @requires tag they will only be run every 4th > week since PIT testing is rotating which GC it runs tests with. We don't specify GC for PIT testing, so it will be executed every week. In promotion testing we specify two GC, so the test will be executed every 2nd week. In nightly testing it will be executed every day. Thanks, Dima > > Bengt > >> >> -- Dima >> >> >>> >>> Bengt >>> >>>> >>>> -- Dima >>>> >>>> >>>>> >>>>> Bengt >>>>> >>>>>> >>>>>> Thanks >>>>>> Dima >>>>>> >>>>>>> >>>>>>> >>>>>>> Otherwise it look ok to me. >>>>>>> >>>>>>> Bengt >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> Evgeniya Stepanova >>>>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>>>> Hi Evgeniya, >>>>>>>>> >>>>>>>>> The fix looks good to me. >>>>>>>>> >>>>>>>>> I noticed the following minor things: >>>>>>>>> - copyrights need to include the year of last change >>>>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among >>>>>>>>> updated files, but doesn't contain any changes >>>>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>>>>> variable 'prohibitedVmOptions' >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Dima >>>>>>>>> >>>>>>>>> >>>>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>>>> Hi everyone! >>>>>>>>>> >>>>>>>>>> Since the decision was made to change only tests that fail >>>>>>>>>> because of conflict for now (skip "selfish" tests), I post >>>>>>>>>> new webrev for hotspot part of the JDK-8019361 >>>>>>>>>> : >>>>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Evgeniya Stepanova >>>>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>>>> Nice plan! Please feel free to send me any >>>>>>>>>>> feedback/questions regarding @requires >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Dima >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Dima, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for the answers. I think the currently proposed >>>>>>>>>>>> patch is a good start. We will have to evolve the @requires >>>>>>>>>>>> tag in the future, but let's have that discussion separate >>>>>>>>>>>> from this review. And we can start that discussion later >>>>>>>>>>>> when we have more experience with the current version of >>>>>>>>>>>> @requires. >>>>>>>>>>>> >>>>>>>>>>>> Thanks for doing this! >>>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>> >>>>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>>>> >>>>>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>>>>> sort of plugins, so you can't provide a VM specific >>>>>>>>>>>>> handler of the @requires or another tag. This is very >>>>>>>>>>>>> annoying limitation and we have to live with it. >>>>>>>>>>>>> >>>>>>>>>>>>> A few more comments inline. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Answers inline. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate >>>>>>>>>>>>>>> it very much! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please review changes for 8062537, the OpenJDK/hotspot >>>>>>>>>>>>>>>>> part of the JDK-8019361 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>>>> fix: >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>>>> jtreg set another GC. >>>>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>>>> "requires" to skip test if there is a conflict >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for fixing this! It is really great that we >>>>>>>>>>>>>>>> finally start sorting this out. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>>>>> developed without much cooperation with the GC team. We >>>>>>>>>>>>>>>> did have a lot of feedback when it was first presented >>>>>>>>>>>>>>>> a year ago, but it does not seem like this feedback was >>>>>>>>>>>>>>>> incorporated into the @requires that was eventually built. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>>>>> expecting us to be able to influence the feature much >>>>>>>>>>>>>> more than was the case now. >>>>>>>>>>>>> >>>>>>>>>>>>> My personal hope: @requires will address ~90% of existing >>>>>>>>>>>>> issues. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think this change that gets proposed now is a big >>>>>>>>>>>>>>>> step forward and I won't object to it. But I am pretty >>>>>>>>>>>>>>>> convinced that we will soon run in to the limitations >>>>>>>>>>>>>>>> of the current @requires implementation and we will >>>>>>>>>>>>>>>> have to redo this work. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Some of the points I don't really like about the >>>>>>>>>>>>>>>> @requires tag are: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than >>>>>>>>>>>>>>>> helping. It would have been better to just "require" >>>>>>>>>>>>>>>> any command line flag. >>>>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>>>>> possible to use: >>>>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>>>> >>>>>>>>>>>>>> The problem with having this matching built in to JTreg >>>>>>>>>>>>>> is that it makes it very hard to change. When we >>>>>>>>>>>>>> discussed this a year ago I think we said that JTreg >>>>>>>>>>>>>> should only provide a means to test against the command >>>>>>>>>>>>>> line and a hook for running some java code in the >>>>>>>>>>>>>> @requires tag. That way we could put logic like this in a >>>>>>>>>>>>>> test library that is under our control. This would make >>>>>>>>>>>>>> it easy for us to change and also enables us to use >>>>>>>>>>>>>> different logic for different versions. >>>>>>>>>>>>> >>>>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>>>>> have to do what you did in this change and use >>>>>>>>>>>>>>>> vm.gc=null even when some tests could actually have >>>>>>>>>>>>>>>> been run when a GC was specified. >>>>>>>>>>>>>>> it would be great, but it will unlikely happen in jtreg, >>>>>>>>>>>>>>> as well as test case support. >>>>>>>>>>>>>> >>>>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>>>> >>>>>>>>>>>>> Under test case support I mean ability to treat each @run >>>>>>>>>>>>> as a separate test. Now >>>>>>>>>>>>> >>>>>>>>>>>>> @test >>>>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>>>> class MyTest { >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> is always a single test. You can't exclude, or re-run a >>>>>>>>>>>>> part of it. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> yes. conflicting GC is just the most popular problem >>>>>>>>>>>>>>> caused by conflicting options. >>>>>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>>>>> solution, we could move further. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>>>>> Personally I would have preferred that the first step was >>>>>>>>>>>>>> a "just run the command line as specified in the @run >>>>>>>>>>>>>> tag" step. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Maybe this is not the right place to discuss the >>>>>>>>>>>>>>>> current implementation of the @requires tag. I just >>>>>>>>>>>>>>>> want to say that I'm not too happy about how the >>>>>>>>>>>>>>>> @requires tag turned out. But assuming we have to use >>>>>>>>>>>>>>>> it the way it is now I guess the proposed changeset >>>>>>>>>>>>>>>> looks good. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>>>>> about jtreg :) >>>>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>>>> >>>>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I have >>>>>>>>>>>>>> not looked at all tests. But if they now pass with the >>>>>>>>>>>>>> combinations that we test with I guess they should be ok. >>>>>>>>>>>>> >>>>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests >>>>>>>>>>>>>>>>> are being excluded as expected. No tests failed >>>>>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will >>>>>>>>>>>>>>> submit) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail >>>>>>>>>>>>>>>> if you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail >>>>>>>>>>>>>>>> if you run with -XX:-UseParNewGC. Could you test these >>>>>>>>>>>>>>>> two cases too? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>>>> Add @requires here is not necessary, but it will allow >>>>>>>>>>>>>>> not execute the tests when not needed. >>>>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>>>>> this purpose? It seems like a way of misusing @requires. >>>>>>>>>>>>>> If we just want the tests to be run once I think Leonid's >>>>>>>>>>>>>> approach with tests lists seems more suitable. >>>>>>>>>>>>> >>>>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> But are you sure that this is the reason for the >>>>>>>>>>>>>> @requires in this case? TestDefNewCMS does sound like a >>>>>>>>>>>>>> test that is DefNew specific. I don't see a reason to run >>>>>>>>>>>>>> it with ParNew. If it doesn't fail today it should >>>>>>>>>>>>>> probably be changed so that it does fail if it is run >>>>>>>>>>>>>> with the wrong GC. >>>>>>>>>>>>> >>>>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy >>>>>>>>>>>>> way to solve a lot of issues. >>>>>>>>>>>>> >>>>>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>>>>> "selfish" tests, which produce a new java process to >>>>>>>>>>>>> ignore vm flags coming from outside. No @requires, no >>>>>>>>>>>>> other mechanism could 100% protect a test from running >>>>>>>>>>>>> with conflicting options, but this is not the goal. >>>>>>>>>>>>> >>>>>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>>>>> collector, there shouldn't mass failures caused by options >>>>>>>>>>>>> conflicts. But a few failures could be handled manually. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Similarly it looks to me like there are tests that will >>>>>>>>>>>>>>>> fail if you run them with -XX:-UseParallelOldGC or >>>>>>>>>>>>>>>> -XX:+UseParallelOldGC. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. >>>>>>>>>>>>>>>> I'm about to push a changeset that removes them: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Is there some way of making sure that all tests are run >>>>>>>>>>>>>>>> at one time or another. With this change there is a >>>>>>>>>>>>>>>> risk that some tests are never run and always skipped. >>>>>>>>>>>>>>>> Will we somehow be tracking what gets skipped and make >>>>>>>>>>>>>>>> sure that all tests are at least run once with the >>>>>>>>>>>>>>>> correct GC so that it is not skipped all the time? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it >>>>>>>>>>>>>>> will do soon, after getting fix of: >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> And yes, tracking tests which are not run is important >>>>>>>>>>>>>>> thing. >>>>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Other examples: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> /* >>>>>>>>>>>>>>> *@ignore >>>>>>>>>>>>>>> *@test >>>>>>>>>>>>>>> */ >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>>>> *@test >>>>>>>>>>>>>>> */ >>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>> Such tests will never be run, because jtreg treats as >>>>>>>>>>>>>>> test only files with @test on the first place... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So, making sure that tests do not disappear is >>>>>>>>>>>>>>> important SQE task, we know about that, we're thinking >>>>>>>>>>>>>>> on solution (may be very actively). But this subject >>>>>>>>>>>>>>> for another discussion, not within RFR :) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>>>> >>>>>>>>>>>>> I was going to say "not very actively", but never mind, we >>>>>>>>>>>>> know about this problem. With introducing @requires >>>>>>>>>>>>> mechanism it will become more important! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your comments! >>>>>>>>>>>>> >>>>>>>>>>>>> -- Dima >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Dima >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> /Evgeniya Stepanova/ >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.hartmann at oracle.com Fri Nov 14 11:17:53 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 14 Nov 2014 12:17:53 +0100 Subject: [9] RFR(S): 8050079: crash while compiling java.lang.ref.Finalizer::runFinalizer In-Reply-To: <54650B17.1030401@oracle.com> References: <5464C576.7040705@oracle.com> <54650B17.1030401@oracle.com> Message-ID: <5465E4E1.40609@oracle.com> Hi Vladimir, thanks for the review. On 13.11.2014 20:48, Vladimir Kozlov wrote: > Should you fix problem (2) other way around by adding check in the following > loop? If you don't bailout on 'nof_impls > 1' on interface what result you get? According to the comment we want to avoid this case: *I.m > { A.m, C }; B.m > C There are two cases were we may encounter an interface in the subclass hierarchy and do not check for 'nof_impls > 1': (1) context_type is java.lang.Object (2) context_type is an interface (1) If context_type is a java.lang.Object O it should be fine not to bailout if we encounter an interface with two implementations. For example: O.m > *I > { A.m, C }; B.m > C Coming from O we would miss the second implementation of I that is inherited from B and overrides m. But since B also inherits from Object O.m > B.m we catch B.m while traversing B and bailout because we now have two witnesses. So no need to bailout early in this case. (2) If context_type is an interface we may have the following hierarchy: *I.m > *I2 > { A.m, C }; B.m > C Again, we would miss the the second implementation of *I.m inherited from B.m. But since nof_impls == 2 for *I, we bailout even before reaching *I2. So I don't think we have to move the check into the loop. What do you think? > I think test should go into new 'dependencies' subdir not 'inlining'. Okay, I moved it to compiler/dependencies/MonomorphicObjectCall/. > Why you fork separate JVM process? Because I need to set the -Xbootclasspath to the test classes directory to make sure that the modified version of java.lang.Object is used. Specifying it in the '@run main/othervm' command with '-Xbootclasspath/p:../classes/compiler/dependencies/TestMonomorphicObjectCall/' depends on the location of classes and the working directory (fails on JPRT). I solve this by reading the directory with 'System.getProperty("test.classes")' and forking a separate JVM process. Another solution would be to use a shell script but I prefer the more readable Java code. New webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.01/ Thanks, Tobias > > Thanks, > Vladimir > > On 11/13/14 6:51 AM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8050079 >> Webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.00/ >> >> Problem (1): >> C1 compiles a call to Object.finalize() in >> 'java.lang.ref.Finalizer::runFinalizer'. To determine if the call is >> monomorphic, the compiler searches through the subclasses of Object until it >> finds a class that overrides finalize(). This check is performed in >> 'ClassHierarchyWalker::find_witness_anywhere' by calling 'is_witness(Klass* k)' >> on the subclasses. After casting with 'InstanceKlass::cast(k)' the (Sub-)Klass k >> is treated as an InstanceKlass and 'InstanceKlass::find_method' is called. The >> crash happens in 'binary_search' when a subclass k is not an InstanceKlass but >> an ObjectArrayKlass and therefore '_methods' is NULL. >> >> Why didn't the bug show up earlier? >> The method Object.finalize() is usually overridden by the subclasses >> 'java.lang.ClassLoader$NativeLibrary' and 'java.lang.Enum' which are loaded >> during startup. That means a call to Object.finalize() is never treated as >> monomorphic and 'ClassHierarchyWalker::find_witness_anywhere' returns before >> encountering an ObjectArrayKlass in the subclass hierarchy of Object. The >> failure only occurs in an internal test environment for JRebel [1]. According to >> the core file of the crash, the usual class hierarchy of the Java Runtime >> Environment was changed by the framework. For example, java.lang.String is no >> longer a subclass of java.lang.Object but of another class called >> 'JaveleonObject' [2]. Especially the two overriding classes >> 'ClassLoader$NativeLibrary' and 'Enum' are not subclasses of Object. This leads >> to the crash because C1 encounters an ObjectArrayKlass in the subclass hierarchy. >> Of course, this bug could also show up if the sequence of subclasses in the >> 'Klass::subklass()' list changes. >> >> Problem (2): >> In case of a worklist overflow we recursively call 'find_witness_anywhere' and >> create a new worklist (see line 1178 of dependencies.cpp). The problem is that >> we now execute the interface related checks that would otherwise not be executed >> (line 1132). For interface subtypes of Object (for example java/lang/Readable) >> we then bail out if 'nof_impls > 1'. >> >> Solution: >> (1) A check for 'oop_is_instance' is added to ignore non-instance Klasses in the >> subclass hierarchy. >> (2) I added a check to only execute the interface related checks if we come from >> a top level call. >> >> I was finally able to write a small jtreg regression test. Although the test is >> a little bit hacky, it is stable and deterministically reproduces the bug. The >> test works by overriding the default implementation of java.lang.Object, >> renaming the method 'finalize' to 'finalizeObject' and thus guaranteeing that >> 'finalizeObject' is never overridden and all calls to it are monomorphic. By >> triggering a C1 compilation of 'Object.finalizeObject()' we trigger the bug >> because there are no overriding instance Klasses in the call hierarchy. >> >> The '-XX:-VerifyDependencies' flag is necessary to not bail out too early >> because of problem (2). >> >> Testing: >> JPRT with new test >> >> Thanks, >> Tobias >> >> >> [1] http://zeroturnaround.com/software/jrebel/ >> [2] https://bugs.openjdk.java.net/secure/attachment/23360/replay.log >> From bengt.rutisson at oracle.com Fri Nov 14 11:47:58 2014 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 14 Nov 2014 12:47:58 +0100 Subject: RFR: 8062537: [TESTBUG] Conflicting GC combinations in hotspot tests In-Reply-To: <5465D31F.2040002@oracle.com> References: <545245C5.4050504@oracle.com> <5452A8F7.8080709@oracle.com> <545386E1.2050402@oracle.com> <545783A1.3050300@oracle.com> <5457EFA6.7050404@oracle.com> <5458910B.2070100@oracle.com> <5458B960.8000305@oracle.com> <5463736F.7050109@oracle.com> <54636D76.3010905@oracle.com> <54638A9F.2030209@oracle.com> <5464B2E6.9070802@oracle.com> <5464A8ED.4070708@oracle.com> <5464B533.9000100@oracle.com> <5464AA83.4030306@oracle.com> <5464B946.90806@oracle.com> <5464B8C0.6050300@oracle.com> <5465C874.10504@oracle.com> <5465D31F.2040002@oracle.com> Message-ID: <5465EBEE.3010900@oracle.com> On 2014-11-14 11:02, Dmitry Fazunenko wrote: > Hi Bengt, > > On 14.11.2014 13:16, Bengt Rutisson wrote: >> >> On 2014-11-13 14:57, Dmitry Fazunenko wrote: >>> >>> On 13.11.2014 17:59, Bengt Rutisson wrote: >>>> >>>> On 2014-11-13 13:56, Dmitry Fazunenko wrote: >>>>> >>>>> On 13.11.2014 17:42, Bengt Rutisson wrote: >>>>>> >>>>>> On 2014-11-13 13:49, Dmitry Fazunenko wrote: >>>>>>> >>>>>>> On 13.11.2014 17:32, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Evgeniya, >>>>>>>> >>>>>>>> On 2014-11-12 17:28, Evgeniya Stepanova wrote: >>>>>>>>> Hi Dmitry, >>>>>>>>> >>>>>>>>> You are right - I've forgotten about copyrights >>>>>>>>> Copyrights and other issues you mentioned fixed. New webrev: >>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.02/ >>>>>>>> >>>>>>>> >>>>>>>> For /test/gc/arguments/TestG1HeapRegionSize.java I think it >>>>>>>> would be good to add -XX:+UseG1GC to the @run tags and then >>>>>>>> use @requires vm.gc=="G1" | vm.gc == null. >>>>>>>> >>>>>>>> >>>>>>>> The change to test/gc/defnew/HeapChangeLogging.java is >>>>>>>> unrelated to the conflicting GC combinations. Should that >>>>>>>> really be part of this changeset? >>>>>>>> >>>>>>>> >>>>>>>> The TestShrinkAuxiliaryDataXX tests are run in driver mode. Do >>>>>>>> we really need @requires for them? >>>>>>> >>>>>>> Yes, we do. >>>>>>> These tests use TestShrinkAuxiliaryData class which implements >>>>>>> its own mechanism to analyze VM options an skip if not >>>>>>> applicable collector is given. @requires - allows to rely on jtreg. >>>>>>> >>>>>>> Driver mode is a kind of indicator, that the test will spawn its >>>>>>> own java process. >>>>>> >>>>>> I thought the point of @driver was that no external vmoptions >>>>>> were passed to such a test. Is that not the case? >>>>> >>>>> In the driver mode VM is started without external VM flags. Those >>>>> flags are passed to the tests via system property. >>>>> The driver mode is a sort of shell to start something else. >>>> >>>> Right. So, why would you need @requires on the >>>> TestShrinkAuxiliaryDataXX tests because the utility >>>> TestShrinkAuxiliaryData picks up the vm flags through >>>> Utils.getTestJavaOpts(). What's the point in running this in a >>>> driver mode when they anyway pick up the vm flags? >>> >>> TestShrinkAuxiliaryData implemented a workaround awaiting for >>> @requires to appear in jtreg. >>> >>> Frankly speaking, the driver mode doesn't bring a lot of value, it's >>> rather confusing and obligate developers to be more careful. If a >>> class just spawns another java process with a real test, it's a big >>> deal to run this class with or without external options. But there >>> is no guarantee, that people will not start run real tests in driver >>> mode... >> >> Ok. So, do we want to keep "driver" for this test or not? > > I believe yes: > TestShrinkAuxiliaryData - is a real test > TestShrinkAuxiliaryDataXX - are drivers that run > TestShrinkAuxiliaryData with various options > > So, this is a good example of usage the 'driver' concept. Ok. Sounds good. > >> >>> >>>> >>>> I'm asking because adding @requires to these tests means that we >>>> will run them less often than we do now. So, I'm a bit worried that >>>> we reduce the amount of testing we do. >>> >>> Don't worry about it. >>> We want to run more tests, believe me. >> >> Sure, but adding @requires means that we run the test less often. The >> TestShrinkAuxiliaryData tests were for example run every week in PIT >> testing but with the @requires tag they will only be run every 4th >> week since PIT testing is rotating which GC it runs tests with. > We don't specify GC for PIT testing, so it will be executed every week. > In promotion testing we specify two GC, so the test will be executed > every 2nd week. > In nightly testing it will be executed every day. Good. Thanks for clarifying. I obviously had PIT and promotion testing mixed up. In that case I think you can consider this change reviewed from my side. Thanks for putting up with all the questions! Bengt > > Thanks, > Dima > >> >> Bengt >> >>> >>> -- Dima >>> >>> >>>> >>>> Bengt >>>> >>>>> >>>>> -- Dima >>>>> >>>>> >>>>>> >>>>>> Bengt >>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> Dima >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Otherwise it look ok to me. >>>>>>>> >>>>>>>> Bengt >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Evgeniya Stepanova >>>>>>>>> On 12.11.2014 18:23, Dmitry Fazunenko wrote: >>>>>>>>>> Hi Evgeniya, >>>>>>>>>> >>>>>>>>>> The fix looks good to me. >>>>>>>>>> >>>>>>>>>> I noticed the following minor things: >>>>>>>>>> - copyrights need to include the year of last change >>>>>>>>>> - test/gc/defnew/HeapChangeLogging.java - is listed among >>>>>>>>>> updated files, but doesn't contain any changes >>>>>>>>>> - test/gc/g1/TestShrinkAuxiliaryData.java - contain unsed >>>>>>>>>> variable 'prohibitedVmOptions' >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Dima >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 12.11.2014 18:49, Evgeniya Stepanova wrote: >>>>>>>>>>> Hi everyone! >>>>>>>>>>> >>>>>>>>>>> Since the decision was made to change only tests that fail >>>>>>>>>>> because of conflict for now (skip "selfish" tests), I post >>>>>>>>>>> new webrev for hotspot part of the JDK-8019361 >>>>>>>>>>> : >>>>>>>>>>> http://cr.openjdk.java.net/~avstepan/eistepan/8062537/webrev.01/ >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>> On 04.11.2014 15:32, Dmitry Fazunenko wrote: >>>>>>>>>>>> Nice plan! Please feel free to send me any >>>>>>>>>>>> feedback/questions regarding @requires >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Dima >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 04.11.2014 11:40, Bengt Rutisson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the answers. I think the currently proposed >>>>>>>>>>>>> patch is a good start. We will have to evolve the >>>>>>>>>>>>> @requires tag in the future, but let's have that >>>>>>>>>>>>> discussion separate from this review. And we can start >>>>>>>>>>>>> that discussion later when we have more experience with >>>>>>>>>>>>> the current version of @requires. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for doing this! >>>>>>>>>>>>> Bengt >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/3/14 10:12 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>>> >>>>>>>>>>>>>> That's great that we have very closed visions! >>>>>>>>>>>>>> >>>>>>>>>>>>>> The general comment: currently, jtreg doesn't support any >>>>>>>>>>>>>> sort of plugins, so you can't provide a VM specific >>>>>>>>>>>>>> handler of the @requires or another tag. This is very >>>>>>>>>>>>>> annoying limitation and we have to live with it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> A few more comments inline. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 03.11.2014 16:31, Bengt Rutisson wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Dima, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Answers inline. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 10/31/14 1:56 PM, Dmitry Fazunenko wrote: >>>>>>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks a lot for your detailed feedback, we appreciate >>>>>>>>>>>>>>>> it very much! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> See comments inline. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 31.10.2014 1:09, Bengt Rutisson wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Evgeniya, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 10/30/14 3:05 PM, Evgeniya Stepanova wrote: >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Please review changes for 8062537, the >>>>>>>>>>>>>>>>>> OpenJDK/hotspot part of the JDK-8019361 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8062537 >>>>>>>>>>>>>>>>>> fix: >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~eistepan/8062537/webrev.00/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Problem: Some tests explicitly set GC and fail when >>>>>>>>>>>>>>>>>> jtreg set another GC. >>>>>>>>>>>>>>>>>> Solution: Such tests marked with the jtreg tag >>>>>>>>>>>>>>>>>> "requires" to skip test if there is a conflict >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for fixing this! It is really great that we >>>>>>>>>>>>>>>>> finally start sorting this out. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> First a general comment. The @requires tag has been >>>>>>>>>>>>>>>>> developed without much cooperation with the GC team. >>>>>>>>>>>>>>>>> We did have a lot of feedback when it was first >>>>>>>>>>>>>>>>> presented a year ago, but it does not seem like this >>>>>>>>>>>>>>>>> feedback was incorporated into the @requires that was >>>>>>>>>>>>>>>>> eventually built. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We tried to implement as much developer's wishes as >>>>>>>>>>>>>>>> possible. But not everything is possible, sorry for that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes, I'm sure you have done your best. It's just that we >>>>>>>>>>>>>>> have been requesting this feature for 3 years and I was >>>>>>>>>>>>>>> expecting us to be able to influence the feature much >>>>>>>>>>>>>>> more than was the case now. >>>>>>>>>>>>>> >>>>>>>>>>>>>> My personal hope: @requires will address ~90% of existing >>>>>>>>>>>>>> issues. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think this change that gets proposed now is a big >>>>>>>>>>>>>>>>> step forward and I won't object to it. But I am pretty >>>>>>>>>>>>>>>>> convinced that we will soon run in to the limitations >>>>>>>>>>>>>>>>> of the current @requires implementation and we will >>>>>>>>>>>>>>>>> have to redo this work. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Some of the points I don't really like about the >>>>>>>>>>>>>>>>> @requires tag are: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - the "vm.gc" abstraction is more limiting than >>>>>>>>>>>>>>>>> helping. It would have been better to just "require" >>>>>>>>>>>>>>>>> any command line flag. >>>>>>>>>>>>>>>> "vm.gc" is an alias to a very popular flag. It's also >>>>>>>>>>>>>>>> possible to use: >>>>>>>>>>>>>>>> vm.opt.UseG1GC == true instead. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The table with all vars available in jtreg: >>>>>>>>>>>>>>>> http://jre.us.oracle.com/java/re/jtreg/4.1/promoted/latest/binaries/jtreg/doc/jtreg/tag-spec.html#requires_names >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The problem with having this matching built in to JTreg >>>>>>>>>>>>>>> is that it makes it very hard to change. When we >>>>>>>>>>>>>>> discussed this a year ago I think we said that JTreg >>>>>>>>>>>>>>> should only provide a means to test against the command >>>>>>>>>>>>>>> line and a hook for running some java code in the >>>>>>>>>>>>>>> @requires tag. That way we could put logic like this in >>>>>>>>>>>>>>> a test library that is under our control. This would >>>>>>>>>>>>>>> make it easy for us to change and also enables us to use >>>>>>>>>>>>>>> different logic for different versions. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I would be glad to have own harness... >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - the requirement should be per @run tag. Right now we >>>>>>>>>>>>>>>>> have to do what you did in this change and use >>>>>>>>>>>>>>>>> vm.gc=null even when some tests could actually have >>>>>>>>>>>>>>>>> been run when a GC was specified. >>>>>>>>>>>>>>>> it would be great, but it will unlikely happen in >>>>>>>>>>>>>>>> jtreg, as well as test case support. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> what do you mean with test case support? Hi Evgeniya, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Under test case support I mean ability to treat each @run >>>>>>>>>>>>>> as a separate test. Now >>>>>>>>>>>>>> >>>>>>>>>>>>>> @test >>>>>>>>>>>>>> @run -XX:g1RegSize=1m MyTest >>>>>>>>>>>>>> @run -XX:g1RegSize=2m MyTest >>>>>>>>>>>>>> @run -XX:g1RegSize=4m MyTest >>>>>>>>>>>>>> class MyTest { >>>>>>>>>>>>>> } >>>>>>>>>>>>>> >>>>>>>>>>>>>> is always a single test. You can't exclude, or re-run a >>>>>>>>>>>>>> part of it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - there are many tests that require more than just a >>>>>>>>>>>>>>>>> specific GC. Often there are other flags that can't be >>>>>>>>>>>>>>>>> changed either for the test to work properly. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> yes. conflicting GC is just the most popular problem >>>>>>>>>>>>>>>> caused by conflicting options. >>>>>>>>>>>>>>>> If we address this issue and we are satisfied with >>>>>>>>>>>>>>>> solution, we could move further. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes, I agree that taking one step at the time is good. >>>>>>>>>>>>>>> Personally I would have preferred that the first step >>>>>>>>>>>>>>> was a "just run the command line as specified in the >>>>>>>>>>>>>>> @run tag" step. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Maybe this is not the right place to discuss the >>>>>>>>>>>>>>>>> current implementation of the @requires tag. I just >>>>>>>>>>>>>>>>> want to say that I'm not too happy about how the >>>>>>>>>>>>>>>>> @requires tag turned out. But assuming we have to use >>>>>>>>>>>>>>>>> it the way it is now I guess the proposed changeset >>>>>>>>>>>>>>>>> looks good. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> yes, this thread is about change made by Evgeniya, not >>>>>>>>>>>>>>>> about jtreg :) >>>>>>>>>>>>>>>> And thanks for reviewing it! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Agreed. And as I said, I think the patch looks ok. I >>>>>>>>>>>>>>> have not looked at all tests. But if they now pass with >>>>>>>>>>>>>>> the combinations that we test with I guess they should >>>>>>>>>>>>>>> be ok. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Excellent! Thanks a lot! >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Tested locally with different GC flags (-XX:+UseG1GC, >>>>>>>>>>>>>>>>>> -XX:+UseParallelGC, -XX:+UseSerialGC, >>>>>>>>>>>>>>>>>> -XX:+UseConcMarkSweep and without any GC flag). Tests >>>>>>>>>>>>>>>>>> are being excluded as expected. No tests failed >>>>>>>>>>>>>>>>>> because of the conflict. >>>>>>>>>>>>>>>>> Have you tested with -Xconcgc too? It's an alias for >>>>>>>>>>>>>>>>> -XX:+UseConcMarkSweepGC. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> '-Xconcgc' is not supported yet. (bug in jtreg, I will >>>>>>>>>>>>>>>> submit) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Ok. Thanks. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I think some of the test, like >>>>>>>>>>>>>>>>> test/gc/startup_warnings/TestDefNewCMS.java, will fail >>>>>>>>>>>>>>>>> if you run with -XX:+UseParNewGC. Others, like >>>>>>>>>>>>>>>>> test/gc/startup_warnings/TestParNewCMS.java, will fail >>>>>>>>>>>>>>>>> if you run with -XX:-UseParNewGC. Could you test these >>>>>>>>>>>>>>>>> two cases too? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> These two tests ignore vm flags. >>>>>>>>>>>>>>>> Add @requires here is not necessary, but it will allow >>>>>>>>>>>>>>>> not execute the tests when not needed. >>>>>>>>>>>>>>>> So, if we run HS tests with 4 GC, we don't need to run >>>>>>>>>>>>>>>> these tests 4 times, 1 should be enough. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Do we really want to use the @requires functionality for >>>>>>>>>>>>>>> this purpose? It seems like a way of misusing @requires. >>>>>>>>>>>>>>> If we just want the tests to be run once I think >>>>>>>>>>>>>>> Leonid's approach with tests lists seems more suitable. >>>>>>>>>>>>>> >>>>>>>>>>>>>> No, it's not a purpose of course, it's just side effect :) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> But are you sure that this is the reason for the >>>>>>>>>>>>>>> @requires in this case? TestDefNewCMS does sound like a >>>>>>>>>>>>>>> test that is DefNew specific. I don't see a reason to >>>>>>>>>>>>>>> run it with ParNew. If it doesn't fail today it should >>>>>>>>>>>>>>> probably be changed so that it does fail if it is run >>>>>>>>>>>>>>> with the wrong GC. >>>>>>>>>>>>>> >>>>>>>>>>>>>> @requires - is not the silver bullet, but it's quite easy >>>>>>>>>>>>>> way to solve a lot of issues. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I hope, @requires will allow to reduce the number of >>>>>>>>>>>>>> "selfish" tests, which produce a new java process to >>>>>>>>>>>>>> ignore vm flags coming from outside. No @requires, no >>>>>>>>>>>>>> other mechanism could 100% protect a test from running >>>>>>>>>>>>>> with conflicting options, but this is not the goal. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If one runs tests with an exotic option, like a new G2 >>>>>>>>>>>>>> collector, there shouldn't mass failures caused by >>>>>>>>>>>>>> options conflicts. But a few failures could be handled >>>>>>>>>>>>>> manually. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Similarly it looks to me like there are tests that >>>>>>>>>>>>>>>>> will fail if you run them with -XX:-UseParallelOldGC >>>>>>>>>>>>>>>>> or -XX:+UseParallelOldGC. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Just a heads up. These two tests will soon be removed. >>>>>>>>>>>>>>>>> I'm about to push a changeset that removes them: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSIncrementalMode.java >>>>>>>>>>>>>>>>> test/gc/startup_warnings/TestCMSNoIncrementalMode.java >>>>>>>>>>>>>>>> okay, thank for letting us know. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Is there some way of making sure that all tests are >>>>>>>>>>>>>>>>> run at one time or another. With this change there is >>>>>>>>>>>>>>>>> a risk that some tests are never run and always >>>>>>>>>>>>>>>>> skipped. Will we somehow be tracking what gets skipped >>>>>>>>>>>>>>>>> and make sure that all tests are at least run once >>>>>>>>>>>>>>>>> with the correct GC so that it is not skipped all the >>>>>>>>>>>>>>>>> time? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This is a very good question! >>>>>>>>>>>>>>>> jtreg now doesn't report skipped tests, hopefully it >>>>>>>>>>>>>>>> will do soon, after getting fix of: >>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/CODETOOLS-7900934 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> And yes, tracking tests which are not run is important >>>>>>>>>>>>>>>> thing. >>>>>>>>>>>>>>>> @requires - is not the only to exclude test from execution. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Other examples: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> /* >>>>>>>>>>>>>>>> *@ignore >>>>>>>>>>>>>>>> *@test >>>>>>>>>>>>>>>> */ >>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> /*@bug 4445555 >>>>>>>>>>>>>>>> *@test >>>>>>>>>>>>>>>> */ >>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>> Such tests will never be run, because jtreg treats as >>>>>>>>>>>>>>>> test only files with @test on the first place... >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> So, making sure that tests do not disappear is >>>>>>>>>>>>>>>> important SQE task, we know about that, we're thinking >>>>>>>>>>>>>>>> on solution (may be very actively). But this subject >>>>>>>>>>>>>>>> for another discussion, not within RFR :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Right. Glad to hear that you are actively working on this! >>>>>>>>>>>>>> >>>>>>>>>>>>>> I was going to say "not very actively", but never mind, >>>>>>>>>>>>>> we know about this problem. With introducing @requires >>>>>>>>>>>>>> mechanism it will become more important! >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your comments! >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- Dima >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Dima >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Bengt >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Evgeniya Stepanova >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> /Evgeniya Stepanova/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /Evgeniya Stepanova/ >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nils.eliasson at oracle.com Fri Nov 14 12:50:38 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 14 Nov 2014 13:50:38 +0100 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out In-Reply-To: <5465D284.2010506@oracle.com> References: <5464BC2A.4020206@oracle.com> <5464C731.4090109@oracle.com> <5464D003.8050907@oracle.com> <5465CEC0.3040707@oracle.com> <5465D284.2010506@oracle.com> Message-ID: <5465FA9E.8000002@oracle.com> On 2014-11-14 10:59, Albert Noll wrote: > Hi Nils, > > > On 11/14/2014 10:43 AM, Nils Eliasson wrote: >> >> On 2014-11-13 16:36, Albert Noll wrote: >>> Hi Nils, >>> >>> On 11/13/2014 03:58 PM, Albert Noll wrote: >>>> Hi Nils, >>>> >>>> CompileQueue::lock() always returns MethodCompileQueue_lock ( see >>>> init_compiler_sweeper_threads() ). It seems that your changes don't >>>> change existing behavior, or am I missing something? Note that we >>>> have 2 compilation queues, but only 1 lock. >>>> >>> It seems that I missed the most important parts of you change >>> indeed. Is it right that >>> Mode evaluation_mode() const { return _no_safepoint; } >>> >>> makes the VM operation not execute at a safepoint? I have a question >>> nevertheless. Why did you choose to return '_no_safepoint' and not >>> '_concurrent'? >> >> The diagnostic commands run in their own thread, and that thread >> still has to wait for the output. Running the command cuncurrently >> would require some care with how the operation is allocated so that >> it can be passed to the VM_thread and be enqueued there. >> > Thanks for the explanation. Thanks for taking a look at this, //Nils > > Best, > Albert >> Regards, >> Nils >> >>> >>> Best, >>> Albert >>> >>>> On 11/13/2014 03:11 PM, Nils Eliasson wrote: >>>>> Hi, >>>>> >>>>> Please review this small change. >>>>> >>>>> 1) Fixing a deadlock in diagnostic command dcmd >>>>> print_compile_queues - between the CompileQueue_lock and >>>>> safepointing. The CompileQueue_lock requires that we are not at a >>>>> safepoint when taking it. Otherwise a java-thread that already >>>>> holds the lock will block when trying to get another lock. In this >>>>> specific case the compiler threads are Java threads, they take the >>>>> CompileQueue_lock frequently and some time takes the >>>>> CompileTaskAlloc_lock after that. >>>>> >>>>> Fixing this by making the diagnostic command not request a safepoint, >>>>> >>>>> 2) Refactoring away the lock() accessor in CompileQueue and >>>>> stating the lock explicitly. This removes an element of surprise >>>>> and makes it easier understanding the code. >>>> What element of surprise are you talking about? Personally, I >>>> prefer the current design, i.e., getting the lock by calling a >>>> function instead of referencing a global variable directly. If we >>>> ever decide to go for a 2 compile queue locks (I think that makes >>>> sense since we have two compilation queues), we would have to >>>> introduce the functions again. >>>> >>>> Best, >>>> Albert >>>> >>>> >>>>> webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8061256 >>>>> >>>>> Thanks, >>>>> Nils Eliasson >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roland.westrelin at oracle.com Fri Nov 14 14:49:10 2014 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 14 Nov 2014 15:49:10 +0100 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: <5464DBAE.2000607@oracle.com> References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> <545BF25E.7010505@oracle.com> <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> <545D5103.9020309@oracle.com> <5463FDD0.4090102@oracle.com> <5464DBAE.2000607@oracle.com> Message-ID: >>> You don't need leftover changes in opaquenode.* files anymore. >> >> I?ll remove the changes. >> >>> jetstream regression may indicate the increase in compilation times. I would suggest to have n->Opcode() == Op_Opaque1 checks in both cases to target only it. Sorry, in my previous mail I incorrectly used Op_CastII in the example code. >> >> >> Are we guaranteed that the type of the limit won?t improve after the opaque node is removed? Maybe it?s some arithmetic computation that isn?t optimized yet? In that case, we want to re-process the phi even after the opaque node is removed as soon as the limit changes. > > My main concern was regressions in your run. Which I think is increase of compilation time. > If you can show that time (-XX:+CITime) does not increase much I am fine with current change. I did another refworkload run with CITime. compile time before the change: jetstream: between 1.521 and 1.806 scimark: between 0.215 and 0.357 jbb2000: between 5.304 and 5.692 jbb2005: between 11.250 and 15.026 jvm98: between 32.062 and 35.720 volano25: between 1.580 and 12.399 compile time after the change: jetstream: between 1.466 and 1.608 scimark: between 0.198 and 0.347 jbb2000: between 5.279 and 5.855 jbb2005: between 11.183 and 14.180 jvm98: between 33.279 and 38.308 volano25: between 1.804 and 10.478 So it looks about the same except maybe a slight increase for jvm98. > And I still want you to run this in aurora.se - I don't want new performance bugs filed 2 months from now. I will do the run. I?ll let you know. Roland. > >> >> Can I go with a single reviewer for this change? > > Ask Igor for second review. > > Thanks, > Vladimir > >> >> Roland. >> >>> >>> Thanks, >>> Vladimir >>> >>> On 11/12/14 2:16 AM, Roland Westrelin wrote: >>>> Vladimir, >>>> >>>> New webrev with your suggested change: >>>> >>>> http://cr.openjdk.java.net/~roland/8054478/webrev.02/ >>>> >>>> I also had to make BoolTest::dump_on available in product builds for that code: >>>> >>>> 138 } else { >>>> 139 stringStream ss; >>>> 140 test.dump_on(&ss); >>>> 141 fatal(err_msg_res("unexpected comparison %s", ss.as_string())); >>>> 142 } >>>> >>>> in castnode.cpp >>>> >>>> I did a refworkload run on x64: >>>> >>>> ============================================================================ >>>> tbase: reference_server >>>> Benchmark Samples Mean Stdev >>>> jetstream 10 102.74 0.10 >>>> Write 10 202.90 0.94 >>>> Parse 10 37.10 0.05 >>>> Read 10 23.20 0.13 >>>> rw.runtime 10 10.60 0.05 >>>> Copy 10 641.70 0.10 >>>> scimark 10 1600.34 0.02 >>>> LU 10 3776.61 0.00 >>>> FFT 10 405.76 0.02 >>>> Monte 10 775.88 0.00 >>>> SOR 10 1275.16 0.01 >>>> Sparse 10 1768.28 0.08 >>>> rw.runtime 10 27.20 0.02 >>>> specjbb2000 10 487411.56 0.04 >>>> Last_Warehouse 10 487411.58 0.04 >>>> First_Warehouse 10 95725.01 0.02 >>>> rw.runtime 10 603.90 0.00 >>>> specjbb2005 10 480939.78 0.01 >>>> last 10 480939.79 0.01 >>>> interval_average 10 5937.40 0.01 >>>> peak 10 534158.07 0.02 >>>> overall_average 10 431366.64 0.02 >>>> last_warehouse 10 8.00 0.00 >>>> peak_warehouse 10 2.30 0.21 >>>> first 10 50951.62 0.03 >>>> rw.runtime 10 461.60 0.00 >>>> specjvm98 10 801.92 0.04 >>>> compress 10 604.68 0.10 >>>> javac 10 405.68 0.17 >>>> db 10 449.78 0.00 >>>> jack 10 631.14 0.05 >>>> mtrt 10 2342.41 0.03 >>>> jess 10 957.35 0.02 >>>> rw.runtime 10 41.10 0.04 >>>> mpegaudio 10 1385.76 0.00 >>>> volano25 10 327418.69 0.06 >>>> time 10 2.45 0.05 >>>> connections 10 400.00 0.00 >>>> rw.runtime 10 26.10 0.01 >>>> -------------------------------------------------------------------------- >>>> Weighted Geomean 31199.92 >>>> ============================================================================ >>>> tnew: reference_server >>>> Benchmark Samples Mean Stdev %Diff P Significant >>>> jetstream 10 85.82 0.12 -16.47 0.002 Yes >>>> Write 10 219.60 0.31 -8.23 0.799 * >>>> Parse 10 36.90 0.06 0.54 0.836 * >>>> Read 10 23.60 0.10 -1.72 0.741 * >>>> rw.runtime 10 13.80 0.03 -30.19 0.000 Yes >>>> Copy 10 1058.70 0.26 -64.98 0.001 Yes >>>> scimark 10 1583.22 0.00 -1.07 0.110 * >>>> LU 10 3778.49 0.00 0.05 0.160 * >>>> FFT 10 407.80 0.01 0.50 0.498 * >>>> Monte 10 775.71 0.00 -0.02 0.279 * >>>> SOR 10 1276.75 0.01 0.12 0.742 * >>>> Sparse 10 1677.37 0.01 -5.14 0.086 * >>>> rw.runtime 10 27.50 0.02 -1.10 0.265 * >>>> specjbb2000 10 491672.03 0.04 0.87 0.624 * >>>> Last_Warehouse 10 491672.04 0.04 0.87 0.624 * >>>> First_Warehouse 10 94172.03 0.02 -1.62 0.050 * >>>> rw.runtime 10 604.00 0.00 -0.02 0.585 * >>>> specjbb2005 10 481051.28 0.01 0.02 0.945 * >>>> last 10 481051.29 0.01 0.02 0.945 * >>>> interval_average 10 5938.90 0.01 0.03 0.940 * >>>> peak 10 538706.19 0.03 0.85 0.461 * >>>> overall_average 10 434244.96 0.02 0.67 0.535 * >>>> last_warehouse 10 8.00 0.00 -0.00 0.000 * >>>> peak_warehouse 10 2.00 0.00 13.04 0.081 * >>>> first 10 51039.34 0.03 0.17 0.889 * >>>> rw.runtime 10 462.20 0.00 -0.13 0.120 * >>>> specjvm98 10 806.31 0.04 0.55 0.738 * >>>> compress 10 623.61 0.10 3.13 0.494 * >>>> javac 10 402.37 0.10 -0.81 0.898 * >>>> db 10 450.58 0.00 0.18 0.327 * >>>> jack 10 627.02 0.05 -0.65 0.785 * >>>> mtrt 10 2281.86 0.03 -2.58 0.106 * >>>> jess 10 1000.65 0.10 4.52 0.220 * >>>> rw.runtime 10 40.90 0.03 0.49 0.761 * >>>> mpegaudio 10 1384.19 0.00 -0.11 0.514 * >>>> volano25 10 324906.00 0.08 -0.77 0.799 * >>>> time 10 2.47 0.07 -1.00 0.733 * >>>> connections 10 400.00 0.00 0.00 0.000 * >>>> rw.runtime 10 26.50 0.02 -1.53 0.058 * >>>> -------------------------------------------------------------------------- >>>> Weighted Geomean 30613.61 -1.88 >>>> ============================================================================ >>>> >>>> tbase is current hotspot-comp. tnew is with the change. It?s ok, right? >>>> >>>> Roland. >>>> >>>> >>>>> On Nov 8, 2014, at 12:08 AM, Vladimir Kozlov wrote: >>>>> >>>>> This looks good but I just realized that what you are doing in Opaque1Node::Ideal() we usually do in PhaseIterGVN::add_users_to_worklist(). >>>>> >>>>> For example you can do there (for Phi): >>>>> >>>>> + uint use_op = use->Opcode(); >>>>> if( use->is_Cmp() ) { // Enable CMP/BOOL optimization >>>>> add_users_to_worklist(use); // Put Bool on worklist >>>>> - // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >>>>> - // phi merging either 0 or 1 onto the worklist >>>>> if (use->outcnt() > 0) { >>>>> Node* bol = use->raw_out(0); >>>>> if (bol->outcnt() > 0) { >>>>> Node* iff = bol->raw_out(0); >>>>> if (use_op == Op_CmpI && iff->is_CountedLoopEnd() && >>>>> n->Opcode() == Op_CastII) { >>>>> + // If this opaque node feeds into the limit condition of a >>>>> + // CountedLoop, we need to process the Phi node for the >>>>> + // induction variable: the range of values taken by the Phi is >>>>> + // known now and so its type is also known. >>>>> + CountedLoopEndNode* cle = iff->as_CountedLoopEnd(); >>>>> + if (cle->limit() == this) { >>>>> + _worklist.push(cle->phi()); >>>>> + } >>>>> - if (iff->outcnt() == 2) { >>>>> + } else if (iff->outcnt() == 2) { >>>>> + // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >>>>> + // phi merging either 0 or 1 onto the worklist >>>>> >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> >>>>> On 11/7/14 2:17 PM, Roland Westrelin wrote: >>>>>> Vladimir, >>>>>> >>>>>> Thanks for the discussion, suggestions and fixes. Here is an updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~roland/8054478/webrev.01/ >>>>>> >>>>>> Roland. >>>>>> >>>>>>> On Nov 6, 2014, at 11:12 PM, Vladimir Kozlov wrote: >>>>>>> >>>>>>> On 11/6/14 7:39 AM, Roland Westrelin wrote: >>>>>>>> >>>>>>>>> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. >>>>>>>> >>>>>>>> I?ll send an updated webrev. >>>>>>>> >>>>>>>>>>>>> opaquenode.cpp >>>>>>>>>>>>> >>>>>>>>>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>>>>>>>>> >>>>>>>>>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>>>>>>>>> >>>>>>>>>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>>>>>>>>> >>>>>>>>>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >>>>>>>>> >>>>>>>>> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. >>>>>>>> >>>>>>>> (I know you don?t like when I remove the text from previous emails but that was really getting confusing) >>>>>>> >>>>>>> At least leave webrev link so I don't need to search for it in previous mails. >>>>>>> >>>>>>>> >>>>>>>> PhiNode::Value() has this code: >>>>>>>> >>>>>>>> // Check for trip-counted loop. If so, be smarter. >>>>>>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>>>>>> if( l && l->can_be_counted_loop(phase) && >>>>>>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>>>>>> // protect against init_trip() or limit() returning NULL >>>>>>>> const Node *init = l->init_trip(); >>>>>>>> const Node *limit = l->limit(); >>>>>>>> if( init != NULL && limit != NULL && l->stride_is_con() ) { >>>>>>>> const TypeInt *lo = init ->bottom_type()->isa_int(); >>>>>>>> const TypeInt *hi = limit->bottom_type()->isa_int(); >>>>>>>> if( lo && hi ) { // Dying loops might have TOP here >>>>>>>> int stride = l->stride_con(); >>>>>>>> if( stride < 0 ) { // Down-counter loop >>>>>>>> const TypeInt *tmp = lo; lo = hi; hi = tmp; >>>>>>>> stride = -stride; >>>>>>>> } >>>>>>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>>>>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>>>>>> } >>>>>>>> } >>>>>>>> } >>>>>>>> >>>>>>>> That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. >>>>>>> >>>>>>> My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: >>>>>>> >>>>>>> // Step B2: Build a zero-trip guard for the main-loop. After leaving the >>>>>>> // pre-loop, the main-loop may not execute at all. Later in life this >>>>>>> // zero-trip guard will become the minimum-trip guard when we unroll >>>>>>> // the main-loop. >>>>>>> Node *min_opaq = new Opaque1Node(C, limit); >>>>>>> Node *min_cmp = new CmpINode( pre_incr, min_opaq ); >>>>>>> >>>>>>> >>>>>>> But you are talking about pre-loop exit check: >>>>>>> >>>>>>> // Step B4: Shorten the pre-loop to run only 1 iteration (for now). >>>>>>> // RCE and alignment may change this later. >>>>>>> Node *cmp_end = pre_end->cmp_node(); >>>>>>> assert( cmp_end->in(2) == limit, "" ); >>>>>>> Node *pre_limit = new AddINode( init, stride ); >>>>>>> >>>>>>> // Save the original loop limit in this Opaque1 node for >>>>>>> // use by range check elimination. >>>>>>> Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); >>>>>>> >>>>>>> >>>>>>> But in this case you can add check (cl->limit() == this) to be clear what you are looking for. >>>>>>> Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>>> >>>>>>>> In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. >>>>>>>> >>>>>>>> Roland. From andreas.eriksson at oracle.com Fri Nov 14 14:50:45 2014 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Fri, 14 Nov 2014 15:50:45 +0100 Subject: RFR: JDK7 Backport of JDK-8058583: Remove CompilationRepeat Message-ID: <546616C5.1060606@oracle.com> Hi, Please review this jdk7 backport of JDK-8058583 which removes the compilation repeat flag. The only difference to the jdk8 changeset is that the obsolete version for the flag is 7u80, not 8. Webrev: http://cr.openjdk.java.net/~aeriksso/8058583/webrev.00/ For reference: JDK8 changeset: http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/rev/b9c94af14fd0 Regards, Andreas From nils.eliasson at oracle.com Fri Nov 14 14:52:22 2014 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Fri, 14 Nov 2014 15:52:22 +0100 Subject: RFR: JDK7 Backport of JDK-8058583: Remove CompilationRepeat In-Reply-To: <546616C5.1060606@oracle.com> References: <546616C5.1060606@oracle.com> Message-ID: <54661726.4050009@oracle.com> Hi, Looks good, Regards, Nils On 2014-11-14 15:50, Andreas Eriksson wrote: > Hi, > > Please review this jdk7 backport of JDK-8058583 which removes the > compilation repeat flag. > The only difference to the jdk8 changeset is that the obsolete version > for the flag is 7u80, not 8. > > Webrev: > http://cr.openjdk.java.net/~aeriksso/8058583/webrev.00/ > > For reference: > JDK8 changeset: > http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/rev/b9c94af14fd0 > > Regards, > Andreas From igor.ignatyev at oracle.com Fri Nov 14 15:03:38 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 14 Nov 2014 18:03:38 +0300 Subject: RFR(M) : 8064669 : compiler/whitebox/AllocationCodeBlobTest.java crashes / asserts Message-ID: <546619CA.7080801@oracle.com> http://cr.openjdk.java.net/~iignatyev/8064669/webrev.00/ 261 lines changed: 231 ins; 22 del; 8 mod; Hi all, Please review patch: Problems: 0. NMethodSweeper::possibly_sweep is executed outside CodeCacheSweeperThread 1. DummyBlob is initialized after the lock is released, so other threads are able to see uninitialized blob 2. NMethodSweeper::sweep_code_cache can pass null to process_nmethod, this also can lead to a crash. this situation can happen if safepoint happens during sweeping and after safepoint another thread iterates over all nmethods. Fix: 0. WB_ForceNMethodSweep use CodeCacheSweeperThread to execute possibly_sweep 1. move ctor invocation to locked section 2. handle safepoint after processing nmethod in NMethodSweeper::sweep_code_cache 3. added new test to verify that forceNMethodSweep actually work 4. AllocationCodeBlobTest is modified - to checks that deoptimization works fine w/ dummy blobs - to execute forceNMethodSweep more often to increase a chance to catch such kind problems jbs:https://bugs.openjdk.java.net/browse/JDK-8064669 testing: jprt + compiler/whitebox tests -- Igor From albert.noll at oracle.com Fri Nov 14 15:25:42 2014 From: albert.noll at oracle.com (Albert Noll) Date: Fri, 14 Nov 2014 16:25:42 +0100 Subject: RFR(M) : 8064669 : compiler/whitebox/AllocationCodeBlobTest.java crashes / asserts In-Reply-To: <546619CA.7080801@oracle.com> References: <546619CA.7080801@oracle.com> Message-ID: <54661EF6.8090705@oracle.com> Hi Igor, I took a quick look at your fix and the changes seem fine to me (not a reviewer). Thanks a lot for taking care of this. Best, Albert On 11/14/2014 04:03 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8064669/webrev.00/ > 261 lines changed: 231 ins; 22 del; 8 mod; > > Hi all, > > Please review patch: > > Problems: > 0. NMethodSweeper::possibly_sweep is executed outside > CodeCacheSweeperThread > 1. DummyBlob is initialized after the lock is released, so other > threads are able to see uninitialized blob > 2. NMethodSweeper::sweep_code_cache can pass null to process_nmethod, > this also can lead to a crash. this situation can happen if safepoint > happens during sweeping and after safepoint another thread iterates > over all nmethods. > > Fix: > 0. WB_ForceNMethodSweep use CodeCacheSweeperThread to execute > possibly_sweep > 1. move ctor invocation to locked section > 2. handle safepoint after processing nmethod in > NMethodSweeper::sweep_code_cache > > 3. added new test to verify that forceNMethodSweep actually work > 4. AllocationCodeBlobTest is modified > - to checks that deoptimization works fine w/ dummy blobs > - to execute forceNMethodSweep more often to increase a chance to > catch such kind problems > > jbs:https://bugs.openjdk.java.net/browse/JDK-8064669 > testing: jprt + compiler/whitebox tests From vladimir.kozlov at oracle.com Fri Nov 14 16:35:41 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Nov 2014 08:35:41 -0800 Subject: [9] RFR(L): 8062854: move compiler jtreg test to corresponding subfolders and use those in TEST.groups In-Reply-To: <5465B2C7.1010205@oracle.com> References: <5464B2FB.1050606@oracle.com> <5464BC57.80200@oracle.com> <5464DC87.1070301@oracle.com> <5464DE9A.8090105@oracle.com> <5464E02E.7060406@oracle.com> <5464E1BF.5050208@oracle.com> <5465B2C7.1010205@oracle.com> Message-ID: <54662F5D.7060901@oracle.com> Looks good. Thanks, Vladimir On 11/13/14 11:44 PM, Zolt?n Maj? wrote: > Hi Vladimir, > > > thank you for the feedback. > > On 11/13/2014 05:52 PM, Vladimir Kozlov wrote: >> On 11/13/14 8:45 AM, Zolt?n Maj? wrote: >>> Hi, >>> >>> >>> On 11/13/2014 05:38 PM, Vladimir Kozlov wrote: >>>> This is good approach. I see you removed all 5091921 tests when before you ran most of them. What is the reason? Can >>>> you still run some? >>> >>> I removed 5091921 test because >>> >>> - 3 of them have a high execution time and >>> - I wanted to keep the exclude list simple (not exclude many individual tests). >>> >>> But if you like, we can include short-running 5091921 tests again (for example into hotspot_compiler_1). That would >>> result in around 1 min extra time and 3 tests excluded. >> >> Yes, please do that. > > I added the short-running 5091921 tests to hotspot_compiler_1. Here are the results (on solaris_sparcv9 cpus=6 > parallelcount=6 cpufreqmhz=2848): > > before: 107 tests / work=15m > after: 121 tests / work=16m14s > > Here is the updated webrev: http://cr.openjdk.java.net/~zmajo/8062854/webrev.02/ > > Thank you and best regards, > > > Zoltan > >> >> Thanks, >> Vladimir >> >>> >>> Thank you and best regards, >>> >>> >>> Zoltan >>> >>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/13/14 8:29 AM, Zolt?n Maj? wrote: >>>>> Hi, >>>>> >>>>> >>>>> On 11/13/2014 03:12 PM, Zolt?n Maj? wrote: >>>>>> Here is only the "work" time for the same runs as before: >>>>>> >>>>>> hotspot_compiler_1 / work=15m6s >>>>>> hotspot_compiler_2 / work=13m55s >>>>>> hotspot_compiler_3 / work=12m43s >>>>> >>>>> I updated the test list. Here are the results: >>>>> >>>>> group / #tests / "work" time on solaris_sparcv9 cpus=6 parallelcount=6 cpufreqmhz=2848 >>>>> >>>>> hotspot_compiler_1 / 107 / work=15m >>>>> hotspot_compiler_2 / 79 / work=15m2s >>>>> hotspot_compiler_3 / 62 / work=14m14s >>>>> >>>>> Here is the updated webrev: http://cr.openjdk.java.net/~zmajo/8062854/webrev.01/ >>>>> >>>>> Thank you and best regards, >>>>> >>>>> >>>>> Zoltan >>>>> >>> > From vladimir.kozlov at oracle.com Fri Nov 14 17:10:32 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Nov 2014 09:10:32 -0800 Subject: RFR: JDK7 Backport of JDK-8058583: Remove CompilationRepeat In-Reply-To: <546616C5.1060606@oracle.com> References: <546616C5.1060606@oracle.com> Message-ID: <54663788.8070004@oracle.com> Looks good. Thanks, Vladimir On 11/14/14 6:50 AM, Andreas Eriksson wrote: > Hi, > > Please review this jdk7 backport of JDK-8058583 which removes the compilation repeat flag. > The only difference to the jdk8 changeset is that the obsolete version for the flag is 7u80, not 8. > > Webrev: > http://cr.openjdk.java.net/~aeriksso/8058583/webrev.00/ > > For reference: > JDK8 changeset: http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/rev/b9c94af14fd0 > > Regards, > Andreas From vladimir.kozlov at oracle.com Fri Nov 14 17:16:05 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Nov 2014 09:16:05 -0800 Subject: RFR(M): 8054478 C2: Incorrectly compiled char[] array access crashes JVM In-Reply-To: References: <523201FA-F50D-4112-B4B4-DA61B6DFCDB4@oracle.com> <5040216D-84CA-4CCF-82CF-5D15DF4B7D08@oracle.com> <54581D5E.2090706@oracle.com> <5459370A.1060305@oracle.com> <9F9B8687-A597-4BF4-9E91-BC39D1D94688@oracle.com> <545A577F.5080805@oracle.com> <92F3888F-CD60-447A-9B69-C9FF7B48368E@oracle.com> <545BF25E.7010505@oracle.com> <3DD265F5-2E0D-44E2-AE9B-9F2B24232FBE@oracle.com> <545D5103.9020309@oracle.com> <5463FDD0.4090102@oracle.com> <5464DBAE.2000607@oracle.com> Message-ID: <546638D5.6010408@oracle.com> Agree, times are reasonable. Changes are good if aurora.se does not show significant regression. Thanks, Vladimir On 11/14/14 6:49 AM, Roland Westrelin wrote: > >>>> You don't need leftover changes in opaquenode.* files anymore. >>> >>> I?ll remove the changes. >>> >>>> jetstream regression may indicate the increase in compilation times. I would suggest to have n->Opcode() == Op_Opaque1 checks in both cases to target only it. Sorry, in my previous mail I incorrectly used Op_CastII in the example code. >>> >>> >>> Are we guaranteed that the type of the limit won?t improve after the opaque node is removed? Maybe it?s some arithmetic computation that isn?t optimized yet? In that case, we want to re-process the phi even after the opaque node is removed as soon as the limit changes. >> >> My main concern was regressions in your run. Which I think is increase of compilation time. >> If you can show that time (-XX:+CITime) does not increase much I am fine with current change. > > I did another refworkload run with CITime. > > compile time before the change: > jetstream: between 1.521 and 1.806 > scimark: between 0.215 and 0.357 > jbb2000: between 5.304 and 5.692 > jbb2005: between 11.250 and 15.026 > jvm98: between 32.062 and 35.720 > volano25: between 1.580 and 12.399 > > compile time after the change: > jetstream: between 1.466 and 1.608 > scimark: between 0.198 and 0.347 > jbb2000: between 5.279 and 5.855 > jbb2005: between 11.183 and 14.180 > jvm98: between 33.279 and 38.308 > volano25: between 1.804 and 10.478 > > So it looks about the same except maybe a slight increase for jvm98. > >> And I still want you to run this in aurora.se - I don't want new performance bugs filed 2 months from now. > > I will do the run. I?ll let you know. > > Roland. > >> >>> >>> Can I go with a single reviewer for this change? >> >> Ask Igor for second review. >> >> Thanks, >> Vladimir >> >>> >>> Roland. >>> >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/12/14 2:16 AM, Roland Westrelin wrote: >>>>> Vladimir, >>>>> >>>>> New webrev with your suggested change: >>>>> >>>>> http://cr.openjdk.java.net/~roland/8054478/webrev.02/ >>>>> >>>>> I also had to make BoolTest::dump_on available in product builds for that code: >>>>> >>>>> 138 } else { >>>>> 139 stringStream ss; >>>>> 140 test.dump_on(&ss); >>>>> 141 fatal(err_msg_res("unexpected comparison %s", ss.as_string())); >>>>> 142 } >>>>> >>>>> in castnode.cpp >>>>> >>>>> I did a refworkload run on x64: >>>>> >>>>> ============================================================================ >>>>> tbase: reference_server >>>>> Benchmark Samples Mean Stdev >>>>> jetstream 10 102.74 0.10 >>>>> Write 10 202.90 0.94 >>>>> Parse 10 37.10 0.05 >>>>> Read 10 23.20 0.13 >>>>> rw.runtime 10 10.60 0.05 >>>>> Copy 10 641.70 0.10 >>>>> scimark 10 1600.34 0.02 >>>>> LU 10 3776.61 0.00 >>>>> FFT 10 405.76 0.02 >>>>> Monte 10 775.88 0.00 >>>>> SOR 10 1275.16 0.01 >>>>> Sparse 10 1768.28 0.08 >>>>> rw.runtime 10 27.20 0.02 >>>>> specjbb2000 10 487411.56 0.04 >>>>> Last_Warehouse 10 487411.58 0.04 >>>>> First_Warehouse 10 95725.01 0.02 >>>>> rw.runtime 10 603.90 0.00 >>>>> specjbb2005 10 480939.78 0.01 >>>>> last 10 480939.79 0.01 >>>>> interval_average 10 5937.40 0.01 >>>>> peak 10 534158.07 0.02 >>>>> overall_average 10 431366.64 0.02 >>>>> last_warehouse 10 8.00 0.00 >>>>> peak_warehouse 10 2.30 0.21 >>>>> first 10 50951.62 0.03 >>>>> rw.runtime 10 461.60 0.00 >>>>> specjvm98 10 801.92 0.04 >>>>> compress 10 604.68 0.10 >>>>> javac 10 405.68 0.17 >>>>> db 10 449.78 0.00 >>>>> jack 10 631.14 0.05 >>>>> mtrt 10 2342.41 0.03 >>>>> jess 10 957.35 0.02 >>>>> rw.runtime 10 41.10 0.04 >>>>> mpegaudio 10 1385.76 0.00 >>>>> volano25 10 327418.69 0.06 >>>>> time 10 2.45 0.05 >>>>> connections 10 400.00 0.00 >>>>> rw.runtime 10 26.10 0.01 >>>>> -------------------------------------------------------------------------- >>>>> Weighted Geomean 31199.92 >>>>> ============================================================================ >>>>> tnew: reference_server >>>>> Benchmark Samples Mean Stdev %Diff P Significant >>>>> jetstream 10 85.82 0.12 -16.47 0.002 Yes >>>>> Write 10 219.60 0.31 -8.23 0.799 * >>>>> Parse 10 36.90 0.06 0.54 0.836 * >>>>> Read 10 23.60 0.10 -1.72 0.741 * >>>>> rw.runtime 10 13.80 0.03 -30.19 0.000 Yes >>>>> Copy 10 1058.70 0.26 -64.98 0.001 Yes >>>>> scimark 10 1583.22 0.00 -1.07 0.110 * >>>>> LU 10 3778.49 0.00 0.05 0.160 * >>>>> FFT 10 407.80 0.01 0.50 0.498 * >>>>> Monte 10 775.71 0.00 -0.02 0.279 * >>>>> SOR 10 1276.75 0.01 0.12 0.742 * >>>>> Sparse 10 1677.37 0.01 -5.14 0.086 * >>>>> rw.runtime 10 27.50 0.02 -1.10 0.265 * >>>>> specjbb2000 10 491672.03 0.04 0.87 0.624 * >>>>> Last_Warehouse 10 491672.04 0.04 0.87 0.624 * >>>>> First_Warehouse 10 94172.03 0.02 -1.62 0.050 * >>>>> rw.runtime 10 604.00 0.00 -0.02 0.585 * >>>>> specjbb2005 10 481051.28 0.01 0.02 0.945 * >>>>> last 10 481051.29 0.01 0.02 0.945 * >>>>> interval_average 10 5938.90 0.01 0.03 0.940 * >>>>> peak 10 538706.19 0.03 0.85 0.461 * >>>>> overall_average 10 434244.96 0.02 0.67 0.535 * >>>>> last_warehouse 10 8.00 0.00 -0.00 0.000 * >>>>> peak_warehouse 10 2.00 0.00 13.04 0.081 * >>>>> first 10 51039.34 0.03 0.17 0.889 * >>>>> rw.runtime 10 462.20 0.00 -0.13 0.120 * >>>>> specjvm98 10 806.31 0.04 0.55 0.738 * >>>>> compress 10 623.61 0.10 3.13 0.494 * >>>>> javac 10 402.37 0.10 -0.81 0.898 * >>>>> db 10 450.58 0.00 0.18 0.327 * >>>>> jack 10 627.02 0.05 -0.65 0.785 * >>>>> mtrt 10 2281.86 0.03 -2.58 0.106 * >>>>> jess 10 1000.65 0.10 4.52 0.220 * >>>>> rw.runtime 10 40.90 0.03 0.49 0.761 * >>>>> mpegaudio 10 1384.19 0.00 -0.11 0.514 * >>>>> volano25 10 324906.00 0.08 -0.77 0.799 * >>>>> time 10 2.47 0.07 -1.00 0.733 * >>>>> connections 10 400.00 0.00 0.00 0.000 * >>>>> rw.runtime 10 26.50 0.02 -1.53 0.058 * >>>>> -------------------------------------------------------------------------- >>>>> Weighted Geomean 30613.61 -1.88 >>>>> ============================================================================ >>>>> >>>>> tbase is current hotspot-comp. tnew is with the change. It?s ok, right? >>>>> >>>>> Roland. >>>>> >>>>> >>>>>> On Nov 8, 2014, at 12:08 AM, Vladimir Kozlov wrote: >>>>>> >>>>>> This looks good but I just realized that what you are doing in Opaque1Node::Ideal() we usually do in PhaseIterGVN::add_users_to_worklist(). >>>>>> >>>>>> For example you can do there (for Phi): >>>>>> >>>>>> + uint use_op = use->Opcode(); >>>>>> if( use->is_Cmp() ) { // Enable CMP/BOOL optimization >>>>>> add_users_to_worklist(use); // Put Bool on worklist >>>>>> - // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >>>>>> - // phi merging either 0 or 1 onto the worklist >>>>>> if (use->outcnt() > 0) { >>>>>> Node* bol = use->raw_out(0); >>>>>> if (bol->outcnt() > 0) { >>>>>> Node* iff = bol->raw_out(0); >>>>>> if (use_op == Op_CmpI && iff->is_CountedLoopEnd() && >>>>>> n->Opcode() == Op_CastII) { >>>>>> + // If this opaque node feeds into the limit condition of a >>>>>> + // CountedLoop, we need to process the Phi node for the >>>>>> + // induction variable: the range of values taken by the Phi is >>>>>> + // known now and so its type is also known. >>>>>> + CountedLoopEndNode* cle = iff->as_CountedLoopEnd(); >>>>>> + if (cle->limit() == this) { >>>>>> + _worklist.push(cle->phi()); >>>>>> + } >>>>>> - if (iff->outcnt() == 2) { >>>>>> + } else if (iff->outcnt() == 2) { >>>>>> + // Look for the 'is_x2logic' pattern: "x ? : 0 : 1" and put the >>>>>> + // phi merging either 0 or 1 onto the worklist >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> >>>>>> On 11/7/14 2:17 PM, Roland Westrelin wrote: >>>>>>> Vladimir, >>>>>>> >>>>>>> Thanks for the discussion, suggestions and fixes. Here is an updated webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~roland/8054478/webrev.01/ >>>>>>> >>>>>>> Roland. >>>>>>> >>>>>>>> On Nov 6, 2014, at 11:12 PM, Vladimir Kozlov wrote: >>>>>>>> >>>>>>>> On 11/6/14 7:39 AM, Roland Westrelin wrote: >>>>>>>>> >>>>>>>>>> I need to see new version of webrev. We talked about moving type change from Ideal() to Value(). You are right if the code, currently in ConstraintCastNode::Value(), will be executed first. >>>>>>>>> >>>>>>>>> I?ll send an updated webrev. >>>>>>>>> >>>>>>>>>>>>>> opaquenode.cpp >>>>>>>>>>>>>> >>>>>>>>>>>>>> What test you are talking about: "The pre loop is guarded by a test on an opaque node which is later removed"? I did not get first part of the code. You are putting on worklist a Phi from *previous* (pre-loop) loop. I would understand if you do that for the following (guarded main-, post-) loop, and that is already taking care by putting CastII on worklist. >>>>>>>>>>>>> >>>>>>>>>>>>> Once the range of values for the pre loop is know, we can optimize the test that guards the main loop. That range of values is only known once the opaque node for the pre loop is removed. >>>>>>>>>>>> >>>>>>>>>>>> That is what I am asking: "range of values for the pre loop is know" - when this happens, which Opaque1 is removed to make "range" to be known? If it is Opaque1 node from loop_limit_check predicate then we may need to make sure that iteration Phi of pre-loop is put on worklist when predicate's Opaque1 node is removed by cleanup_loop_predicates(). Then you don't need first part in Opaque1Node::Ideal. >>>>>>>>>>> >>>>>>>>>>> The Opaque1 nodes are the ones created by PhaseIdealLoop::insert_pre_post_loops() (loop limit checks). They are not in the Compile::_predicate_opaqs list and so they are not removed by cleanup_loop_predicates(). >>>>>>>>>> >>>>>>>>>> So how these Opaque1 nodes affects type range of Phi node in pre-loop? That is what I don't understand. >>>>>>>>> >>>>>>>>> (I know you don?t like when I remove the text from previous emails but that was really getting confusing) >>>>>>>> >>>>>>>> At least leave webrev link so I don't need to search for it in previous mails. >>>>>>>> >>>>>>>>> >>>>>>>>> PhiNode::Value() has this code: >>>>>>>>> >>>>>>>>> // Check for trip-counted loop. If so, be smarter. >>>>>>>>> CountedLoopNode *l = r->is_CountedLoop() ? r->as_CountedLoop() : NULL; >>>>>>>>> if( l && l->can_be_counted_loop(phase) && >>>>>>>>> ((const Node*)l->phi() == this) ) { // Trip counted loop! >>>>>>>>> // protect against init_trip() or limit() returning NULL >>>>>>>>> const Node *init = l->init_trip(); >>>>>>>>> const Node *limit = l->limit(); >>>>>>>>> if( init != NULL && limit != NULL && l->stride_is_con() ) { >>>>>>>>> const TypeInt *lo = init ->bottom_type()->isa_int(); >>>>>>>>> const TypeInt *hi = limit->bottom_type()->isa_int(); >>>>>>>>> if( lo && hi ) { // Dying loops might have TOP here >>>>>>>>> int stride = l->stride_con(); >>>>>>>>> if( stride < 0 ) { // Down-counter loop >>>>>>>>> const TypeInt *tmp = lo; lo = hi; hi = tmp; >>>>>>>>> stride = -stride; >>>>>>>>> } >>>>>>>>> if( lo->_hi < hi->_lo ) // Reversed endpoints are well defined :-( >>>>>>>>> return TypeInt::make(lo->_lo,hi->_hi,3); >>>>>>>>> } >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> That code can only return something for the induction variable Phi once the opaque node for the loop limit check is removed. That?s why when the opaque node is removed I enqueue the induction variable Phi. >>>>>>>> >>>>>>>> My mistake was that I thought you are looking on Opaque1 node generated for main-loop guard: >>>>>>>> >>>>>>>> // Step B2: Build a zero-trip guard for the main-loop. After leaving the >>>>>>>> // pre-loop, the main-loop may not execute at all. Later in life this >>>>>>>> // zero-trip guard will become the minimum-trip guard when we unroll >>>>>>>> // the main-loop. >>>>>>>> Node *min_opaq = new Opaque1Node(C, limit); >>>>>>>> Node *min_cmp = new CmpINode( pre_incr, min_opaq ); >>>>>>>> >>>>>>>> >>>>>>>> But you are talking about pre-loop exit check: >>>>>>>> >>>>>>>> // Step B4: Shorten the pre-loop to run only 1 iteration (for now). >>>>>>>> // RCE and alignment may change this later. >>>>>>>> Node *cmp_end = pre_end->cmp_node(); >>>>>>>> assert( cmp_end->in(2) == limit, "" ); >>>>>>>> Node *pre_limit = new AddINode( init, stride ); >>>>>>>> >>>>>>>> // Save the original loop limit in this Opaque1 node for >>>>>>>> // use by range check elimination. >>>>>>>> Node *pre_opaq = new Opaque1Node(C, pre_limit, limit); >>>>>>>> >>>>>>>> >>>>>>>> But in this case you can add check (cl->limit() == this) to be clear what you are looking for. >>>>>>>> Also instead of looking up through AddI you can look down for CountedLoopEnd and get cle->phi() and cle->limit() from it. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimir >>>>>>>> >>>>>>>>> >>>>>>>>> In the case of this test case, the pre loop iterates from 0 as long as i > -1. So the Phi for the pre loop has values within [-1, 0]. The main loop is guarded by a test that checks whether The value from the Phi - 1 is greater than 0. With a correct range of values for the Phi from the code above, that checks is statically false and the main loop is optimized away. Otherwise it?s not. >>>>>>>>> >>>>>>>>> Roland. > From vladimir.kozlov at oracle.com Fri Nov 14 18:01:02 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Nov 2014 10:01:02 -0800 Subject: RFR(S): 8061256: com/sun/management/DiagnosticCommandMBean/DcmdMBeanPermissionsTest.java timed out In-Reply-To: <5464BC2A.4020206@oracle.com> References: <5464BC2A.4020206@oracle.com> Message-ID: <5466435E.7050605@oracle.com> Hi Nils, Looks like you answered Albert's questions. I am not happy with refactoring lock() accessor for the same reason as Albert said. But I understand why you did it and fine with that. We can investigate separate locks for queues but I don't think it is a bottleneck currently. Also some parts of code relay on one lock. Thanks, Vladimir On 11/13/14 6:11 AM, Nils Eliasson wrote: > Hi, > > Please review this small change. > > 1) Fixing a deadlock in diagnostic command dcmd print_compile_queues - between the CompileQueue_lock and safepointing. > The CompileQueue_lock requires that we are not at a safepoint when taking it. Otherwise a java-thread that already holds > the lock will block when trying to get another lock. In this specific case the compiler threads are Java threads, they > take the CompileQueue_lock frequently and some time takes the CompileTaskAlloc_lock after that. > > Fixing this by making the diagnostic command not request a safepoint, > > 2) Refactoring away the lock() accessor in CompileQueue and stating the lock explicitly. This removes an element of > surprise and makes it easier understanding the code. > > webrev: http://cr.openjdk.java.net/~neliasso/8061256/webrev.02/ > bug: https://bugs.openjdk.java.net/browse/JDK-8061256 > > Thanks, > Nils Eliasson > From vladimir.kozlov at oracle.com Fri Nov 14 18:02:33 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Nov 2014 10:02:33 -0800 Subject: [9] RFR(XS): 8059596: VM startup fails with 'Invalid code heap sizes' if -XX:ReservedCodeCacheSize is set In-Reply-To: <5465EA51.8050802@oracle.com> References: <542D1847.7090107@oracle.com> <5465574F.2040401@oracle.com> <5465EA51.8050802@oracle.com> Message-ID: <546643B9.2080801@oracle.com> Thank you, Tobias Vladimir On 11/14/14 3:41 AM, Tobias Hartmann wrote: > Hi Vladimir, > > On 14.11.2014 02:13, Vladimir Kozlov wrote: >> Hi Tobias, >> >> At the line 1149 we set ReservedCodeCacheSize to ERGO so it is not DEFAULT anymore: >> >> 1149 FLAG_SET_ERGO(uintx, ReservedCodeCacheSize, ReservedCodeCacheSize * 5); >> >> As result segments sizes are not set there. They will be set to in >> CodeCache::initialize_heaps() as (ReservedCodeCacheSize - >> NonNMethodCodeHeapSize) / 2: >> >> CodeHeap 'non-nmethods': size=5700Kb used=2278Kb max_used=2279Kb free=3421Kb >> CodeHeap 'profiled nmethods': size=120032Kb used=120Kb max_used=120Kb free=119912Kb >> CodeHeap 'non-profiled nmethods': size=120032Kb used=22Kb max_used=22Kb free=120 >> >> But it is not *5 sizes: >> >> define_pd_global(intx, NonProfiledCodeHeapSize, 21*M); >> define_pd_global(intx, ProfiledCodeHeapSize, 22*M); >> >> And it skips the assert in arguments.cpp > > Thanks for pointing that out. As Vladimir I. already suggested, I will move the > logic into CodeCache::initialize_heaps() and check consistency there. > > I filed JDK-8059611 for that. > > Thanks, > Tobias > >> >> Vladimir >> >> On 10/2/14 2:17 AM, Tobias Hartmann wrote: >>> Hi, >>> >>> please review this small patch. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8059596 >>> Webrev: http://cr.openjdk.java.net/~thartmann/8059596/webrev.00/ >>> >>> Problem: >>> The VM startup fails with 'Invalid code heap sizes' if >>> -XX:ReservedCodeCacheSize >= 240M is specified. The problem is that in >>> Arguments::set_tiered_flags() the code cache size is increased by 5 if >>> TieredCompilation is enabled. This should only be done for default values. >>> >>> Solution: >>> Add missing FLAG_IS_DEFAULT(ReservedCodeCacheSize) check. >>> >>> Thanks, >>> Tobias From john.r.rose at oracle.com Fri Nov 14 18:38:46 2014 From: john.r.rose at oracle.com (John Rose) Date: Fri, 14 Nov 2014 10:38:46 -0800 Subject: [9] RFR(S): 8050079: crash while compiling java.lang.ref.Finalizer::runFinalizer In-Reply-To: <5464C576.7040705@oracle.com> References: <5464C576.7040705@oracle.com> Message-ID: On Nov 13, 2014, at 6:51 AM, Tobias Hartmann wrote: > > Solution: > (1) A check for 'oop_is_instance' is added to ignore non-instance Klasses in the > subclass hierarchy. > (2) I added a check to only execute the interface related checks if we come from > a top level call. > Good work. For (1) I would prefer putting the check near to the operation it protects: diff --git a/src/share/vm/code/dependencies.cpp b/src/share/vm/code/dependencies.cpp --- a/src/share/vm/code/dependencies.cpp +++ b/src/share/vm/code/dependencies.cpp @@ -904,6 +904,8 @@ bool is_witness(Klass* k) { if (doing_subtype_search()) { return Dependencies::is_concrete_klass(k); + } else if (!k->oop_is_instance()) { + return false; // no methods to find in an array type } else { Method* m = InstanceKlass::cast(k)->find_method(_name, _signature); if (m == NULL || !Dependencies::is_concrete_method(m)) return false; If we ever put methods on arrays (clone, freeze, asList, etc.) it will be a little easier to see how to adjust this code. ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kozlov at oracle.com Fri Nov 14 20:41:59 2014 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Nov 2014 12:41:59 -0800 Subject: [9] RFR(S): 8050079: crash while compiling java.lang.ref.Finalizer::runFinalizer In-Reply-To: <5465E4E1.40609@oracle.com> References: <5464C576.7040705@oracle.com> <54650B17.1030401@oracle.com> <5465E4E1.40609@oracle.com> Message-ID: <54666917.2060104@oracle.com> Tobias, you are right. I missed that B is inherited from Object too and we process it anyway. Please, address John's comment. And we need sqe help with test. I want to run it without forking process by installing modified Object.class into working directory so we can use -Xbootclasspath/p:. I think ClassFileInstaller will do that. Thanks, Vladimir On 11/14/14 3:17 AM, Tobias Hartmann wrote: > Hi Vladimir, > > thanks for the review. > > On 13.11.2014 20:48, Vladimir Kozlov wrote: >> Should you fix problem (2) other way around by adding check in the following >> loop? If you don't bailout on 'nof_impls > 1' on interface what result you get? > > According to the comment we want to avoid this case: > > *I.m > { A.m, C }; B.m > C > > There are two cases were we may encounter an interface in the subclass hierarchy > and do not check for 'nof_impls > 1': > (1) context_type is java.lang.Object > (2) context_type is an interface > > (1) If context_type is a java.lang.Object O it should be fine not to bailout if > we encounter an interface with two implementations. For example: > > O.m > *I > { A.m, C }; B.m > C > > Coming from O we would miss the second implementation of I that is inherited > from B and overrides m. But since B also inherits from Object > > O.m > B.m > > we catch B.m while traversing B and bailout because we now have two witnesses. > So no need to bailout early in this case. > > (2) If context_type is an interface we may have the following hierarchy: > > *I.m > *I2 > { A.m, C }; B.m > C > > Again, we would miss the the second implementation of *I.m inherited from B.m. > But since nof_impls == 2 for *I, we bailout even before reaching *I2. > > So I don't think we have to move the check into the loop. What do you think? > >> I think test should go into new 'dependencies' subdir not 'inlining'. > > Okay, I moved it to compiler/dependencies/MonomorphicObjectCall/. > >> Why you fork separate JVM process? > > Because I need to set the -Xbootclasspath to the test classes directory to make > sure that the modified version of java.lang.Object is used. Specifying it in the > '@run main/othervm' command with > > '-Xbootclasspath/p:../classes/compiler/dependencies/TestMonomorphicObjectCall/' > > depends on the location of classes and the working directory (fails on JPRT). I > solve this by reading the directory with 'System.getProperty("test.classes")' > and forking a separate JVM process. Another solution would be to use a shell > script but I prefer the more readable Java code. > > New webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.01/ > > Thanks, > Tobias > >> >> Thanks, >> Vladimir >> >> On 11/13/14 6:51 AM, Tobias Hartmann wrote: >>> Hi, >>> >>> please review the following patch. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8050079 >>> Webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.00/ >>> >>> Problem (1): >>> C1 compiles a call to Object.finalize() in >>> 'java.lang.ref.Finalizer::runFinalizer'. To determine if the call is >>> monomorphic, the compiler searches through the subclasses of Object until it >>> finds a class that overrides finalize(). This check is performed in >>> 'ClassHierarchyWalker::find_witness_anywhere' by calling 'is_witness(Klass* k)' >>> on the subclasses. After casting with 'InstanceKlass::cast(k)' the (Sub-)Klass k >>> is treated as an InstanceKlass and 'InstanceKlass::find_method' is called. The >>> crash happens in 'binary_search' when a subclass k is not an InstanceKlass but >>> an ObjectArrayKlass and therefore '_methods' is NULL. >>> >>> Why didn't the bug show up earlier? >>> The method Object.finalize() is usually overridden by the subclasses >>> 'java.lang.ClassLoader$NativeLibrary' and 'java.lang.Enum' which are loaded >>> during startup. That means a call to Object.finalize() is never treated as >>> monomorphic and 'ClassHierarchyWalker::find_witness_anywhere' returns before >>> encountering an ObjectArrayKlass in the subclass hierarchy of Object. The >>> failure only occurs in an internal test environment for JRebel [1]. According to >>> the core file of the crash, the usual class hierarchy of the Java Runtime >>> Environment was changed by the framework. For example, java.lang.String is no >>> longer a subclass of java.lang.Object but of another class called >>> 'JaveleonObject' [2]. Especially the two overriding classes >>> 'ClassLoader$NativeLibrary' and 'Enum' are not subclasses of Object. This leads >>> to the crash because C1 encounters an ObjectArrayKlass in the subclass hierarchy. >>> Of course, this bug could also show up if the sequence of subclasses in the >>> 'Klass::subklass()' list changes. >>> >>> Problem (2): >>> In case of a worklist overflow we recursively call 'find_witness_anywhere' and >>> create a new worklist (see line 1178 of dependencies.cpp). The problem is that >>> we now execute the interface related checks that would otherwise not be executed >>> (line 1132). For interface subtypes of Object (for example java/lang/Readable) >>> we then bail out if 'nof_impls > 1'. >>> >>> Solution: >>> (1) A check for 'oop_is_instance' is added to ignore non-instance Klasses in the >>> subclass hierarchy. >>> (2) I added a check to only execute the interface related checks if we come from >>> a top level call. >>> >>> I was finally able to write a small jtreg regression test. Although the test is >>> a little bit hacky, it is stable and deterministically reproduces the bug. The >>> test works by overriding the default implementation of java.lang.Object, >>> renaming the method 'finalize' to 'finalizeObject' and thus guaranteeing that >>> 'finalizeObject' is never overridden and all calls to it are monomorphic. By >>> triggering a C1 compilation of 'Object.finalizeObject()' we trigger the bug >>> because there are no overriding instance Klasses in the call hierarchy. >>> >>> The '-XX:-VerifyDependencies' flag is necessary to not bail out too early >>> because of problem (2). >>> >>> Testing: >>> JPRT with new test >>> >>> Thanks, >>> Tobias >>> >>> >>> [1] http://zeroturnaround.com/software/jrebel/ >>> [2] https://bugs.openjdk.java.net/secure/attachment/23360/replay.log >>> From igor.ignatyev at oracle.com Sun Nov 16 19:59:00 2014 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sun, 16 Nov 2014 22:59:00 +0300 Subject: [9] RFR(S): 8050079: crash while compiling java.lang.ref.Finalizer::runFinalizer In-Reply-To: <54666917.2060104@oracle.com> References: <5464C576.7040705@oracle.com> <54650B17.1030401@oracle.com> <5465E4E1.40609@oracle.com> <54666917.2060104@oracle.com> Message-ID: <54690204.70402@oracle.com> Vladimir, > I think ClassFileInstaller will do that. unfortunately, ClassFileInstaller can't help w/ it. because of classloader delegation model, ClassFileInstaller always finds the original Object class instead of custom one. Tobias, 0. > 35 * @compile -Xbootclasspath/p:. java/lang/Object.java TestMonomorphicObjectCall.java to allow compilation of Object, you should use '-XDignore.symbol.file' 1. > 36 * @run main/othervm TestMonomorphicObjectCall to decrease count of spawned processes, you should use agentvm mode instead of othervm. 2. > 48 static public void main(String[] args) throws Throwable { > 58 static public void executeTestJvm() throws Throwable just a code style, you have disordered method modifiers. the right order is 'public static void' Igor On 11/14/2014 11:41 PM, Vladimir Kozlov wrote: > Tobias, you are right. I missed that B is inherited from Object too and > we process it anyway. Please, address John's comment. > > And we need sqe help with test. I want to run it without forking process > by installing modified Object.class into working directory so we can use > -Xbootclasspath/p:. > > I think ClassFileInstaller will do that. > > Thanks, > Vladimir > > On 11/14/14 3:17 AM, Tobias Hartmann wrote: >> Hi Vladimir, >> >> thanks for the review. >> >> On 13.11.2014 20:48, Vladimir Kozlov wrote: >>> Should you fix problem (2) other way around by adding check in the >>> following >>> loop? If you don't bailout on 'nof_impls > 1' on interface what >>> result you get? >> >> According to the comment we want to avoid this case: >> >> *I.m > { A.m, C }; B.m > C >> >> There are two cases were we may encounter an interface in the subclass >> hierarchy >> and do not check for 'nof_impls > 1': >> (1) context_type is java.lang.Object >> (2) context_type is an interface >> >> (1) If context_type is a java.lang.Object O it should be fine not to >> bailout if >> we encounter an interface with two implementations. For example: >> >> O.m > *I > { A.m, C }; B.m > C >> >> Coming from O we would miss the second implementation of I that is >> inherited >> from B and overrides m. But since B also inherits from Object >> >> O.m > B.m >> >> we catch B.m while traversing B and bailout because we now have two >> witnesses. >> So no need to bailout early in this case. >> >> (2) If context_type is an interface we may have the following hierarchy: >> >> *I.m > *I2 > { A.m, C }; B.m > C >> >> Again, we would miss the the second implementation of *I.m inherited >> from B.m. >> But since nof_impls == 2 for *I, we bailout even before reaching *I2. >> >> So I don't think we have to move the check into the loop. What do you >> think? >> >>> I think test should go into new 'dependencies' subdir not 'inlining'. >> >> Okay, I moved it to compiler/dependencies/MonomorphicObjectCall/. >> >>> Why you fork separate JVM process? >> >> Because I need to set the -Xbootclasspath to the test classes >> directory to make >> sure that the modified version of java.lang.Object is used. Specifying >> it in the >> '@run main/othervm' command with >> >> >> '-Xbootclasspath/p:../classes/compiler/dependencies/TestMonomorphicObjectCall/' >> >> >> depends on the location of classes and the working directory (fails on >> JPRT). I >> solve this by reading the directory with >> 'System.getProperty("test.classes")' >> and forking a separate JVM process. Another solution would be to use a >> shell >> script but I prefer the more readable Java code. >> >> New webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.01/ >> >> Thanks, >> Tobias >> >>> >>> Thanks, >>> Vladimir >>> >>> On 11/13/14 6:51 AM, Tobias Hartmann wrote: >>>> Hi, >>>> >>>> please review the following patch. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8050079 >>>> Webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.00/ >>>> >>>> Problem (1): >>>> C1 compiles a call to Object.finalize() in >>>> 'java.lang.ref.Finalizer::runFinalizer'. To determine if the call is >>>> monomorphic, the compiler searches through the subclasses of Object >>>> until it >>>> finds a class that overrides finalize(). This check is performed in >>>> 'ClassHierarchyWalker::find_witness_anywhere' by calling >>>> 'is_witness(Klass* k)' >>>> on the subclasses. After casting with 'InstanceKlass::cast(k)' the >>>> (Sub-)Klass k >>>> is treated as an InstanceKlass and 'InstanceKlass::find_method' is >>>> called. The >>>> crash happens in 'binary_search' when a subclass k is not an >>>> InstanceKlass but >>>> an ObjectArrayKlass and therefore '_methods' is NULL. >>>> >>>> Why didn't the bug show up earlier? >>>> The method Object.finalize() is usually overridden by the subclasses >>>> 'java.lang.ClassLoader$NativeLibrary' and 'java.lang.Enum' which are >>>> loaded >>>> during startup. That means a call to Object.finalize() is never >>>> treated as >>>> monomorphic and 'ClassHierarchyWalker::find_witness_anywhere' >>>> returns before >>>> encountering an ObjectArrayKlass in the subclass hierarchy of >>>> Object. The >>>> failure only occurs in an internal test environment for JRebel [1]. >>>> According to >>>> the core file of the crash, the usual class hierarchy of the Java >>>> Runtime >>>> Environment was changed by the framework. For example, >>>> java.lang.String is no >>>> longer a subclass of java.lang.Object but of another class called >>>> 'JaveleonObject' [2]. Especially the two overriding classes >>>> 'ClassLoader$NativeLibrary' and 'Enum' are not subclasses of Object. >>>> This leads >>>> to the crash because C1 encounters an ObjectArrayKlass in the >>>> subclass hierarchy. >>>> Of course, this bug could also show up if the sequence of subclasses >>>> in the >>>> 'Klass::subklass()' list changes. >>>> >>>> Problem (2): >>>> In case of a worklist overflow we recursively call >>>> 'find_witness_anywhere' and >>>> create a new worklist (see line 1178 of dependencies.cpp). The >>>> problem is that >>>> we now execute the interface related checks that would otherwise not >>>> be executed >>>> (line 1132). For interface subtypes of Object (for example >>>> java/lang/Readable) >>>> we then bail out if 'nof_impls > 1'. >>>> >>>> Solution: >>>> (1) A check for 'oop_is_instance' is added to ignore non-instance >>>> Klasses in the >>>> subclass hierarchy. >>>> (2) I added a check to only execute the interface related checks if >>>> we come from >>>> a top level call. >>>> >>>> I was finally able to write a small jtreg regression test. Although >>>> the test is >>>> a little bit hacky, it is stable and deterministically reproduces >>>> the bug. The >>>> test works by overriding the default implementation of >>>> java.lang.Object, >>>> renaming the method 'finalize' to 'finalizeObject' and thus >>>> guaranteeing that >>>> 'finalizeObject' is never overridden and all calls to it are >>>> monomorphic. By >>>> triggering a C1 compilation of 'Object.finalizeObject()' we trigger >>>> the bug >>>> because there are no overriding instance Klasses in the call hierarchy. >>>> >>>> The '-XX:-VerifyDependencies' flag is necessary to not bail out too >>>> early >>>> because of problem (2). >>>> >>>> Testing: >>>> JPRT with new test >>>> >>>> Thanks, >>>> Tobias >>>> >>>> >>>> [1] http://zeroturnaround.com/software/jrebel/ >>>> [2] https://bugs.openjdk.java.net/secure/attachment/23360/replay.log >>>> From tobias.hartmann at oracle.com Mon Nov 17 06:44:15 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 17 Nov 2014 07:44:15 +0100 Subject: [8u40] Backport RFR: 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is not compilable after 3 iterations' In-Reply-To: <546474BD.2080201@oracle.com> References: <546474BD.2080201@oracle.com> Message-ID: <5469993F.90001@oracle.com> Hi, can I get a review for this? Thanks, Tobias On 13.11.2014 10:07, Tobias Hartmann wrote: > Hi, > > please review the following backport request for 8u40. > > 8056071: compiler/whitebox/IsMethodCompilableTest.java fails with 'method() is > not compilable after 3 iterations' > https://bugs.openjdk.java.net/browse/JDK-8056071 > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/0d599246de33 > > The changes were pushed on Tuesday. Nightly testing showed no problems. The > changes apply cleanly to 8u40. > > Thanks, > Tobias > From aleksey.shipilev at oracle.com Mon Nov 17 11:13:37 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 17 Nov 2014 14:13:37 +0300 Subject: RFR (M) 8059461: Refactor IndexSet for better performance (preliminary) In-Reply-To: <546412DC.2020401@oracle.com> References: <545A960C.7010008@oracle.com> <545AAB7D.3010100@oracle.com> <5462616B.3030003@oracle.com> <546412DC.2020401@oracle.com> Message-ID: <5469D861.9040602@oracle.com> Thanks for taking a look, Vladimir! This is a change in performance-sensitive part of the compiler, and so I share your worry. Nothing that's worthwhile is ever easy. I will attend to your comments later. The methodology for measuring the compiler footprint seems reasonable. -Aleksey. On 11/13/2014 05:09 AM, Vladimir Kozlov wrote: > I am start worrying about this change. Nested ResourceMark may cause > problems. > > PhaseLive::getfreeset() uses thread local resource area for > _free_IndexSet and PhaseLive::compute() resets it on exit: > > // Init the sparse arrays for delta-sets. > ResourceMark rm; // Nuke temp storage on exit > > But the rest of IndexSet objects uses indexSet_arena which is > 'live_arena' in Register_Allocate(). It lives longer then > PhaseLive::compute() and PhaseLive::compute() may 'nuke' its data > incorrectly if you use thread local resource area for everything. > > May be you should also separate arenas for BitMap used as IndexSet. > > I think you need to instrument JVM to get correct data. If you use > separate ResourceArea (as in original code 'live_arena') you can print > its size at the end of its use scope. > > I would recommend to run with -Xbatch -XX:-TieredCompilation > -XX:CICompilerCount=1 to get serialized data which can be compared. > > Clean up: > > Remove commented assert in lrg_union() since it is not applicable anymore. > > Fix style in the code you are changing. No spaces in if(): > if( vec->is_empty() ) return; > should be > if (vec->is_empty()) { > return; > } > > Thanks, > Vladimir > > On 11/11/14 11:20 AM, Aleksey Shipilev wrote: >> Thanks for review, Vladimir! >> >> FTR, here's an updated webrev: >> http://cr.openjdk.java.net/~shade/8059461/webrev.03/ >> >> On 11/06/2014 01:58 AM, Vladimir Kozlov wrote: >>> Aleksey, can you compare average memory consumed by IndexSet before and >>> after? >> >> Any ideas how to estimate this reliably? Current code has some >> instrumentation, but only about usage, not the concrete footprint. NMT >> shows very flaky numbers for Nashorn/Octane: >> >> $ grep "Compiler" before/* >> Compiler (reserved=165KB, committed=165KB) >> Compiler (reserved=174KB, committed=174KB) >> Compiler (reserved=201KB, committed=201KB) >> Compiler (reserved=189KB, committed=189KB) >> Compiler (reserved=192KB, committed=192KB) >> >> $ grep "Compiler" after/* >> Compiler (reserved=162KB, committed=162KB) >> Compiler (reserved=167KB, committed=167KB) >> Compiler (reserved=198KB, committed=198KB) >> Compiler (reserved=240KB, committed=240KB) >> Compiler (reserved=188KB, committed=188KB) >> >> $ grep "Arena" before/* >> Arena Chunk (reserved=10952KB, committed=10952KB) >> Arena Chunk (reserved=11433KB, committed=11433KB) >> Arena Chunk (reserved=10506KB, committed=10506KB) >> Arena Chunk (reserved=10825KB, committed=10825KB) >> Arena Chunk (reserved=7179KB, committed=7179KB) >> >> $ grep "Arena" after/* >> Arena Chunk (reserved=19971KB, committed=19971KB) >> Arena Chunk (reserved=20738KB, committed=20738KB) >> Arena Chunk (reserved=21090KB, committed=21090KB) >> Arena Chunk (reserved=22304KB, committed=22304KB) >> Arena Chunk (reserved=14342KB, committed=14342KB) >> >> It *looks* like a memory leak in Arenas, but it might be as well the >> larger peak usage. Or, it might be NMT not telling me the whole story. >> >> I am running Footprint tests from our performance regression suites now >> -- they will show if there is something real. >> >>> Why you need initialize_in_resource_arena()? by default BitMap() uses >>> resource area: >>> >>> BitMap(idx_t size_in_bits, bool in_resource_area = true); >> >> Yes, pruned that. >> >>> Make lrg_union() PhaseConservativeCoalesce class's method. >> >> Yes, did that. >> >> Thanks, >> -Aleksey. >> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From tobias.hartmann at oracle.com Mon Nov 17 11:33:03 2014 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 17 Nov 2014 12:33:03 +0100 Subject: [9] RFR(S): 8050079: crash while compiling java.lang.ref.Finalizer::runFinalizer In-Reply-To: <54690204.70402@oracle.com> References: <5464C576.7040705@oracle.com> <54650B17.1030401@oracle.com> <5465E4E1.40609@oracle.com> <54666917.2060104@oracle.com> <54690204.70402@oracle.com> Message-ID: <5469DCEF.8020709@oracle.com> Vladimir, Igor, thanks for the review. >> I think ClassFileInstaller will do that. > unfortunately, ClassFileInstaller can't help w/ it. because of classloader > delegation model, ClassFileInstaller always finds the original Object class > instead of custom one. Yes, I've tried that while writing the test. > > Tobias, > 0. >> 35 * @compile -Xbootclasspath/p:. java/lang/Object.java >> TestMonomorphicObjectCall.java > to allow compilation of Object, you should use '-XDignore.symbol.file' I removed '-Xbootclasspath/p:.' and added '-XDignore.symbol.file'. > 1. >> 36 * @run main/othervm TestMonomorphicObjectCall > to decrease count of spawned processes, you should use agentvm mode instead of > othervm. I changed the code to '@run main'. > 2. >> 48 static public void main(String[] args) throws Throwable { >> 58 static public void executeTestJvm() throws Throwable > just a code style, you have disordered method modifiers. the right order is > 'public static void' Fixed. New webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.02/ Thanks, Tobias > > Igor > > On 11/14/2014 11:41 PM, Vladimir Kozlov wrote: >> Tobias, you are right. I missed that B is inherited from Object too and >> we process it anyway. Please, address John's comment. >> >> And we need sqe help with test. I want to run it without forking process >> by installing modified Object.class into working directory so we can use >> -Xbootclasspath/p:. >> >> I think ClassFileInstaller will do that. >> >> Thanks, >> Vladimir >> >> On 11/14/14 3:17 AM, Tobias Hartmann wrote: >>> Hi Vladimir, >>> >>> thanks for the review. >>> >>> On 13.11.2014 20:48, Vladimir Kozlov wrote: >>>> Should you fix problem (2) other way around by adding check in the >>>> following >>>> loop? If you don't bailout on 'nof_impls > 1' on interface what >>>> result you get? >>> >>> According to the comment we want to avoid this case: >>> >>> *I.m > { A.m, C }; B.m > C >>> >>> There are two cases were we may encounter an interface in the subclass >>> hierarchy >>> and do not check for 'nof_impls > 1': >>> (1) context_type is java.lang.Object >>> (2) context_type is an interface >>> >>> (1) If context_type is a java.lang.Object O it should be fine not to >>> bailout if >>> we encounter an interface with two implementations. For example: >>> >>> O.m > *I > { A.m, C }; B.m > C >>> >>> Coming from O we would miss the second implementation of I that is >>> inherited >>> from B and overrides m. But since B also inherits from Object >>> >>> O.m > B.m >>> >>> we catch B.m while traversing B and bailout because we now have two >>> witnesses. >>> So no need to bailout early in this case. >>> >>> (2) If context_type is an interface we may have the following hierarchy: >>> >>> *I.m > *I2 > { A.m, C }; B.m > C >>> >>> Again, we would miss the the second implementation of *I.m inherited >>> from B.m. >>> But since nof_impls == 2 for *I, we bailout even before reaching *I2. >>> >>> So I don't think we have to move the check into the loop. What do you >>> think? >>> >>>> I think test should go into new 'dependencies' subdir not 'inlining'. >>> >>> Okay, I moved it to compiler/dependencies/MonomorphicObjectCall/. >>> >>>> Why you fork separate JVM process? >>> >>> Because I need to set the -Xbootclasspath to the test classes >>> directory to make >>> sure that the modified version of java.lang.Object is used. Specifying >>> it in the >>> '@run main/othervm' command with >>> >>> >>> '-Xbootclasspath/p:../classes/compiler/dependencies/TestMonomorphicObjectCall/' >>> >>> >>> depends on the location of classes and the working directory (fails on >>> JPRT). I >>> solve this by reading the directory with >>> 'System.getProperty("test.classes")' >>> and forking a separate JVM process. Another solution would be to use a >>> shell >>> script but I prefer the more readable Java code. >>> >>> New webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.01/ >>> >>> Thanks, >>> Tobias >>> >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/13/14 6:51 AM, Tobias Hartmann wrote: >>>>> Hi, >>>>> >>>>> please review the following patch. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8050079 >>>>> Webrev: http://cr.openjdk.java.net/~thartmann/8050079/webrev.00/ >>>>> >>>>> Problem (1): >>>>> C1 compiles a call to Object.finalize() in >>>>> 'java.lang.ref.Finalizer::runFinalizer'. To determine if the call is >>>>> monomorphic, the compiler searches through the subclasses of Object >>>>> until it >>>>> finds a class that overrides finalize(). This check is performed in >>>>> 'ClassHierarchyWalker::find_witness_anywhere' by calling >>>>> 'is_witness(Klass* k)' >>>>> on the subclasses. After casting with 'InstanceKlass::cast(k)' the >>>>> (Sub-)Klass k >>>>> is treated as an InstanceKlass and 'InstanceKlass::find_method' is >>>>> called. The >>>>> crash happens in 'binary_search' when a subclass k is not an >>>>> InstanceKlass but >>>>> an ObjectArrayKlass and therefore '_methods' is NULL. >>>>> >>>>> Why didn't the bug show up earlier? >>>>> The method Object.finalize() is usually overridden by the subclasses >>>>> 'java.lang.ClassLoader$NativeLibrary' and 'java.lang.Enum' which are >>>>> loaded >>>>> during startup. That means a call to Object.finalize() is never >>>>> treated as >>>>> monomorphic and 'ClassHierarchyWalker::find_witness_anywhere' >>>>> returns before >>>>> encountering an ObjectArrayKlass in the subclass hierarchy of >>>>> Object. The >>>>> failure only occurs in an internal test environment for JRebel [1]. >>>>> According to >>>>> the core file of the crash, the usual class hierarchy of the Java >>>>> Runtime >>>>> Environment was changed by the framework. For example, >>>>> java.lang.String is no >>>>> longer a subclass of java.lang.Object but of another class called >>>>> 'JaveleonObject' [2]. Especially the two overriding classes >>>>> 'ClassLoader$NativeLibrary' and 'Enum' are not subclasses of Object. >>>>> This leads >>>>> to the crash because C1 encounters an ObjectArrayKlass in the >>>>> subclass hierarchy. >>>>> Of course, this bug could also show up if the sequence of subclasses >>>>> in the >>>>> 'Klass::subklass()' list changes. >>>>> >>>>> Problem (2): >>>>> In case of a worklist overflow we recursively call >>>>> 'find_witness_anywhere' and >>>>> create a new worklist (see line 1178 of dependencies.cpp). The >>>>> problem is that >>>>> we now execute the interface related checks that would otherwise not >>>>> be executed >>>>> (line 1132). For interface subtypes of Object (for example >>>>> java/lang/Readable) >>>>> we then bail out if 'nof_impls > 1'. >>>>> >>>>> Solution: >>>>> (1) A check for 'oop_is_instance' is added to ignore non-instance >>>>> Klasses in the >>>>> subclass hierarchy. >>>>> (2) I added a check to only execute the interface related checks if >>>>> we come from >>>>> a top level call. >>>>> >>>>> I was finally able to write a small jtreg regression test. Although >>>>> the test is >>>>> a little bit hacky, it is stable and deterministically reproduces >>>>> the bug. The >>>>> test works by overriding the default implementation of >>>>> java.lang.Object, >>>>> renaming the method 'finalize' to 'finalizeObject' and thus >>>>> guaranteeing that >>>>> 'finalizeObject' is never overridden and all calls to it are >>>>> monomorphic. By >>>>> triggering a C1 compilation of 'Object.finalizeObject()' we trigger >>>>> the bug >>>>> because t