From ioi.lam at oracle.com Wed Mar 1 02:25:24 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 28 Feb 2017 18:25:24 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> Message-ID: <58B63114.6010806@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8005165 http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ Hi, This is the official review (follow up of the "Determining the size of C++ vtables" thread on hotspot-dev at openjdk.java.net). The new code has the same assumption as the existing code in JDK 10: for a C++ object that contains virtual methods (e.g., ConstantPool), we assume the first intptr_t slot of the object is a _vptr, which points to a vtable, which consists of no more than 150 intptr_t's. ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] [ field #0 ] [ vtable slot 1 ] [ field #1 ] [ vtable slot 2 ] [ field #2 ] [ vtable slot 3 ] [ .... ] [ vtable slot 4] [ vtable slot 5 ] [ ... ] + In the existing code, we were pointing the vtable slots to code that's generated by HotSpot. + In the new code, we copy the vtable slots from an existing vtable (generated by the C++ linker). Per Thomas St?fe's advice, I don't try to determine the size of the vtable (as that would add one more compiler requirement where new virtual methods added by a subclass must be placed at a higher offset in the vtable). Instead, I have added code in non-product builds to ensure that the vtables are no longer than 150 entries. You can run with "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size of the vtables for your particular platform: ConstantPool has 12 virtual methods InstanceKlass has 113 virtual methods InstanceClassLoaderKlass has 113 virtual methods InstanceMirrorKlass has 113 virtual methods InstanceRefKlass has 113 virtual methods Method has 12 virtual methods ObjArrayKlass has 114 virtual methods TypeArrayKlass has 114 virtual methods As mentioned in the code comments, if you have an esoteric C++ compiler, the verify_sufficient_size() function will probably fail, but hopefully that would give you some hints for porting this code. To avoid accidentally touching an unmapped page, the code uses SafeFetchN for copying the vtable contents, and would shrink the vtable to less than 150 entries if necessary. I can't test this for real, but I've added some code to simulate an error: for (int i=0; i 120 /* uncomment this line to test */ ) { _info->set_vtab_size(i-1); break; } dstvtab[i] = num; } Results: + Removed 850 lines of CPU-dependent code + CDS image is about 50K smaller + Previously Metadata objects must live in the read-write section in the CDS archive, because their _vptr was updated at run time. Now _vptr is no longer updated, so ConstantPool can be moved to the read-only section (see JDK-8171392). Thanks - Ioi From ioi.lam at oracle.com Wed Mar 1 02:37:16 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 28 Feb 2017 18:37:16 -0800 Subject: Determining the size of C++ vtables In-Reply-To: References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> Message-ID: <58B633DC.30308@oracle.com> On 2/24/17 11:52 PM, Thomas St?fe wrote: > Hi ioi, > > On Fri, Feb 24, 2017 at 7:50 PM, Ioi Lam > wrote: > > > > On 2/23/17 11:17 PM, Thomas St?fe wrote: >> Hi Ioi, >> >> After reading through the original mailthread referenced in the >> bug report, the only reason the "just copy enough (200) slots >> from the vtable in libjvm.so to the vtable in the image" approach >> does not work for you is that you are afraid of hitting unmapped >> memory? If I understand it correctly. >> >> If that is the case, why not copy 200 slots with SafeFetch and >> stop at the first error? That should not incur much additional >> costs as long as you do not hit an unmapped area, in which case >> you'd stop anyway. >> > Oh, that's a good simplification.I hadn't thought about using > SafeFetch :-( > > One issue with it is the 200 slots are too big (ConstantPool needs > just 10 slots), so finding the exact size is a little more > efficient and saves a few KBs. > > > How many vtables are you copying? Does it really matter? > I am copying only 8 vtables, so it doesn't really matter. > As I said, I think your approach is quite elegant, but not terribly > portable. We already make some assumptions (that vtables are organized > as one continuous block of function pointers) and you add some more > assumptions about the order of the methods in the vtable. > > I think even if you can make sure that it works now with our set of > platforms and compilers, this adds the risk that a future compiler > update may change the vtable layout and break the code. E.g. a vtable > dumped with a VM compilers by an older compiler could not be loaded > with the newly compiled VM. This shouldn't be a problem with usual compiler upgrades: You can load a CDS image only with the VM that generates it (we have a VM version string check), so if you have upgraded the compiler (or enabled RTTI and rebuilt the VM), so you have larger vtables, you won't be able to load any old CDS image anyway. However, if your compiler completely changes the way it implements vtables (e.g., adding one level of indirection to vptr), then, things will break, but the existing CDS vtable code also has the same problem. > Also, you set the bar a bit higher for future ports, because you add > requirements for the C++ compiler. > > But I may be too cautious here and I am waiting for others to chime in. > > As a side note, it would make sense to write a tiny C++ test which > copies vtables around using your technique and then checks if they > still work; such a test we could use on out platforms to check if our > compilers can cope. Such a test would also be a good sanity check to > add to the VM initialization if CDS is active. > I've taken your suggestion to use SafeFetchN instead of trying to figure out the vtable size. That simplifies the product-build a bit. I put my vtable sizing code in debug builds. It's not strictly portable, but it probably fail quickly on an esoteric c++ compiler so you can figure out what's wrong. It will also assert if you have added too many new virtual method to overflow the current limit of 150. Please reply your comments to the RFR e-mail thread. Thanks - Ioi > Kind Regards, Thomas > > Thanks > - Ioi > > >> That way you make yourself a bit less dependent on compiler >> internals. I agree that your approach is more elegant, though. >> >> Kind Regards, Thomas >> >> On Fri, Feb 24, 2017 at 4:55 AM, Ioi Lam > > wrote: >> >> >> >> On 2/23/17 7:47 PM, Ioi Lam wrote: >> >> Hi, >> >> I am working on >> https://bugs.openjdk.java.net/browse/JDK-8005165 >> (Remove >> CPU-dependent code in self-patching vtables), I need a >> way find out the size >> of a C++ vtable. I ended up doing this: >> >> >> // Objects of the Metadata types (such as Klass and >> ConstantPool) have C++ vtables. >> // (In GCC this is the field ::_vptr, i.e., first >> word in the object.) >> // >> // Addresses of the vtables and the methods may be >> different across JVM runs, >> // if libjvm.so is dynamically loaded at a different base >> address. >> // >> // To ensure that the Metadata objects in the CDS archive >> always have the correct vtable: >> // >> // + at dump time: we redirect the _vptr to point to our >> own vtables inside >> // the CDS image >> // + at run time: we clone the actual contents of the >> vtables from libjvm.so >> // into our own tables. >> // >> // To determine the size of the vtable for each type, we >> use the following >> // trick by declaring 2 subclasses: >> // >> // class CppVtabTesterA: public InstanceKlass { >> // virtual int last_virtual_method() {return 1;} >> // }; >> // class CppVtabTesterB: public InstanceKlass { >> // virtual void* last_virtual_method() {return >> NULL}; >> // }; >> // >> // CppVtabTesterA and CppVtabTesterB's vtables have the >> following properties: >> // - Their size (N+1) is exactly one more than the size >> of InstanceKlass's vtable (N) >> // - The first N entries have are exactly the same as in >> InstanceKlass's vtable. >> // - Their last entry is different. >> // >> // So to determine the value of N, we just walk >> CppVtabTesterA and CppVtabTesterB's tables >> // and find the first entry that's different >> >> >> Could anyone comment if this is acceptable? I know it's >> not 100% portable (C++ doesn't >> specify where to find the vtable, or what's inside), but >> my assumptions is the same as >> the existing code. I.e., _vptr is a pointer located at >> offset 0 of the object, and it >> points to a one-dimensional array. >> >> So at least it's not any worse than before? >> >> Thanks >> - Ioi >> >> By the way, I first tried having only a single "tester" class >> and walk the vtable to look for &last_virtual_method, but the >> C++ compiler told me that taking the address of a non-static >> function is not allowed ..... so I ended up creating two >> tester classes and checking their differences. >> >> >> >> > > From robbin.ehn at oracle.com Wed Mar 1 06:52:05 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 1 Mar 2017 07:52:05 +0100 Subject: [10]RFR: 8136650: Add support for custom jtreg native tests In-Reply-To: <0a0001d291f3$40b09340$c211b9c0$@oracle.com> References: <62136515-9397-08d2-2baa-1957cae14eb2@oracle.com> <0a0001d291f3$40b09340$c211b9c0$@oracle.com> Message-ID: Thanks Christian! On 02/28/2017 07:48 PM, Christian Tornqvist wrote: > Hi Robbin, > > This looks good, thanks for doing this! > > Thanks, > Christian > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Robbin Ehn > Sent: Tuesday, February 28, 2017 11:52 AM > To: hotspot-dev at openjdk.java.net > Subject: [10]RFR: 8136650: Add support for custom jtreg native tests > > Hi, please review. > > This adds support for custom extensions the jtreg native tests. > > The issues is closed due to internal tests but contains no more info than this subject: > https://bugs.openjdk.java.net/browse/JDK-8136650 > > Open webrev: > http://cr.openjdk.java.net/~rehn/8136650/ > > Tested with dummy native test. > > Thanks, Robbin > From thomas.stuefe at gmail.com Wed Mar 1 07:38:47 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 1 Mar 2017 08:38:47 +0100 Subject: Determining the size of C++ vtables In-Reply-To: <58B633DC.30308@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B633DC.30308@oracle.com> Message-ID: Hi Ioi, On Wed, Mar 1, 2017 at 3:37 AM, Ioi Lam wrote: > > > On 2/24/17 11:52 PM, Thomas St?fe wrote: > > Hi ioi, > > On Fri, Feb 24, 2017 at 7:50 PM, Ioi Lam wrote: > >> >> >> On 2/23/17 11:17 PM, Thomas St?fe wrote: >> >> Hi Ioi, >> >> After reading through the original mailthread referenced in the bug >> report, the only reason the "just copy enough (200) slots from the vtable >> in libjvm.so to the vtable in the image" approach does not work for you is >> that you are afraid of hitting unmapped memory? If I understand it >> correctly. >> >> If that is the case, why not copy 200 slots with SafeFetch and stop at >> the first error? That should not incur much additional costs as long as you >> do not hit an unmapped area, in which case you'd stop anyway. >> >> Oh, that's a good simplification.I hadn't thought about using SafeFetch >> :-( >> >> One issue with it is the 200 slots are too big (ConstantPool needs just >> 10 slots), so finding the exact size is a little more efficient and saves a >> few KBs. >> >> > How many vtables are you copying? Does it really matter? > > I am copying only 8 vtables, so it doesn't really matter. > > As I said, I think your approach is quite elegant, but not terribly > portable. We already make some assumptions (that vtables are organized as > one continuous block of function pointers) and you add some more > assumptions about the order of the methods in the vtable. > > I think even if you can make sure that it works now with our set of > platforms and compilers, this adds the risk that a future compiler update > may change the vtable layout and break the code. E.g. a vtable dumped with > a VM compilers by an older compiler could not be loaded with the newly > compiled VM. > > This shouldn't be a problem with usual compiler upgrades: > > You can load a CDS image only with the VM that generates it (we have a VM > version string check), so if you have upgraded the compiler (or enabled > RTTI and rebuilt the VM), so you have larger vtables, you won't be able to > load any old CDS image anyway. > > However, if your compiler completely changes the way it implements vtables > (e.g., adding one level of indirection to vptr), then, things will break, > but the existing CDS vtable code also has the same problem. > > Also, you set the bar a bit higher for future ports, because you add > requirements for the C++ compiler. > > But I may be too cautious here and I am waiting for others to chime in. > > As a side note, it would make sense to write a tiny C++ test which copies > vtables around using your technique and then checks if they still work; > such a test we could use on out platforms to check if our compilers can > cope. Such a test would also be a good sanity check to add to the VM > initialization if CDS is active. > > I've taken your suggestion to use SafeFetchN instead of trying to figure > out the vtable size. That simplifies the product-build a bit. > > I put my vtable sizing code in debug builds. It's not strictly portable, > but it probably fail quickly on an esoteric c++ compiler so you can figure > out what's wrong. It will also assert if you have added too many new > virtual method to overflow the current limit of 150. > > Please reply your comments to the RFR e-mail thread. > > Thank you for taking my suggestions, I will take a look at that RFR. Thomas > Thanks > - Ioi > > > Kind Regards, Thomas > > > >> Thanks >> - Ioi >> >> >> That way you make yourself a bit less dependent on compiler internals. I >> agree that your approach is more elegant, though. >> >> Kind Regards, Thomas >> >> On Fri, Feb 24, 2017 at 4:55 AM, Ioi Lam wrote: >> >>> >>> >>> On 2/23/17 7:47 PM, Ioi Lam wrote: >>> >>>> Hi, >>>> >>>> I am working on https://bugs.openjdk.java.net/browse/JDK-8005165 >>>> (Remove >>>> CPU-dependent code in self-patching vtables), I need a way find out the >>>> size >>>> of a C++ vtable. I ended up doing this: >>>> >>>> >>>> // Objects of the Metadata types (such as Klass and ConstantPool) have >>>> C++ vtables. >>>> // (In GCC this is the field ::_vptr, i.e., first word in the >>>> object.) >>>> // >>>> // Addresses of the vtables and the methods may be different across JVM >>>> runs, >>>> // if libjvm.so is dynamically loaded at a different base address. >>>> // >>>> // To ensure that the Metadata objects in the CDS archive always have >>>> the correct vtable: >>>> // >>>> // + at dump time: we redirect the _vptr to point to our own vtables >>>> inside >>>> // the CDS image >>>> // + at run time: we clone the actual contents of the vtables from >>>> libjvm.so >>>> // into our own tables. >>>> // >>>> // To determine the size of the vtable for each type, we use the >>>> following >>>> // trick by declaring 2 subclasses: >>>> // >>>> // class CppVtabTesterA: public InstanceKlass { >>>> // virtual int last_virtual_method() {return 1;} >>>> // }; >>>> // class CppVtabTesterB: public InstanceKlass { >>>> // virtual void* last_virtual_method() {return NULL}; >>>> // }; >>>> // >>>> // CppVtabTesterA and CppVtabTesterB's vtables have the following >>>> properties: >>>> // - Their size (N+1) is exactly one more than the size of >>>> InstanceKlass's vtable (N) >>>> // - The first N entries have are exactly the same as in >>>> InstanceKlass's vtable. >>>> // - Their last entry is different. >>>> // >>>> // So to determine the value of N, we just walk CppVtabTesterA and >>>> CppVtabTesterB's tables >>>> // and find the first entry that's different >>>> >>>> >>>> Could anyone comment if this is acceptable? I know it's not 100% >>>> portable (C++ doesn't >>>> specify where to find the vtable, or what's inside), but my assumptions >>>> is the same as >>>> the existing code. I.e., _vptr is a pointer located at offset 0 of the >>>> object, and it >>>> points to a one-dimensional array. >>>> >>>> So at least it's not any worse than before? >>>> >>>> Thanks >>>> - Ioi >>>> >>>> By the way, I first tried having only a single "tester" class and walk >>> the vtable to look for &last_virtual_method, but the C++ compiler told me >>> that taking the address of a non-static function is not allowed ..... so I >>> ended up creating two tester classes and checking their differences. >>> >>> >>> >>> >> >> > > From thomas.stuefe at gmail.com Wed Mar 1 10:22:04 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 1 Mar 2017 11:22:04 +0100 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58B63114.6010806@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> Message-ID: Hi Ioi, I did not yet look closely at your change yet, just a small nit: To prevent the copying SafeFetch coding from accidentally tripping over a real "deadbeef" value which may be a valid part of the vtable - however completely unlikely this is - one could call SafeFetch twice with two different bad values,eg: const intptr_t bad1 = intptr_t(0xdeadbeef); const intptr_t bad2 = intptr_t(0xbeefdead); if (SafeFetchN(ptr, bad1) == bad1 && SafeFetchN(ptr, bad2) == bad2) { ... break out ... } Kind Regards, Thomas On Wed, Mar 1, 2017 at 3:25 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8005165 > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind > ependent-cds-vtable.v02/ > > Hi, > > This is the official review (follow up of the "Determining the size of C++ > vtables" thread on hotspot-dev at openjdk.java.net). > > The new code has the same assumption as the existing code in JDK 10: for a > C++ object that contains virtual methods (e.g., ConstantPool), we assume > the first intptr_t slot of the object is a _vptr, which points to a vtable, > which consists of no more than 150 intptr_t's. > > ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] > [ field #0 ] [ vtable slot 1 ] > [ field #1 ] [ vtable slot 2 ] > [ field #2 ] [ vtable slot 3 ] > [ .... ] [ vtable slot 4] > [ vtable slot 5 ] > [ ... ] > > + In the existing code, we were pointing the vtable slots to > code that's generated by HotSpot. > > + In the new code, we copy the vtable slots from an existing > vtable (generated by the C++ linker). > > Per Thomas St?fe's advice, I don't try to determine the size of the vtable > (as that would add one more compiler requirement where new virtual methods > added by a subclass must be placed at a higher offset in the vtable). > > Instead, I have added code in non-product builds to ensure that the > vtables are no longer than 150 entries. You can run with > "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size of the > vtables for your particular platform: > > ConstantPool has 12 virtual methods > InstanceKlass has 113 virtual methods > InstanceClassLoaderKlass has 113 virtual methods > InstanceMirrorKlass has 113 virtual methods > InstanceRefKlass has 113 virtual methods > Method has 12 virtual methods > ObjArrayKlass has 114 virtual methods > TypeArrayKlass has 114 virtual methods > > As mentioned in the code comments, if you have an esoteric C++ compiler, > the verify_sufficient_size() function will probably fail, but hopefully > that would give you some hints for porting this code. > > To avoid accidentally touching an unmapped page, the code uses SafeFetchN > for copying the vtable contents, and would shrink the vtable to less than > 150 entries if necessary. I can't test this for real, but I've added some > code to simulate an error: > > for (int i=0; i const intptr_t bad = intptr_t(0xdeadbeef); > intptr_t num = SafeFetchN(&srcvtab[i], bad); > if (num == bad > // || i > 120 /* uncomment this line to test */ > ) { > _info->set_vtab_size(i-1); > break; > } > dstvtab[i] = num; > } > > Results: > > + Removed 850 lines of CPU-dependent code > > + CDS image is about 50K smaller > > + Previously Metadata objects must live in the read-write section in the > CDS > archive, because their _vptr was updated at run time. Now _vptr is no > longer > updated, so ConstantPool can be moved to the read-only section (see > JDK-8171392). > > Thanks > - Ioi > > > > From shafi.s.ahmad at oracle.com Wed Mar 1 10:57:06 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 1 Mar 2017 02:57:06 -0800 (PST) Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample Message-ID: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> Hi, Summary: It's a very small change to a single file. void PerfLongVariant::sample() { - assert(_sample_helper != NULL || _sampled != NULL, "unexpected state"); + // JJJ - This should not happen. Maybe the first sample is taken // + while the _sample_helper is being null'ed out. + // assert(_sample_helper != NULL || _sampled != NULL, "unexpected state"); + if (_sample_helper == NULL) return; if (_sample_helper != NULL) { *(jlong*)_valuep = _sample_helper->take_sample(); } else if (_sampled != NULL) { *(jlong*)_valuep = *_sampled; } } Above five lines are modified in changeset http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.12 Due to the addition of NULL check and return if (_sample_helper == NULL) return; the else-if block becomes redundant and statement 'if (_sample_helper != NULL)' is not needed. Webrev link: http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 Testing: jprt. Thanks, Shafi From Jacob.Wieland at spirent.com Wed Mar 1 12:47:52 2017 From: Jacob.Wieland at spirent.com (Wieland, Jacob) Date: Wed, 1 Mar 2017 12:47:52 +0000 Subject: throwing static exceptions sometimes VERY slow! In-Reply-To: References: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com>, Message-ID: Hi, here are the general system infos: uname -a: Linux ber-28753 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17) x86_64 GNU/Linux lscpu: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 58 Model name: Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz Stepping: 9 CPU MHz: 2094.558 CPU max MHz: 3500.0000 CPU min MHz: 1600.0000 BogoMIPS: 6186.30 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 6144K NUMA node0 CPU(s): 0-3 java -version: java version "1.8.0_102" Java(TM) SE Runtime Environment (build 1.8.0_102-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode) Rest of the answers inline. BR [X] Dr. Jacob Wieland Senior Software Engineer main +49.30.7261919.34 mobile +49.173.6446443 jacob.wieland at spirent.com www.spirent.com Follow us on: Spirent Communications [X]|[X]|[X] Michaelkirchstra?e 17/18 10179 Berlin, Germany +++++ N E W S F L A S H +++++ Spirent Communications and Testing Technologies join forces to become your test automation power-house. Read more at http://conta.cc/1S62BEY. ________________________________ From: Vitaly Davidovich Sent: Tuesday, February 28, 2017 1:46:09 PM To: David Holmes Cc: Wieland, Jacob; hotspot-dev at openjdk.java.net Subject: Re: throwing static exceptions sometimes VERY slow! On Tue, Feb 28, 2017 at 7:37 AM, David Holmes > wrote: Hi, On 28/02/2017 9:02 PM, Wieland, Jacob wrote: Hi, I am observing a very strange behavior. In our generated code (due to various reasons I won't go into here unless I have to, but trust me, they are legit), we throw static exceptions for control flow purposes, This seems to work fine and without performance loss most of the time. However, we are observing now that every few seconds, a throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to the normal almost non-measurable time). Update: it seems that the observed behavior was kind of related with some extraneous circumstances (the computer seems to have been more lagged than usual at that point). By now, I can only observe throws that take up to 20 milliseconds (which opposed to the normal time still seems slow). It does not seem to be GC related, the only idea that I have is the jitter. What jitter? I'm assuming Jacob is referring to the JIT. But yes, he needs to provide more information. In particular, it would be good to know the following, as David mentioned, (off the top of my head): 1) JVM cmdline flags -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=y 2) Type of exception thrown (with a stacktrace or not) Exception class derived from java.lang.Error that overrides fillInStackTrace() with 'return this', so stackless. Same instances stored in static final fields are always thrown. 3) Is the call stack depth about the same during the slow and fast throws? It is always exactly the same stack, as it happens in a loop. 4) Is the exception thrown frequently or infrequently? In the test where this was observed, the exception was thrown every 40ms, so, I guess, frequently. 5) Is there -XX:+PrintCompilation output available around the time when the slowdown is observed Unfortunately not, and at the moment, I cannot reproduce the behavior. I made a mockup-test (see attachment) that mimicks kind of what my generated code does and adds measurements before throw and after catch. Mostly, in my experiments with this, the throw takes on average 20 ns, but between a 1 and 3 per million iterations take longer than 1 ms, on average around 7 ms. It can be observed that doing a garbage collection in a different java process (namely my eclipse that started the test) greatly influences the frequency of the > 1 ms throws, so maybe it is simply a load issue. Using multiple threads in parallel also seems to heighten the Also, when introducing an additional sleep between iterations (like in my original scenario where I was waiting for a timer to timeout before executing the construct that used the throw), I cannot reproduce such outliers at all. So, my question is. Is this a known (and for some strange reason maybe even accepted) behavior or is this a bug that I should file with Oracle (or you guys). You really are not giving us anything to go on here. How are you observing this slowness? Exactly how do you throw? What exactly are you measuring? What's the execution context, the machine, processors etc etc etc. I was observing the slowness in a testcase where I wanted regular intervals of 40 ms between sending a message via UDP but frequently (every few seconds) got a lag of around 1.5 seconds (and a few days later 0.5 seconds) before the timer which timed out at the correct time was noted. When eliminating one kind of such thrown control exception, the next kind of thrown control exception (from a surrounding construct) was having the same effect which actually led me to the slow-throw-hypothesis. Eliminating all exceptions in the small example confirmed this hypothesis, but going that route would take a lot of effort for all cases and introduce other additional imperformances (as I basically have to mock the exception throwing via cascading return and if-checks on every level) which I would like to avoid. Regards, David Spirent Communications e-mail confidentiality. ------------------------------------------------------------------------ This e-mail contains confidential and / or privileged information belonging to Spirent Communications plc, its affiliates and / or subsidiaries. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution and / or the taking of any action based upon reliance on the contents of this transmission is strictly forbidden. If you have received this message in error please notify the sender by return e-mail and delete it from your system. Spirent Communications plc Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom. Tel No. +44 (0) 1293 767676 Fax No. +44 (0) 1293 767677 Registered in England Number 470893 Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom. Or if within the US, Spirent Communications, 27349 Agoura Road, Calabasas, CA, 91301, USA. Tel No. 1-818-676- 2300 From erik.joelsson at oracle.com Wed Mar 1 13:01:15 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 1 Mar 2017 14:01:15 +0100 Subject: [10]RFR: 8136650: Add support for custom jtreg native tests In-Reply-To: <62136515-9397-08d2-2baa-1957cae14eb2@oracle.com> References: <62136515-9397-08d2-2baa-1957cae14eb2@oracle.com> Message-ID: <201f3bbf-affd-41ff-e0a8-a294934911ec@oracle.com> (adding build-dev) Looks good to me. /Erik On 2017-02-28 17:52, Robbin Ehn wrote: > Hi, please review. > > This adds support for custom extensions the jtreg native tests. > > The issues is closed due to internal tests but contains no more info > than this subject: > https://bugs.openjdk.java.net/browse/JDK-8136650 > > Open webrev: > http://cr.openjdk.java.net/~rehn/8136650/ > > Tested with dummy native test. > > Thanks, Robbin From robbin.ehn at oracle.com Wed Mar 1 15:07:57 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 1 Mar 2017 16:07:57 +0100 Subject: [10]RFR: 8136650: Add support for custom jtreg native tests In-Reply-To: <201f3bbf-affd-41ff-e0a8-a294934911ec@oracle.com> References: <62136515-9397-08d2-2baa-1957cae14eb2@oracle.com> <201f3bbf-affd-41ff-e0a8-a294934911ec@oracle.com> Message-ID: <9500a8dc-54ae-7fd9-936d-dd4cf0160e7c@oracle.com> Thanks and thanks for adding the correct mailing list! /Robbin On 03/01/2017 02:01 PM, Erik Joelsson wrote: > (adding build-dev) > > Looks good to me. > > /Erik > > > On 2017-02-28 17:52, Robbin Ehn wrote: >> Hi, please review. >> >> This adds support for custom extensions the jtreg native tests. >> >> The issues is closed due to internal tests but contains no more info than this subject: >> https://bugs.openjdk.java.net/browse/JDK-8136650 >> >> Open webrev: >> http://cr.openjdk.java.net/~rehn/8136650/ >> >> Tested with dummy native test. >> >> Thanks, Robbin > From gromero at linux.vnet.ibm.com Wed Mar 1 20:53:47 2017 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Wed, 1 Mar 2017 17:53:47 -0300 Subject: Linux/PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: References: <5898EF94.5060904@linux.vnet.ibm.com> <8a0d9151-8ed7-9409-20a1-10258023ae7d@oracle.com> <58B020D7.3020508@linux.vnet.ibm.com> <4088ef7e-3620-3e6a-fd3f-0ad84baba2c9@oracle.com> Message-ID: <58B734DB.3010800@linux.vnet.ibm.com> Hi Sangheon, David On 27-02-2017 20:17, sangheon.kim at oracle.com wrote: > Hi Gustabvo, > > I am not in the PPC64 ML, so replying late. > >> >>> After a due community review, could you sponsor that change? > Sure, I can sponsor this patch after the review. Please initiate the review on jdk 10 base. Thank you! @David thanks for your comments on current state of the NUMA support. Which bug should I refer to since the one I opened was closed because it was marked as duplicated? Could I reopen it since I don't know if the bug reported before by Sangheon on x64 had the exactly same root cause I found on PPC64? (there is no numa topology information that could help on a guess...) Thanks and best regards, Gustavo > Thanks, > Sangheon > > >> On Feb 27, 2017, at 8:10 AM, David Holmes wrote: >> >> Hi Gustavo, >> >> I am not a NUMA expert but it seems to me that our NUMA support is both incomplete and bit-rotting. It seems evident that UseNUMA is only working in limited contexts that match our testing environment. There were a couple of JEPS proposed to enhance NUMA support back in 2012: >> >> JDK-8046147 JEP 157: G1 GC: NUMA-Aware Allocation >> JDK-8046153 JEP 163: Enable NUMA Mode by Default When Appropriate >> >> but they have not progressed. If they were to progress then it seems our overall approach to NUMA would need serious review and update - as per your patch. >> >> I'm also unclear about the distinctions between memory and non-memory nodes wrt. the existing os::Linux NUMA API. It isn't at all clear to me what functions should only be dealing with memory-nodes and which should deal with any kind eg. I expect cpu to node map to map cpu to nodes not cpu to nearest node with memory configured. If that is what is needed then the API's need to be changed and the usage checked - aas that distinction does not presently exist in the code AFAICS. >> >> It is too late to take this patch into 9 IMHO as we don't have the ability to test it effectively, nor is there time for NUMA users to put it through its paces. I think this would have to be part of a bigger NUMA project for 10 that addresses the NUMA API and how it is used. >> >> Thanks, >> David >> >>> On 24/02/2017 10:02 PM, Gustavo Romero wrote: >>> Hi Sangheon, >>> >>> Please find my comments inline. >>> >>>> On 06-02-2017 20:23, sangheon wrote: >>>> Hi Gustavo, >>>> >>>>> On 02/06/2017 01:50 PM, Gustavo Romero wrote: >>>>> Hi, >>>>> >>>>> On Linux/PPC64 I'm getting a series of "mbind: Invalid argument" that seems >>>>> exactly the same as reported for x64 [1]: >>>>> >>>>> [root at spocfire3 ~]# java -XX:+UseNUMA -version >>>>> mbind: Invalid argument >>>>> mbind: Invalid argument >>>>> mbind: Invalid argument >>>>> mbind: Invalid argument >>>>> mbind: Invalid argument >>>>> mbind: Invalid argument >>>>> mbind: Invalid argument >>>>> openjdk version "1.8.0_121" >>>>> OpenJDK Runtime Environment (build 1.8.0_121-b13) >>>>> OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) >>>>> >>>>> [root at spocfire3 ~]# uname -a >>>>> Linux spocfire3.aus.stglabs.ibm.com 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29 17:31:13 EDT 2015 ppc64le ppc64le ppc64le GNU/Linux >>>>> >>>>> [root at spocfire3 ~]# lscpu >>>>> Architecture: ppc64le >>>>> Byte Order: Little Endian >>>>> CPU(s): 160 >>>>> On-line CPU(s) list: 0-159 >>>>> Thread(s) per core: 8 >>>>> Core(s) per socket: 10 >>>>> Socket(s): 2 >>>>> NUMA node(s): 2 >>>>> Model: 2.0 (pvr 004d 0200) >>>>> Model name: POWER8 (raw), altivec supported >>>>> L1d cache: 64K >>>>> L1i cache: 32K >>>>> L2 cache: 512K >>>>> L3 cache: 8192K >>>>> NUMA node0 CPU(s): 0-79 >>>>> NUMA node8 CPU(s): 80-159 >>>>> >>>>> On chasing down it, looks like it comes from PSYoungGen::initialize() in >>>>> src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp that calls >>>>> initialize_work(), that calls the MutableNUMASpace() constructor if >>>>> UseNUMA is set: >>>>> http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/567e410935e5/src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp#l77 >>>>> >>>>> MutableNUMASpace() then calls os::numa_make_local(), that in the end calls >>>>> numa_set_bind_policy() in libnuma.so.1 [2]. >>>>> >>>>> I've traced some values for which mbind() syscall fails: >>>>> http://termbin.com/ztfs (search for "Invalid argument" in the log). >>>>> >>>>> Assuming it's the same bug as reported in [1] and so it's not fixed on 9 and 10: >>>>> >>>>> - Is there any WIP or known workaround? >>>> There's no progress on JDK-8163796 and no workaround found yet. >>>> And unfortunately, I'm not planning to fix it soon. >>> >>> Hive, a critical component of Hadoop ecosystem, comes with a shell and uses java >>> (with UseNUMA flag) in the background to run mysql-kind of queries. On PPC64 the >>> mbind() messages in question make the shell pretty cumbersome. For instance: >>> >>> hive> show databases; >>> mbind: Invalid argument >>> mbind: Invalid argument >>> mbind: Invalid argument (repeat message more 28 times...) >>> ... >>> OK >>> mbind: Invalid argument >>> mbind: Invalid argument >>> mbind: Invalid argument >>> default >>> tpcds_bin_partitioned_orc_10 >>> tpcds_text_10 >>> Time taken: 1.036 seconds, Fetched: 3 row(s) >>> hive> mbind: Invalid argument >>> mbind: Invalid argument >>> mbind: Invalid argument >>> >>> Also, on PPC64 a simple "java -XX:+UseParallelGC -XX:+UseNUMA -version" will >>> trigger the problem, without any additional flags. So I'd like to correct that >>> behavior (please see my next comment on that). >>> >>> >>>>> - Should I append this output in [1] description or open a new one and make it >>>>> related to" [1]? >>>> I think your problem seems same as JDK-8163796, so adding your output on the CR seems good. >>>> And please add logs as well. I recommend to enabling something like "-Xlog:gc*,gc+heap*=trace". >>>> IIRC, the problem was only occurred when the -Xmx was small in my case. >>> >>> JVM code used to discover which numa nodes it can bind assumes that nodes are >>> consecutive and tries to bind from 0 to numa_max_node() [1, 2, 3, 4], i.e. from >>> 0 to the highest node number available on the system. However, at least on PPC64 >>> that assumption is not always true. For instance, consider the following numa >>> topology: >>> >>> available: 4 nodes (0-1,16-17) >>> node 0 cpus: 0 8 16 24 32 >>> node 0 size: 130706 MB >>> node 0 free: 145 MB >>> node 1 cpus: 40 48 56 64 72 >>> node 1 size: 0 MB >>> node 1 free: 0 MB >>> node 16 cpus: 80 88 96 104 112 >>> node 16 size: 130630 MB >>> node 16 free: 529 MB >>> node 17 cpus: 120 128 136 144 152 >>> node 17 size: 0 MB >>> node 17 free: 0 MB >>> node distances: >>> node 0 1 16 17 >>> 0: 10 20 40 40 >>> 1: 20 10 40 40 >>> 16: 40 40 10 20 >>> 17: 40 40 20 10 >>> >>> In that case we have four nodes, 2 without memory (1 and 17), where the >>> highest node id is 17. Hence if the JVM tries to bind from 0 to 17, mbind() will >>> fail except for nodes 0 and 16, which are configured and have memory. mbind() >>> failures will generate the "mbind: Invalid argument" messages. >>> >>> A solution would be to use in os::numa_get_group_num() not numa_max_node() but >>> instead numa_num_configured_nodes() which returns the total number of nodes with >>> memory in the system (so in our example above it will return exactly 2 nodes) >>> and then inspect numa_all_node_ptr in os::numa_get_leaf_groups() to find the >>> correct node ids to append (in our case, map ids[0] = 0 [node 0] and ids[1] = 16 >>> [node 16]). >>> >>> One thing is that os::numa_get_leaf_groups() argument "size" will not be >>> required anymore and will be loose, so the interface will have to be adapted on >>> other OSs besides Linux I guess [5]. >>> >>> It would be necessary to adapt os::Linux::rebuild_cpu_to_node_map() >>> since not all numa nodes are suitable to be returned by a call to >>> os::numa_get_group_id() as some cpus would be in a node without memory. >>> In that case we can return the closest numa node instead. A new way to translate >>> indices to nodes is also useful since nodes are not always consecutive. >>> >>> Finally, although "numa_nodes_ptr" is not present in libnuma's manual it's what >>> is used in numactl to find out the total number of nodes in the system [6]. I >>> could not find a function that would return that number readily. I asked to >>> libnuma ML if a better solution exists [7]. >>> >>> The following webrev implements the proposed changes on jdk9 (backport to 8 is >>> simple): >>> >>> webrev: http://cr.openjdk.java.net/~gromero/8175813/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8175813 >>> >>> Here are the logs with "-Xlog:gc*,gc+heap*=trace": >>> >>> http://cr.openjdk.java.net/~gromero/logs/pristine.log (current state) >>> http://cr.openjdk.java.net/~gromero/logs/numa_patched.log (proposed change) >>> >>> I've tested on 8 against SPECjvm2008 on the aforementioned machine and >>> performance improved ~5% in comparison to the same version packaged by >>> the distro, but I don't expect any difference on machines where nodes >>> are always consecutive and where nodes always have memory. >>> >>> After a due community review, could you sponsor that change? >>> >>> Thank you. >>> >>> >>> Best regards, >>> Gustavo >>> >>> [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l241 >>> [2] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2745 >>> [3] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l243 >>> [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2761 >>> [5] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/runtime/os.hpp#l356 >>> [6] https://github.com/numactl/numactl/blob/master/numactl.c#L251 >>> [7] http://www.spinics.net/lists/linux-numa/msg01173.html >>> >>>> >>>> Thanks, >>>> Sangheon >>>> >>>> >>>>> >>>>> Thank you. >>>>> >>>>> >>>>> Best regards, >>>>> Gustavo >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8163796 >>>>> [2] https://da.gd/4vXF >>>>> >>>> >>> > From david.holmes at oracle.com Wed Mar 1 23:02:46 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 2 Mar 2017 09:02:46 +1000 Subject: Linux/PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used In-Reply-To: <58B734DB.3010800@linux.vnet.ibm.com> References: <5898EF94.5060904@linux.vnet.ibm.com> <8a0d9151-8ed7-9409-20a1-10258023ae7d@oracle.com> <58B020D7.3020508@linux.vnet.ibm.com> <4088ef7e-3620-3e6a-fd3f-0ad84baba2c9@oracle.com> <58B734DB.3010800@linux.vnet.ibm.com> Message-ID: Hi Gustavo, On 2/03/2017 6:53 AM, Gustavo Romero wrote: > Hi Sangheon, David > > On 27-02-2017 20:17, sangheon.kim at oracle.com wrote: >> Hi Gustabvo, >> >> I am not in the PPC64 ML, so replying late. >> >>> >>>> After a due community review, could you sponsor that change? >> Sure, I can sponsor this patch after the review. Please initiate the review on jdk 10 base. > > Thank you! > > @David thanks for your comments on current state of the NUMA support. > > Which bug should I refer to since the one I opened was closed because it was > marked as duplicated? > > Could I reopen it since I don't know if the bug reported before by Sangheon on > x64 had the exactly same root cause I found on PPC64? (there is no numa topology > information that could help on a guess...) You are right and I have reopened JDK-8175813. While they both have a similar symptom, the EINVAL from mbind can have many different causes and the two failure modes seem likely to be different. That said I'm still concerned that your patch makes a distinction between memory and non-memory nodes that the existing NUMA code seems oblivious of. I would want to see the owners of that code evaluate this patch in the context of that. I also think a lot more work is needed in the NUMA area in general - ref the JEPS I mentioned earlier. Thanks, David > Thanks and best regards, > Gustavo > > >> Thanks, >> Sangheon >> >> >>> On Feb 27, 2017, at 8:10 AM, David Holmes wrote: >>> >>> Hi Gustavo, >>> >>> I am not a NUMA expert but it seems to me that our NUMA support is both incomplete and bit-rotting. It seems evident that UseNUMA is only working in limited contexts that match our testing environment. There were a couple of JEPS proposed to enhance NUMA support back in 2012: >>> >>> JDK-8046147 JEP 157: G1 GC: NUMA-Aware Allocation >>> JDK-8046153 JEP 163: Enable NUMA Mode by Default When Appropriate >>> >>> but they have not progressed. If they were to progress then it seems our overall approach to NUMA would need serious review and update - as per your patch. >>> >>> I'm also unclear about the distinctions between memory and non-memory nodes wrt. the existing os::Linux NUMA API. It isn't at all clear to me what functions should only be dealing with memory-nodes and which should deal with any kind eg. I expect cpu to node map to map cpu to nodes not cpu to nearest node with memory configured. If that is what is needed then the API's need to be changed and the usage checked - aas that distinction does not presently exist in the code AFAICS. >>> >>> It is too late to take this patch into 9 IMHO as we don't have the ability to test it effectively, nor is there time for NUMA users to put it through its paces. I think this would have to be part of a bigger NUMA project for 10 that addresses the NUMA API and how it is used. >>> >>> Thanks, >>> David >>> >>>> On 24/02/2017 10:02 PM, Gustavo Romero wrote: >>>> Hi Sangheon, >>>> >>>> Please find my comments inline. >>>> >>>>> On 06-02-2017 20:23, sangheon wrote: >>>>> Hi Gustavo, >>>>> >>>>>> On 02/06/2017 01:50 PM, Gustavo Romero wrote: >>>>>> Hi, >>>>>> >>>>>> On Linux/PPC64 I'm getting a series of "mbind: Invalid argument" that seems >>>>>> exactly the same as reported for x64 [1]: >>>>>> >>>>>> [root at spocfire3 ~]# java -XX:+UseNUMA -version >>>>>> mbind: Invalid argument >>>>>> mbind: Invalid argument >>>>>> mbind: Invalid argument >>>>>> mbind: Invalid argument >>>>>> mbind: Invalid argument >>>>>> mbind: Invalid argument >>>>>> mbind: Invalid argument >>>>>> openjdk version "1.8.0_121" >>>>>> OpenJDK Runtime Environment (build 1.8.0_121-b13) >>>>>> OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) >>>>>> >>>>>> [root at spocfire3 ~]# uname -a >>>>>> Linux spocfire3.aus.stglabs.ibm.com 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29 17:31:13 EDT 2015 ppc64le ppc64le ppc64le GNU/Linux >>>>>> >>>>>> [root at spocfire3 ~]# lscpu >>>>>> Architecture: ppc64le >>>>>> Byte Order: Little Endian >>>>>> CPU(s): 160 >>>>>> On-line CPU(s) list: 0-159 >>>>>> Thread(s) per core: 8 >>>>>> Core(s) per socket: 10 >>>>>> Socket(s): 2 >>>>>> NUMA node(s): 2 >>>>>> Model: 2.0 (pvr 004d 0200) >>>>>> Model name: POWER8 (raw), altivec supported >>>>>> L1d cache: 64K >>>>>> L1i cache: 32K >>>>>> L2 cache: 512K >>>>>> L3 cache: 8192K >>>>>> NUMA node0 CPU(s): 0-79 >>>>>> NUMA node8 CPU(s): 80-159 >>>>>> >>>>>> On chasing down it, looks like it comes from PSYoungGen::initialize() in >>>>>> src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp that calls >>>>>> initialize_work(), that calls the MutableNUMASpace() constructor if >>>>>> UseNUMA is set: >>>>>> http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/567e410935e5/src/share/vm/gc_implementation/parallelScavenge/psYoungGen.cpp#l77 >>>>>> >>>>>> MutableNUMASpace() then calls os::numa_make_local(), that in the end calls >>>>>> numa_set_bind_policy() in libnuma.so.1 [2]. >>>>>> >>>>>> I've traced some values for which mbind() syscall fails: >>>>>> http://termbin.com/ztfs (search for "Invalid argument" in the log). >>>>>> >>>>>> Assuming it's the same bug as reported in [1] and so it's not fixed on 9 and 10: >>>>>> >>>>>> - Is there any WIP or known workaround? >>>>> There's no progress on JDK-8163796 and no workaround found yet. >>>>> And unfortunately, I'm not planning to fix it soon. >>>> >>>> Hive, a critical component of Hadoop ecosystem, comes with a shell and uses java >>>> (with UseNUMA flag) in the background to run mysql-kind of queries. On PPC64 the >>>> mbind() messages in question make the shell pretty cumbersome. For instance: >>>> >>>> hive> show databases; >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> mbind: Invalid argument (repeat message more 28 times...) >>>> ... >>>> OK >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> default >>>> tpcds_bin_partitioned_orc_10 >>>> tpcds_text_10 >>>> Time taken: 1.036 seconds, Fetched: 3 row(s) >>>> hive> mbind: Invalid argument >>>> mbind: Invalid argument >>>> mbind: Invalid argument >>>> >>>> Also, on PPC64 a simple "java -XX:+UseParallelGC -XX:+UseNUMA -version" will >>>> trigger the problem, without any additional flags. So I'd like to correct that >>>> behavior (please see my next comment on that). >>>> >>>> >>>>>> - Should I append this output in [1] description or open a new one and make it >>>>>> related to" [1]? >>>>> I think your problem seems same as JDK-8163796, so adding your output on the CR seems good. >>>>> And please add logs as well. I recommend to enabling something like "-Xlog:gc*,gc+heap*=trace". >>>>> IIRC, the problem was only occurred when the -Xmx was small in my case. >>>> >>>> JVM code used to discover which numa nodes it can bind assumes that nodes are >>>> consecutive and tries to bind from 0 to numa_max_node() [1, 2, 3, 4], i.e. from >>>> 0 to the highest node number available on the system. However, at least on PPC64 >>>> that assumption is not always true. For instance, consider the following numa >>>> topology: >>>> >>>> available: 4 nodes (0-1,16-17) >>>> node 0 cpus: 0 8 16 24 32 >>>> node 0 size: 130706 MB >>>> node 0 free: 145 MB >>>> node 1 cpus: 40 48 56 64 72 >>>> node 1 size: 0 MB >>>> node 1 free: 0 MB >>>> node 16 cpus: 80 88 96 104 112 >>>> node 16 size: 130630 MB >>>> node 16 free: 529 MB >>>> node 17 cpus: 120 128 136 144 152 >>>> node 17 size: 0 MB >>>> node 17 free: 0 MB >>>> node distances: >>>> node 0 1 16 17 >>>> 0: 10 20 40 40 >>>> 1: 20 10 40 40 >>>> 16: 40 40 10 20 >>>> 17: 40 40 20 10 >>>> >>>> In that case we have four nodes, 2 without memory (1 and 17), where the >>>> highest node id is 17. Hence if the JVM tries to bind from 0 to 17, mbind() will >>>> fail except for nodes 0 and 16, which are configured and have memory. mbind() >>>> failures will generate the "mbind: Invalid argument" messages. >>>> >>>> A solution would be to use in os::numa_get_group_num() not numa_max_node() but >>>> instead numa_num_configured_nodes() which returns the total number of nodes with >>>> memory in the system (so in our example above it will return exactly 2 nodes) >>>> and then inspect numa_all_node_ptr in os::numa_get_leaf_groups() to find the >>>> correct node ids to append (in our case, map ids[0] = 0 [node 0] and ids[1] = 16 >>>> [node 16]). >>>> >>>> One thing is that os::numa_get_leaf_groups() argument "size" will not be >>>> required anymore and will be loose, so the interface will have to be adapted on >>>> other OSs besides Linux I guess [5]. >>>> >>>> It would be necessary to adapt os::Linux::rebuild_cpu_to_node_map() >>>> since not all numa nodes are suitable to be returned by a call to >>>> os::numa_get_group_id() as some cpus would be in a node without memory. >>>> In that case we can return the closest numa node instead. A new way to translate >>>> indices to nodes is also useful since nodes are not always consecutive. >>>> >>>> Finally, although "numa_nodes_ptr" is not present in libnuma's manual it's what >>>> is used in numactl to find out the total number of nodes in the system [6]. I >>>> could not find a function that would return that number readily. I asked to >>>> libnuma ML if a better solution exists [7]. >>>> >>>> The following webrev implements the proposed changes on jdk9 (backport to 8 is >>>> simple): >>>> >>>> webrev: http://cr.openjdk.java.net/~gromero/8175813/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8175813 >>>> >>>> Here are the logs with "-Xlog:gc*,gc+heap*=trace": >>>> >>>> http://cr.openjdk.java.net/~gromero/logs/pristine.log (current state) >>>> http://cr.openjdk.java.net/~gromero/logs/numa_patched.log (proposed change) >>>> >>>> I've tested on 8 against SPECjvm2008 on the aforementioned machine and >>>> performance improved ~5% in comparison to the same version packaged by >>>> the distro, but I don't expect any difference on machines where nodes >>>> are always consecutive and where nodes always have memory. >>>> >>>> After a due community review, could you sponsor that change? >>>> >>>> Thank you. >>>> >>>> >>>> Best regards, >>>> Gustavo >>>> >>>> [1] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l241 >>>> [2] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2745 >>>> [3] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/gc/parallel/mutableNUMASpace.cpp#l243 >>>> [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/os/linux/vm/os_linux.cpp#l2761 >>>> [5] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/636271c3697a/src/share/vm/runtime/os.hpp#l356 >>>> [6] https://github.com/numactl/numactl/blob/master/numactl.c#L251 >>>> [7] http://www.spinics.net/lists/linux-numa/msg01173.html >>>> >>>>> >>>>> Thanks, >>>>> Sangheon >>>>> >>>>> >>>>>> >>>>>> Thank you. >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Gustavo >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8163796 >>>>>> [2] https://da.gd/4vXF >>>>>> >>>>> >>>> >> > From vitalyd at gmail.com Thu Mar 2 00:04:41 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 02 Mar 2017 00:04:41 +0000 Subject: Fw: throwing static exceptions sometimes VERY slow! In-Reply-To: References: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com> Message-ID: +hotspot dev as it got dropped On Wed, Mar 1, 2017 at 7:03 PM Vitaly Davidovich wrote: > Instead of overriding fillInStacktrace, which is a synchronized method and > you're sharing those instances between threads, can you construct the > Continue and Break exceptions telling base class that you don't want > writable stacktrace? > > The jitter you're seeing could be lock contention due to above, biased > lock revocation, JIT deoptimizations (although in your Test I'm not sure if > it'll deoptimize since the throws are not uncommon). > > You can add -XX:+PrintSafepointStatistics > -XX:PrintSafepointStatisticsCount=1 to get reasons for safepoints printed > to stdout. > > As mentioned, run your test with -XX:+PrintCompilation to see if JIT is > doing something. > > On Wed, Mar 1, 2017 at 7:52 AM Wieland, Jacob > wrote: > > > > > sorry, forgot the attachment > > > BR > > > > *Dr. Jacob Wieland* > > *Senior Software Engineer* > > main +49.30.7261919.34 > > mobile +49.173.6446443 > > > jacob.wieland at spirent.com > > > www.spirent.com > > > > > Follow us on: > > > Spirent Communications > > > | > | > > > > Michaelkirchstra?e 17/18 > > 10179 Berlin, Germany > > > > *+++++ N E W S F L A S H +++++* > > > > Spirent Communications and Testing Technologies join forces to become your > test automation power-house. Read more at http://conta.cc/1S62BEY. > > ------------------------------ > *From:* Wieland, Jacob > *Sent:* Wednesday, March 1, 2017 1:47:52 PM > *To:* Vitaly Davidovich; David Holmes > *Cc:* hotspot-dev at openjdk.java.net > > *Subject:* Re: throwing static exceptions sometimes VERY slow! > > Hi, > > > here are the general system infos: > > > uname -a: > > > Linux ber-28753 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 > (2016-01-17) x86_64 GNU/Linux > > > lscpu: > > > Architecture: x86_64 > CPU op-mode(s): 32-bit, 64-bit > Byte Order: Little Endian > CPU(s): 4 > On-line CPU(s) list: 0-3 > Thread(s) per core: 1 > Core(s) per socket: 4 > Socket(s): 1 > NUMA node(s): 1 > Vendor ID: GenuineIntel > CPU family: 6 > Model: 58 > Model name: Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz > Stepping: 9 > CPU MHz: 2094.558 > CPU max MHz: 3500.0000 > CPU min MHz: 1600.0000 > BogoMIPS: 6186.30 > Virtualization: VT-x > L1d cache: 32K > L1i cache: 32K > L2 cache: 256K > L3 cache: 6144K > NUMA node0 CPU(s): 0-3 > > java -version: > > > java version "1.8.0_102" > Java(TM) SE Runtime Environment (build 1.8.0_102-b14) > Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode) > > Rest of the answers inline. > > BR > > > > *Dr. Jacob Wieland* > > *Senior Software Engineer* > > main +49.30.7261919.34 > > mobile +49.173.6446443 > > > jacob.wieland at spirent.com > > > www.spirent.com > > > > > Follow us on: > > > Spirent Communications > > > | > | > > > > Michaelkirchstra?e 17/18 > > 10179 Berlin, Germany > > > > *+++++ N E W S F L A S H +++++* > > > > Spirent Communications and Testing Technologies join forces to become your > test automation power-house. Read more at http://conta.cc/1S62BEY. > > ------------------------------ > *From:* Vitaly Davidovich > *Sent:* Tuesday, February 28, 2017 1:46:09 PM > *To:* David Holmes > *Cc:* Wieland, Jacob; hotspot-dev at openjdk.java.net > *Subject:* Re: throwing static exceptions sometimes VERY slow! > > > > On Tue, Feb 28, 2017 at 7:37 AM, David Holmes > wrote: > > Hi, > > On 28/02/2017 9:02 PM, Wieland, Jacob wrote: > > Hi, > > I am observing a very strange behavior. > > In our generated code (due to various reasons I won't go into here unless > I have to, but trust me, they are legit), we throw static exceptions for > control flow purposes, This seems to work fine and without performance loss > most of the time. However, we are observing now that every few seconds, a > throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to the > normal almost non-measurable time). > > Update: it seems that the observed behavior was kind of related with some > extraneous circumstances (the computer seems to have been more lagged than > usual at that point). By now, I can only observe throws that take up to 20 > milliseconds (which opposed to the normal time still seems slow). > > It does not seem to be GC related, the only idea that I have is the jitter. > > > What jitter? > > I'm assuming Jacob is referring to the JIT. But yes, he needs to provide > more information. > > In particular, it would be good to know the following, as David mentioned, > (off the top of my head): > 1) JVM cmdline flags > > -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=y > > 2) Type of exception thrown (with a stacktrace or not) > > Exception class derived from java.lang.Error that overrides > fillInStackTrace() with 'return this', so stackless. Same instances stored > in static final fields are always thrown. > > 3) Is the call stack depth about the same during the slow and fast throws? > > It is always exactly the same stack, as it happens in a loop. > > 4) Is the exception thrown frequently or infrequently? > > In the test where this was observed, the exception was thrown every 40ms, > so, I guess, frequently. > > 5) Is there -XX:+PrintCompilation output available around the time when > the slowdown is observed > > Unfortunately not, and at the moment, I cannot reproduce the behavior. > > I made a mockup-test (see attachment) that mimicks kind of what my > generated code does and adds measurements before throw and after catch. > Mostly, in my experiments with this, the throw takes on average 20 ns, > but between a 1 and 3 per million iterations take longer than 1 ms, on > average around 7 ms. It can be observed that doing a garbage collection in > a different java process (namely my eclipse that started the test) greatly > influences the frequency of the > 1 ms throws, so maybe it is simply a load > issue. Using multiple threads in parallel also seems to heighten the > > Also, when introducing an additional sleep between iterations (like in my > original scenario where I was waiting for a timer to timeout before > executing the construct that used the throw), I cannot reproduce such > outliers at all. > > > > So, my question is. Is this a known (and for some strange reason maybe > even accepted) behavior or is this a bug that I should file with Oracle (or > you guys). > > > You really are not giving us anything to go on here. How are you observing > this slowness? Exactly how do you throw? What exactly are you measuring? > What's the execution context, the machine, processors etc etc etc. > > > I was observing the slowness in a testcase where I wanted regular > intervals of 40 ms between sending a message via UDP but frequently (every > few seconds) got a lag of around 1.5 seconds (and a few days later 0.5 > seconds) before the timer which timed out at the correct time was noted. > > > When eliminating one kind of such thrown control exception, the next kind > of thrown control exception (from a surrounding construct) was having the > same effect which actually led me to the slow-throw-hypothesis. Eliminating > all exceptions in the small example confirmed this hypothesis, but going > that route would take a lot of effort for all cases and introduce other > additional imperformances (as I basically have to mock the exception > throwing via cascading return and if-checks on every level) which I would > like to avoid. > > Regards, > David > > > > > > Spirent Communications e-mail confidentiality. > ------------------------------------------------------------------------ > This e-mail contains confidential and / or privileged information > belonging to Spirent Communications plc, its affiliates and / or > subsidiaries. If you are not the intended recipient, you are hereby > notified that any disclosure, copying, distribution and / or the taking of > any action based upon reliance on the contents of this transmission is > strictly forbidden. If you have received this message in error please > notify the sender by return e-mail and delete it from your system. > > Spirent Communications plc > Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United > Kingdom. > Tel No. +44 (0) 1293 767676 > Fax No. +44 (0) 1293 767677 > > Registered in England Number 470893 > Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 > 9XN, United Kingdom. > > Or if within the US, > > Spirent Communications, > 27349 Agoura Road, Calabasas, CA, 91301, USA. > Tel No. 1-818-676- 2300 > > -- > Sent from my phone > -- Sent from my phone From david.holmes at oracle.com Thu Mar 2 02:51:08 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 2 Mar 2017 12:51:08 +1000 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> Message-ID: <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> I've moved this discussion over to hotspot-dev as this seems no longer a build issue but a C1 soft-float issue. David On 2/03/2017 12:35 PM, Zhu Yong wrote: > If run with -Xint, it works. > > I have created simplified FP test by remove all non-related code from > Whetstone test code. > The test code is here: > https://gist.github.com/yongzhy/d65c26d39fe5d549d1b406c7c84eacd4 > > I am not sure if the test code can help or not. The behaviour is weird : > - If I print both itime and q, program will run OK > - inside condition block if(q<1.0f), if I use exit code 2,3,4,5, problem > appears, however, if I use exit code >=6, program run OK. > > I always get exit code 5, the line q = (double)itime is wrong? > > public static double getTime() > { > double q; > long itime; > itime = System.currentTimeMillis(); > if(itime<1000000) { > System.exit(1); > } > //System.out.printf("time long value %d\n", itime); > q = (double) itime; > if(q < 1.0f) { > System.exit(5); // I always get exit code 5 > } > //System.out.printf("time float value %f\n", q); > return q / 1000.0; > } > > > > On Wed, Mar 1, 2017 at 5:31 PM, David Holmes > wrote: > > On 1/03/2017 6:45 PM, Zhu Yong wrote: > > OK, for the Whetstone Benchmark, I added debug printing and > found it's > due to function getTime(), it only get good value on 1st call, all > following calls get 0s. > Debug printing of itime value is correct. So reason should be > the return > division line. Could it because toolchain's floating point > operation??? > But why server VM Ok? > > > Server and client implement FP logic differently, and particularly > as this is soft-fp, they may well not be in sync. Does -Xint work? > > Can you isolate to a simple FP test? > > David > > public static double getTime() > { > double q; > long itime; > itime = System.currentTimeMillis(); > q = (double) itime; > return q / 1000.0; > } > > On Wed, Mar 1, 2017 at 3:14 PM, David Holmes > > >> wrote: > > On 1/03/2017 4:26 PM, Zhu Yong wrote: > > We use armv7-marvell-linux-gnueabi-softfp toolchain > > gcc version 4.6.4 (Marvell GCC release 20150204-c4af733b 64K > MAXPAGESIZE > ALIGN CVE-2015-0235) > > Is this toolchain too old? I am asking because I saw > other strange > issues as well: > > > We last used gcc 4.7.2. I can't really say if 4.6.4 is "too > old". > > I have successfully build server, client and minimal VM, > when I run > Dhrystone benchmark file (the same jar file from > http://www.okayan.jp/DhrystoneApplet/ > > >), the performance from > server VM > is much better than client and minimal VM. > (minimal: 1629852, client: 1615508, server: 2338871 ) > > > That's expected. The server VM is high performance. > > And when run Whetstone Benchmark > from > http://www.roylongbottom.org.uk/online/whetjava2.html > > >, server VM > finished with good result, client and minimal VM not > able to finish > (program stuck there forever after print 1st line of > output). > > > That needs investigating. You need to try and generate a > stackdump > to see where things are stuck. > > David > > > On Wed, Mar 1, 2017 at 1:56 PM, David Holmes > > > > >>> wrote: > > On 1/03/2017 3:39 PM, Zhu Yong wrote: > > Thank you for the information, I managed to make > minimal > build > pass now. > > Initially I though JVM_EXCLUDE_FILES need to add > additional 3 > generated > files (they appeared > in _BUILD_LIBJVM_objectfilenames.txt) : > bytecodeInterpreterWithChecks.cpp jvmtiEnter.cpp > jvmtiEnterTrace.cpp > But build still fail with same error message. > > Finally I figured out it's due to those 8 doit() > functions > implementation in jvmtiEnvBase.hpp file. After > move all > those 8 > doit() > implementations to jvmtiEnvBase.cpp, build of > minimal VM > passed > without > any issue. > > > That seems to be a compiler issue. jvmtiEnvBase.hpp is > unconditionally included in a couple of places > because we > have to > have enough of the JVMTI code to say JVMTI is not > available. > Those > doit() implementations, though arguably could be > conditional on > INCLUDE_JVMTI, are not referenced by any called code > and so > should > not linked in. But in your case it seems they are. > > What toolchain are you using? > > David > ----- > > Changes appended > ============= > > --- > .../src/share/vm/prims/jvmtiEnvBase.cpp > | 74 > ++++++++++++++++++++++ > .../src/share/vm/prims/jvmtiEnvBase.hpp > | 68 > +++----------------- > 2 files changed, 82 insertions(+), 60 deletions(-) > mode change 100644 => 100755 > > hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > mode change 100644 => 100755 > > hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > > diff --git > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > old mode 100644 > new mode 100755 > index dd241a0..e5832ba > --- > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > +++ > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > @@ -1283,6 +1283,80 @@ > > VM_GetMultipleStackTraces::allocate_and_fill_stacks(jint > thread_count) { > "the last copied frame info must be > the last > record"); > } > > +void > +VM_UpdateForPopTopFrame::doit() { > + JavaThread* jt = _state->get_thread(); > + if (Threads::includes(jt) && !jt->is_exiting() && > jt->threadObj() != > NULL) { > + _state->update_for_pop_top_frame(); > + } else { > + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > + } > +} > + > +void > +VM_SetFramePop::doit() { > + JavaThread* jt = _state->get_thread(); > + if (Threads::includes(jt) && !jt->is_exiting() && > jt->threadObj() != > NULL) { > + int frame_number = _state->count_frames() - > _depth; > + > > > _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_pop(frame_number); > + } else { > + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > + } > +} > + > +void > +VM_GetOwnedMonitorInfo::doit() { > + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > + if (Threads::includes(_java_thread) && > !_java_thread->is_exiting() > + && > _java_thread->threadObj() != > NULL) { > + _result = ((JvmtiEnvBase > *)_env)->get_owned_monitors(_calling_thread, > _java_thread, > + > _owned_monitors_list); > + } > +} > + > +void > +VM_GetObjectMonitorUsage::doit() { > + _result = ((JvmtiEnvBase*) > _env)->get_object_monitor_usage(_calling_thread, > _object, > _info_ptr); > +} > + > +void > +VM_GetCurrentContendedMonitor::doit() { > + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > + if (Threads::includes(_java_thread) && > !_java_thread->is_exiting() && > + _java_thread->threadObj() != NULL) { > + _result = ((JvmtiEnvBase > > > *)_env)->get_current_contended_monitor(_calling_thread,_java_thread,_owned_monitor_ptr); > + } > +} > + > +void > +VM_GetStackTrace::doit() { > + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > + if (Threads::includes(_java_thread) && > !_java_thread->is_exiting() > + && > _java_thread->threadObj() != > NULL) { > + _result = ((JvmtiEnvBase > *)_env)->get_stack_trace(_java_thread, > + > _start_depth, > _max_count, > + > _frame_buffer, > _count_ptr); > + } > +} > + > +void > +VM_GetFrameCount::doit() { > + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > + JavaThread* jt = _state->get_thread(); > + if (Threads::includes(jt) && !jt->is_exiting() && > jt->threadObj() != > NULL) { > + _result = > ((JvmtiEnvBase*)_env)->get_frame_count(_state, > _count_ptr); > + } > +} > + > +void > +VM_GetFrameLocation::doit() { > + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > + if (Threads::includes(_java_thread) && > !_java_thread->is_exiting() && > + _java_thread->threadObj() != NULL) { > + _result = > > ((JvmtiEnvBase*)_env)->get_frame_location(_java_thread, > _depth, > + > _method_ptr, > _location_ptr); > + } > +} > > void > VM_GetThreadListStackTraces::doit() { > diff --git > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > old mode 100644 > new mode 100755 > index 04e6869..00b9890 > --- > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > +++ > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > @@ -359,14 +359,7 @@ public: > } > VMOp_Type type() const { return > VMOp_UpdateForPopTopFrame; } > jvmtiError result() { return _result; } > - void doit() { > - JavaThread* jt = _state->get_thread(); > - if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - _state->update_for_pop_top_frame(); > - } else { > - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > - } > - } > + void doit(); > }; > > // VM operation to set frame pop. > @@ -389,15 +382,7 @@ public: > bool allow_nested_vm_operations() const { > return true; } > VMOp_Type type() const { return > VMOp_SetFramePop; } > jvmtiError result() { return _result; } > - void doit() { > - JavaThread* jt = _state->get_thread(); > - if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - int frame_number = _state->count_frames() > - _depth; > - > > > _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_pop(frame_number); > - } else { > - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > - } > - } > + void doit(); > }; > > > @@ -421,14 +406,7 @@ public: > _result = JVMTI_ERROR_NONE; > } > VMOp_Type type() const { return > VMOp_GetOwnedMonitorInfo; } > - void doit() { > - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > - if (Threads::includes(_java_thread) && > !_java_thread->is_exiting() > - && > _java_thread->threadObj() != > NULL) { > - _result = ((JvmtiEnvBase > *)_env)->get_owned_monitors(_calling_thread, > _java_thread, > - > _owned_monitors_list); > - } > - } > + void doit(); > jvmtiError result() { return _result; } > }; > > @@ -451,10 +429,7 @@ public: > } > VMOp_Type type() const { return > VMOp_GetObjectMonitorUsage; } > jvmtiError result() { return _result; } > - void doit() { > - _result = ((JvmtiEnvBase*) > _env)->get_object_monitor_usage(_calling_thread, > _object, > _info_ptr); > - } > - > + void doit(); > }; > > // VM operation to get current contended monitor. > @@ -475,13 +450,7 @@ public: > } > VMOp_Type type() const { return > VMOp_GetCurrentContendedMonitor; } > jvmtiError result() { return _result; } > - void doit() { > - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > - if (Threads::includes(_java_thread) && > !_java_thread->is_exiting() && > - _java_thread->threadObj() != NULL) { > - _result = ((JvmtiEnvBase > > > *)_env)->get_current_contended_monitor(_calling_thread,_java_thread,_owned_monitor_ptr); > - } > - } > + void doit(); > }; > > // VM operation to get stack trace at safepoint. > @@ -508,15 +477,7 @@ public: > } > jvmtiError result() { return _result; } > VMOp_Type type() const { return > VMOp_GetStackTrace; } > - void doit() { > - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > - if (Threads::includes(_java_thread) && > !_java_thread->is_exiting() > - && > _java_thread->threadObj() != > NULL) { > - _result = ((JvmtiEnvBase > *)_env)->get_stack_trace(_java_thread, > - > _start_depth, > _max_count, > - > _frame_buffer, > _count_ptr); > - } > - } > + void doit(); > }; > > // forward declaration > @@ -606,13 +567,7 @@ public: > } > VMOp_Type type() const { return > VMOp_GetFrameCount; } > jvmtiError result() { return _result; } > - void doit() { > - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > - JavaThread* jt = _state->get_thread(); > - if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - _result = > ((JvmtiEnvBase*)_env)->get_frame_count(_state, > _count_ptr); > - } > - } > + void doit(); > }; > > // VM operation to frame location at safepoint. > @@ -636,14 +591,7 @@ public: > } > VMOp_Type type() const { return > VMOp_GetFrameLocation; } > jvmtiError result() { return _result; } > - void doit() { > - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; > - if (Threads::includes(_java_thread) && > !_java_thread->is_exiting() && > - _java_thread->threadObj() != NULL) { > - _result = > > ((JvmtiEnvBase*)_env)->get_frame_location(_java_thread, > _depth, > - > _method_ptr, > _location_ptr); > - } > - } > + void doit(); > }; > > > -- > 2.1.4 > > > On Tue, Feb 28, 2017 at 6:11 PM, David Holmes > > > > > >> > > > > > > >>>> wrote: > > On 28/02/2017 7:35 PM, David Holmes wrote: > > On 28/02/2017 6:51 PM, Zhu Yong wrote: > > Dear All, > > I am testing cross compile > OpenJDK-9+158 for our > embedded > system using > buildroot, I can build server and client > variants > successfully, but the > output compact1 profile file size is too > big, then I > tried > to build > minimal > variant, failed when linking libjvm.so. > > I checked > > > > build/linux-arm-normal-minimal-release/hotspot/variant-minimal/ > > libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt, > jvmtiEnvBase.o > and jvmtiEnvThreadState.o are not > listed in > the file > (which > is required > from the error message below). > > > Right - JVM TI is not part of the > Minimal VM. At the > moment it looks > like some service has either been > enabled when > it should not > have been, > or has not been correctly if'def in the > source. > > > As far as I can see your error messages are > complaining about > missing functions that are called from the same > sources as the > functions that are missing! ie. > > undefined reference to > > `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, > JavaThread*, > _jobject**)' > /tmp/cc27HS5M.ltrans0.ltrans.o: In function > `VM_GetOwnedMonitorInfo::doit() > > but VM_GetOwnedMonitorInfo is defined in > jvmtiEnv.cpp which > should > be included or excluded the same as > jvmtiEnBase.cpp. > Here's the > logic in the makefile that controls this: > > ifneq ($(call check-jvm-feature, jvmti), true) > JVM_CFLAGS_FEATURES += -DINCLUDE_JVMTI=0 > JVM_EXCLUDE_FILES += jvmtiGetLoadedClasses.cpp > jvmtiThreadState.cpp jvmtiExtensions.cpp \ > jvmtiImpl.cpp jvmtiManageCapabilities.cpp > jvmtiRawMonitor.cpp > jvmtiUtil.cpp jvmtiTrace.cpp \ > jvmtiCodeBlobEvents.cpp jvmtiEnv.cpp > jvmtiRedefineClasses.cpp > jvmtiEnvBase.cpp jvmtiEnvThreadState.cpp \ > jvmtiTagMap.cpp jvmtiEventController.cpp > evmCompat.cpp > jvmtiEnter.xsl jvmtiExport.cpp \ > jvmtiClassFileReconstituter.cpp > endif > > Can you run with LOG=trace so that each > compiled file is > listed in > the build log, then see if there are any jvmti* > files listed > there. > > Thanks, > David > > > > > Can you provide the lines from your spec.gmk > that define the > JVM_FEATURES_* variables please. > > Thanks, > David > ------ > > > === Output from failing command(s) > repeated > here === > * For target > > hotspot_variant-minimal_libjvm_objs_BUILD_LIBJVM_link: > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_GetStackTrace::doit() > [clone .local.640]': > cc27HS5M.ltrans0.o:(.text+0xce5e): > undefined > reference to > > `JvmtiEnvBase::get_stack_trace(JavaThread*, > int, int, > _jvmtiFrameInfo*, > int*)' > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_GetCurrentContendedMonitor::doit() > [clone .local.639]': > cc27HS5M.ltrans0.o:(.text+0xcec2): > undefined > reference to > > > `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, > JavaThread*, > _jobject**)' > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_GetOwnedMonitorInfo::doit() > [clone .local.638]': > cc27HS5M.ltrans0.o:(.text+0xcf26): > undefined > reference to > > `JvmtiEnvBase::get_owned_monitors(JavaThread*, > JavaThread*, > GrowableArray<_ > jvmtiMonitorStackDepthInfo*>*)' > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_GetFrameCount::doit() > [clone .local.637]': > cc27HS5M.ltrans0.o:(.text+0xcf8a): > undefined > reference to > > `JvmtiEnvBase::get_frame_count(JvmtiThreadState*, int*)' > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_SetFramePop::doit() > [clone > .local.636]': > cc27HS5M.ltrans0.o:(.text+0xcfea): > undefined > reference to > `JvmtiThreadState::count_frames()' > cc27HS5M.ltrans0.o:(.text+0xd030): > undefined > reference to > > `JvmtiEnvThreadState::set_frame_pop(int)' > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_GetFrameLocation::doit() > [clone .local.641]': > ... (rest of output omitted) > > > My configuration: > > --with-jdk-variant=normal \ > --with-jvm-variants=minimal \ > --with-debug-level=release \ > --disable-warnings-as-errors \ > --disable-hotspot-gtest \ > --with-abi-profile=arm-vfp-sflt \ > --openjdk-target=$(GNU_TARGET_NAME) \ > --with-sys-root=$(STAGING_DIR) \ > --with-tools-dir=$(HOST_DIR) \ > > --with-freetype-include=$(STAGING_DIR)/usr/include \ > > --with-freetype-lib=$(STAGING_DIR)/usr/lib \ > --with-freetype=$(STAGING_DIR)/usr/ \ > > --with-alsa-include=$(STAGING_DIR)/usr/include \ > --with-alsa-lib=$(STAGING_DIR)/usr/lib \ > --with-alsa=$(STAGING_DIR)/usr/ \ > --with-cups=$(STAGING_DIR)/usr/ \ > --with-x=$(STAGING_DIR)/usr/include \ > > --with-extra-ldflags=--sysroot=$(STAGING_DIR) \ > --enable-headless-only \ > --disable-freetype-bundling \ > --enable-unlimited-crypto \ > --with-boot-jdk=/opt/java/jdk1.8.0_102 > > > > > From david.holmes at oracle.com Thu Mar 2 04:32:00 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 2 Mar 2017 14:32:00 +1000 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> Message-ID: <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> I built your sample program with latest JDK 9 for arm-vfp-sflt and ran on an emulator I have available, and I had no issues. So I still have concerns this may be a build/compiler issue. David On 2/03/2017 12:51 PM, David Holmes wrote: > I've moved this discussion over to hotspot-dev as this seems no longer a > build issue but a C1 soft-float issue. > > David > > On 2/03/2017 12:35 PM, Zhu Yong wrote: >> If run with -Xint, it works. >> >> I have created simplified FP test by remove all non-related code from >> Whetstone test code. >> The test code is here: >> https://gist.github.com/yongzhy/d65c26d39fe5d549d1b406c7c84eacd4 >> >> I am not sure if the test code can help or not. The behaviour is weird : >> - If I print both itime and q, program will run OK >> - inside condition block if(q<1.0f), if I use exit code 2,3,4,5, problem >> appears, however, if I use exit code >=6, program run OK. >> >> I always get exit code 5, the line q = (double)itime is wrong? >> >> public static double getTime() >> { >> double q; >> long itime; >> itime = System.currentTimeMillis(); >> if(itime<1000000) { >> System.exit(1); >> } >> //System.out.printf("time long value %d\n", itime); >> q = (double) itime; >> if(q < 1.0f) { >> System.exit(5); // I always get exit code 5 >> } >> //System.out.printf("time float value %f\n", q); >> return q / 1000.0; >> } >> >> >> >> On Wed, Mar 1, 2017 at 5:31 PM, David Holmes > > wrote: >> >> On 1/03/2017 6:45 PM, Zhu Yong wrote: >> >> OK, for the Whetstone Benchmark, I added debug printing and >> found it's >> due to function getTime(), it only get good value on 1st call, >> all >> following calls get 0s. >> Debug printing of itime value is correct. So reason should be >> the return >> division line. Could it because toolchain's floating point >> operation??? >> But why server VM Ok? >> >> >> Server and client implement FP logic differently, and particularly >> as this is soft-fp, they may well not be in sync. Does -Xint work? >> >> Can you isolate to a simple FP test? >> >> David >> >> public static double getTime() >> { >> double q; >> long itime; >> itime = System.currentTimeMillis(); >> q = (double) itime; >> return q / 1000.0; >> } >> >> On Wed, Mar 1, 2017 at 3:14 PM, David Holmes >> >> > >> wrote: >> >> On 1/03/2017 4:26 PM, Zhu Yong wrote: >> >> We use armv7-marvell-linux-gnueabi-softfp toolchain >> >> gcc version 4.6.4 (Marvell GCC release >> 20150204-c4af733b 64K >> MAXPAGESIZE >> ALIGN CVE-2015-0235) >> >> Is this toolchain too old? I am asking because I saw >> other strange >> issues as well: >> >> >> We last used gcc 4.7.2. I can't really say if 4.6.4 is "too >> old". >> >> I have successfully build server, client and minimal VM, >> when I run >> Dhrystone benchmark file (the same jar file from >> http://www.okayan.jp/DhrystoneApplet/ >> >> > >), the performance from >> server VM >> is much better than client and minimal VM. >> (minimal: 1629852, client: 1615508, server: 2338871 ) >> >> >> That's expected. The server VM is high performance. >> >> And when run Whetstone Benchmark >> from >> http://www.roylongbottom.org.uk/online/whetjava2.html >> >> > >, >> server VM >> finished with good result, client and minimal VM not >> able to finish >> (program stuck there forever after print 1st line of >> output). >> >> >> That needs investigating. You need to try and generate a >> stackdump >> to see where things are stuck. >> >> David >> >> >> On Wed, Mar 1, 2017 at 1:56 PM, David Holmes >> > > > >> > >> > >>> wrote: >> >> On 1/03/2017 3:39 PM, Zhu Yong wrote: >> >> Thank you for the information, I managed to make >> minimal >> build >> pass now. >> >> Initially I though JVM_EXCLUDE_FILES need to add >> additional 3 >> generated >> files (they appeared >> in _BUILD_LIBJVM_objectfilenames.txt) : >> bytecodeInterpreterWithChecks.cpp jvmtiEnter.cpp >> jvmtiEnterTrace.cpp >> But build still fail with same error message. >> >> Finally I figured out it's due to those 8 doit() >> functions >> implementation in jvmtiEnvBase.hpp file. After >> move all >> those 8 >> doit() >> implementations to jvmtiEnvBase.cpp, build of >> minimal VM >> passed >> without >> any issue. >> >> >> That seems to be a compiler issue. >> jvmtiEnvBase.hpp is >> unconditionally included in a couple of places >> because we >> have to >> have enough of the JVMTI code to say JVMTI is not >> available. >> Those >> doit() implementations, though arguably could be >> conditional on >> INCLUDE_JVMTI, are not referenced by any called code >> and so >> should >> not linked in. But in your case it seems they are. >> >> What toolchain are you using? >> >> David >> ----- >> >> Changes appended >> ============= >> >> --- >> .../src/share/vm/prims/jvmtiEnvBase.cpp >> | 74 >> ++++++++++++++++++++++ >> .../src/share/vm/prims/jvmtiEnvBase.hpp >> | 68 >> +++----------------- >> 2 files changed, 82 insertions(+), 60 >> deletions(-) >> mode change 100644 => 100755 >> >> hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> mode change 100644 => 100755 >> >> hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> >> diff --git >> >> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> >> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> old mode 100644 >> new mode 100755 >> index dd241a0..e5832ba >> --- >> >> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> +++ >> >> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> @@ -1283,6 +1283,80 @@ >> >> VM_GetMultipleStackTraces::allocate_and_fill_stacks(jint >> thread_count) { >> "the last copied frame info must be >> the last >> record"); >> } >> >> +void >> +VM_UpdateForPopTopFrame::doit() { >> + JavaThread* jt = _state->get_thread(); >> + if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + _state->update_for_pop_top_frame(); >> + } else { >> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> + } >> +} >> + >> +void >> +VM_SetFramePop::doit() { >> + JavaThread* jt = _state->get_thread(); >> + if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + int frame_number = _state->count_frames() - >> _depth; >> + >> >> >> >> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_pop(frame_number); >> >> + } else { >> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> + } >> +} >> + >> +void >> +VM_GetOwnedMonitorInfo::doit() { >> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() >> + && >> _java_thread->threadObj() != >> NULL) { >> + _result = ((JvmtiEnvBase >> *)_env)->get_owned_monitors(_calling_thread, >> _java_thread, >> + >> _owned_monitors_list); >> + } >> +} >> + >> +void >> +VM_GetObjectMonitorUsage::doit() { >> + _result = ((JvmtiEnvBase*) >> _env)->get_object_monitor_usage(_calling_thread, >> _object, >> _info_ptr); >> +} >> + >> +void >> +VM_GetCurrentContendedMonitor::doit() { >> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() && >> + _java_thread->threadObj() != NULL) { >> + _result = ((JvmtiEnvBase >> >> >> >> *)_env)->get_current_contended_monitor(_calling_thread,_java_thread,_owned_monitor_ptr); >> >> + } >> +} >> + >> +void >> +VM_GetStackTrace::doit() { >> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() >> + && >> _java_thread->threadObj() != >> NULL) { >> + _result = ((JvmtiEnvBase >> *)_env)->get_stack_trace(_java_thread, >> + >> _start_depth, >> _max_count, >> + >> _frame_buffer, >> _count_ptr); >> + } >> +} >> + >> +void >> +VM_GetFrameCount::doit() { >> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> + JavaThread* jt = _state->get_thread(); >> + if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + _result = >> ((JvmtiEnvBase*)_env)->get_frame_count(_state, >> _count_ptr); >> + } >> +} >> + >> +void >> +VM_GetFrameLocation::doit() { >> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() && >> + _java_thread->threadObj() != NULL) { >> + _result = >> >> ((JvmtiEnvBase*)_env)->get_frame_location(_java_thread, >> _depth, >> + >> _method_ptr, >> _location_ptr); >> + } >> +} >> >> void >> VM_GetThreadListStackTraces::doit() { >> diff --git >> >> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> >> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> old mode 100644 >> new mode 100755 >> index 04e6869..00b9890 >> --- >> >> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> +++ >> >> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> @@ -359,14 +359,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_UpdateForPopTopFrame; } >> jvmtiError result() { return _result; } >> - void doit() { >> - JavaThread* jt = _state->get_thread(); >> - if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() >> != NULL) { >> - _state->update_for_pop_top_frame(); >> - } else { >> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> - } >> - } >> + void doit(); >> }; >> >> // VM operation to set frame pop. >> @@ -389,15 +382,7 @@ public: >> bool allow_nested_vm_operations() const { >> return true; } >> VMOp_Type type() const { return >> VMOp_SetFramePop; } >> jvmtiError result() { return _result; } >> - void doit() { >> - JavaThread* jt = _state->get_thread(); >> - if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() >> != NULL) { >> - int frame_number = _state->count_frames() >> - _depth; >> - >> >> >> >> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_pop(frame_number); >> >> - } else { >> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> - } >> - } >> + void doit(); >> }; >> >> >> @@ -421,14 +406,7 @@ public: >> _result = JVMTI_ERROR_NONE; >> } >> VMOp_Type type() const { return >> VMOp_GetOwnedMonitorInfo; } >> - void doit() { >> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> - if (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() >> - && >> _java_thread->threadObj() != >> NULL) { >> - _result = ((JvmtiEnvBase >> *)_env)->get_owned_monitors(_calling_thread, >> _java_thread, >> - >> _owned_monitors_list); >> - } >> - } >> + void doit(); >> jvmtiError result() { return _result; } >> }; >> >> @@ -451,10 +429,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_GetObjectMonitorUsage; } >> jvmtiError result() { return _result; } >> - void doit() { >> - _result = ((JvmtiEnvBase*) >> _env)->get_object_monitor_usage(_calling_thread, >> _object, >> _info_ptr); >> - } >> - >> + void doit(); >> }; >> >> // VM operation to get current contended >> monitor. >> @@ -475,13 +450,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_GetCurrentContendedMonitor; } >> jvmtiError result() { return _result; } >> - void doit() { >> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> - if (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() && >> - _java_thread->threadObj() != NULL) { >> - _result = ((JvmtiEnvBase >> >> >> >> *)_env)->get_current_contended_monitor(_calling_thread,_java_thread,_owned_monitor_ptr); >> >> - } >> - } >> + void doit(); >> }; >> >> // VM operation to get stack trace at safepoint. >> @@ -508,15 +477,7 @@ public: >> } >> jvmtiError result() { return _result; } >> VMOp_Type type() const { return >> VMOp_GetStackTrace; } >> - void doit() { >> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> - if (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() >> - && >> _java_thread->threadObj() != >> NULL) { >> - _result = ((JvmtiEnvBase >> *)_env)->get_stack_trace(_java_thread, >> - >> _start_depth, >> _max_count, >> - >> _frame_buffer, >> _count_ptr); >> - } >> - } >> + void doit(); >> }; >> >> // forward declaration >> @@ -606,13 +567,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_GetFrameCount; } >> jvmtiError result() { return _result; } >> - void doit() { >> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> - JavaThread* jt = _state->get_thread(); >> - if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() >> != NULL) { >> - _result = >> ((JvmtiEnvBase*)_env)->get_frame_count(_state, >> _count_ptr); >> - } >> - } >> + void doit(); >> }; >> >> // VM operation to frame location at safepoint. >> @@ -636,14 +591,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_GetFrameLocation; } >> jvmtiError result() { return _result; } >> - void doit() { >> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >> - if (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() && >> - _java_thread->threadObj() != NULL) { >> - _result = >> >> ((JvmtiEnvBase*)_env)->get_frame_location(_java_thread, >> _depth, >> - >> _method_ptr, >> _location_ptr); >> - } >> - } >> + void doit(); >> }; >> >> >> -- >> 2.1.4 >> >> >> On Tue, Feb 28, 2017 at 6:11 PM, David Holmes >> > >> > > >> >> > >> >> > >> > > >> >> > >> > >>>> wrote: >> >> On 28/02/2017 7:35 PM, David Holmes wrote: >> >> On 28/02/2017 6:51 PM, Zhu Yong wrote: >> >> Dear All, >> >> I am testing cross compile >> OpenJDK-9+158 for our >> embedded >> system using >> buildroot, I can build server and >> client >> variants >> successfully, but the >> output compact1 profile file size >> is too >> big, then I >> tried >> to build >> minimal >> variant, failed when linking >> libjvm.so. >> >> I checked >> >> >> >> build/linux-arm-normal-minimal-release/hotspot/variant-minimal/ >> >> libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt, >> jvmtiEnvBase.o >> and jvmtiEnvThreadState.o are not >> listed in >> the file >> (which >> is required >> from the error message below). >> >> >> Right - JVM TI is not part of the >> Minimal VM. At the >> moment it looks >> like some service has either been >> enabled when >> it should not >> have been, >> or has not been correctly if'def in the >> source. >> >> >> As far as I can see your error messages are >> complaining about >> missing functions that are called from the >> same >> sources as the >> functions that are missing! ie. >> >> undefined reference to >> >> `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, >> JavaThread*, >> _jobject**)' >> /tmp/cc27HS5M.ltrans0.ltrans.o: In function >> `VM_GetOwnedMonitorInfo::doit() >> >> but VM_GetOwnedMonitorInfo is defined in >> jvmtiEnv.cpp which >> should >> be included or excluded the same as >> jvmtiEnBase.cpp. >> Here's the >> logic in the makefile that controls this: >> >> ifneq ($(call check-jvm-feature, jvmti), >> true) >> JVM_CFLAGS_FEATURES += -DINCLUDE_JVMTI=0 >> JVM_EXCLUDE_FILES += >> jvmtiGetLoadedClasses.cpp >> jvmtiThreadState.cpp jvmtiExtensions.cpp \ >> jvmtiImpl.cpp >> jvmtiManageCapabilities.cpp >> jvmtiRawMonitor.cpp >> jvmtiUtil.cpp jvmtiTrace.cpp \ >> jvmtiCodeBlobEvents.cpp jvmtiEnv.cpp >> jvmtiRedefineClasses.cpp >> jvmtiEnvBase.cpp jvmtiEnvThreadState.cpp \ >> jvmtiTagMap.cpp >> jvmtiEventController.cpp >> evmCompat.cpp >> jvmtiEnter.xsl jvmtiExport.cpp \ >> jvmtiClassFileReconstituter.cpp >> endif >> >> Can you run with LOG=trace so that each >> compiled file is >> listed in >> the build log, then see if there are any >> jvmti* >> files listed >> there. >> >> Thanks, >> David >> >> >> >> >> Can you provide the lines from your >> spec.gmk >> that define the >> JVM_FEATURES_* variables please. >> >> Thanks, >> David >> ------ >> >> >> === Output from failing command(s) >> repeated >> here === >> * For target >> >> hotspot_variant-minimal_libjvm_objs_BUILD_LIBJVM_link: >> /tmp/cc27HS5M.ltrans0.ltrans.o: In >> function >> `VM_GetStackTrace::doit() >> [clone .local.640]': >> cc27HS5M.ltrans0.o:(.text+0xce5e): >> undefined >> reference to >> >> `JvmtiEnvBase::get_stack_trace(JavaThread*, >> int, int, >> _jvmtiFrameInfo*, >> int*)' >> /tmp/cc27HS5M.ltrans0.ltrans.o: In >> function >> >> `VM_GetCurrentContendedMonitor::doit() >> [clone .local.639]': >> cc27HS5M.ltrans0.o:(.text+0xcec2): >> undefined >> reference to >> >> >> `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, >> JavaThread*, >> _jobject**)' >> /tmp/cc27HS5M.ltrans0.ltrans.o: In >> function >> `VM_GetOwnedMonitorInfo::doit() >> [clone .local.638]': >> cc27HS5M.ltrans0.o:(.text+0xcf26): >> undefined >> reference to >> >> `JvmtiEnvBase::get_owned_monitors(JavaThread*, >> JavaThread*, >> GrowableArray<_ >> jvmtiMonitorStackDepthInfo*>*)' >> /tmp/cc27HS5M.ltrans0.ltrans.o: In >> function >> `VM_GetFrameCount::doit() >> [clone .local.637]': >> cc27HS5M.ltrans0.o:(.text+0xcf8a): >> undefined >> reference to >> >> `JvmtiEnvBase::get_frame_count(JvmtiThreadState*, int*)' >> /tmp/cc27HS5M.ltrans0.ltrans.o: In >> function >> `VM_SetFramePop::doit() >> [clone >> .local.636]': >> cc27HS5M.ltrans0.o:(.text+0xcfea): >> undefined >> reference to >> `JvmtiThreadState::count_frames()' >> cc27HS5M.ltrans0.o:(.text+0xd030): >> undefined >> reference to >> >> `JvmtiEnvThreadState::set_frame_pop(int)' >> /tmp/cc27HS5M.ltrans0.ltrans.o: In >> function >> `VM_GetFrameLocation::doit() >> [clone .local.641]': >> ... (rest of output omitted) >> >> >> My configuration: >> >> --with-jdk-variant=normal \ >> --with-jvm-variants=minimal \ >> --with-debug-level=release \ >> --disable-warnings-as-errors \ >> --disable-hotspot-gtest \ >> --with-abi-profile=arm-vfp-sflt \ >> --openjdk-target=$(GNU_TARGET_NAME) \ >> --with-sys-root=$(STAGING_DIR) \ >> --with-tools-dir=$(HOST_DIR) \ >> >> --with-freetype-include=$(STAGING_DIR)/usr/include \ >> >> --with-freetype-lib=$(STAGING_DIR)/usr/lib \ >> --with-freetype=$(STAGING_DIR)/usr/ \ >> >> --with-alsa-include=$(STAGING_DIR)/usr/include \ >> >> --with-alsa-lib=$(STAGING_DIR)/usr/lib \ >> --with-alsa=$(STAGING_DIR)/usr/ \ >> --with-cups=$(STAGING_DIR)/usr/ \ >> --with-x=$(STAGING_DIR)/usr/include \ >> >> --with-extra-ldflags=--sysroot=$(STAGING_DIR) \ >> --enable-headless-only \ >> --disable-freetype-bundling \ >> --enable-unlimited-crypto \ >> >> --with-boot-jdk=/opt/java/jdk1.8.0_102 >> >> >> >> >> From ioi.lam at oracle.com Thu Mar 2 06:12:14 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 01 Mar 2017 22:12:14 -0800 Subject: Fw: throwing static exceptions sometimes VERY slow! In-Reply-To: References: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com> Message-ID: <58B7B7BE.5050405@oracle.com> Vitaly, Thanks for forwarding the email. Unfortunately, the HTML formatting has been stripped by the mailing list so it's hard to see who said what :-( > I made a mockup-test (see attachment) that mimicks kind of what my > generated code does and adds measurements before throw and after catch. > Mostly, in my experiments with this, the throw takes on average 20 ns, > but between a 1 and 3 per million iterations take longer than 1 ms, on > average around 7 ms. It can be observed that doing a garbage collection in > a different java process (namely my eclipse that started the test) greatly > influences the frequency of the > 1 ms throws, so maybe it is simply a load > issue. Using multiple threads in parallel also seems to heighten the Perhaps the slow down is caused by GC? One simple way to try is to run with different heap sizes, and use different type of collectors. If you observe different frequency of the outlier cases, then it's probably caused by GC pauses. Jacob, could you post the mockup test again? Thanks - Ioi On 3/1/17 4:04 PM, Vitaly Davidovich wrote: > +hotspot dev as it got dropped > > On Wed, Mar 1, 2017 at 7:03 PM Vitaly Davidovich wrote: > >> Instead of overriding fillInStacktrace, which is a synchronized method and >> you're sharing those instances between threads, can you construct the >> Continue and Break exceptions telling base class that you don't want >> writable stacktrace? >> >> The jitter you're seeing could be lock contention due to above, biased >> lock revocation, JIT deoptimizations (although in your Test I'm not sure if >> it'll deoptimize since the throws are not uncommon). >> >> You can add -XX:+PrintSafepointStatistics >> -XX:PrintSafepointStatisticsCount=1 to get reasons for safepoints printed >> to stdout. >> >> As mentioned, run your test with -XX:+PrintCompilation to see if JIT is >> doing something. >> >> On Wed, Mar 1, 2017 at 7:52 AM Wieland, Jacob >> wrote: >> >> >> >> >> sorry, forgot the attachment >> >> >> BR >> >> >> >> *Dr. Jacob Wieland* >> >> *Senior Software Engineer* >> >> main +49.30.7261919.34 >> >> mobile +49.173.6446443 >> >> >> jacob.wieland at spirent.com >> >> >> www.spirent.com >> >> >> >> >> Follow us on: >> >> >> Spirent Communications >> >> >> | >> | >> >> >> >> Michaelkirchstra?e 17/18 >> >> 10179 Berlin, Germany >> >> >> >> *+++++ N E W S F L A S H +++++* >> >> >> >> Spirent Communications and Testing Technologies join forces to become your >> test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> ------------------------------ >> *From:* Wieland, Jacob >> *Sent:* Wednesday, March 1, 2017 1:47:52 PM >> *To:* Vitaly Davidovich; David Holmes >> *Cc:* hotspot-dev at openjdk.java.net >> >> *Subject:* Re: throwing static exceptions sometimes VERY slow! >> >> Hi, >> >> >> here are the general system infos: >> >> >> uname -a: >> >> >> Linux ber-28753 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 >> (2016-01-17) x86_64 GNU/Linux >> >> >> lscpu: >> >> >> Architecture: x86_64 >> CPU op-mode(s): 32-bit, 64-bit >> Byte Order: Little Endian >> CPU(s): 4 >> On-line CPU(s) list: 0-3 >> Thread(s) per core: 1 >> Core(s) per socket: 4 >> Socket(s): 1 >> NUMA node(s): 1 >> Vendor ID: GenuineIntel >> CPU family: 6 >> Model: 58 >> Model name: Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz >> Stepping: 9 >> CPU MHz: 2094.558 >> CPU max MHz: 3500.0000 >> CPU min MHz: 1600.0000 >> BogoMIPS: 6186.30 >> Virtualization: VT-x >> L1d cache: 32K >> L1i cache: 32K >> L2 cache: 256K >> L3 cache: 6144K >> NUMA node0 CPU(s): 0-3 >> >> java -version: >> >> >> java version "1.8.0_102" >> Java(TM) SE Runtime Environment (build 1.8.0_102-b14) >> Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode) >> >> Rest of the answers inline. >> >> BR >> >> >> >> *Dr. Jacob Wieland* >> >> *Senior Software Engineer* >> >> main +49.30.7261919.34 >> >> mobile +49.173.6446443 >> >> >> jacob.wieland at spirent.com >> >> >> www.spirent.com >> >> >> >> >> Follow us on: >> >> >> Spirent Communications >> >> >> | >> | >> >> >> >> Michaelkirchstra?e 17/18 >> >> 10179 Berlin, Germany >> >> >> >> *+++++ N E W S F L A S H +++++* >> >> >> >> Spirent Communications and Testing Technologies join forces to become your >> test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> ------------------------------ >> *From:* Vitaly Davidovich >> *Sent:* Tuesday, February 28, 2017 1:46:09 PM >> *To:* David Holmes >> *Cc:* Wieland, Jacob; hotspot-dev at openjdk.java.net >> *Subject:* Re: throwing static exceptions sometimes VERY slow! >> >> >> >> On Tue, Feb 28, 2017 at 7:37 AM, David Holmes >> wrote: >> >> Hi, >> >> On 28/02/2017 9:02 PM, Wieland, Jacob wrote: >> >> Hi, >> >> I am observing a very strange behavior. >> >> In our generated code (due to various reasons I won't go into here unless >> I have to, but trust me, they are legit), we throw static exceptions for >> control flow purposes, This seems to work fine and without performance loss >> most of the time. However, we are observing now that every few seconds, a >> throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to the >> normal almost non-measurable time). >> >> Update: it seems that the observed behavior was kind of related with some >> extraneous circumstances (the computer seems to have been more lagged than >> usual at that point). By now, I can only observe throws that take up to 20 >> milliseconds (which opposed to the normal time still seems slow). >> >> It does not seem to be GC related, the only idea that I have is the jitter. >> >> >> What jitter? >> >> I'm assuming Jacob is referring to the JIT. But yes, he needs to provide >> more information. >> >> In particular, it would be good to know the following, as David mentioned, >> (off the top of my head): >> 1) JVM cmdline flags >> >> -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=y >> >> 2) Type of exception thrown (with a stacktrace or not) >> >> Exception class derived from java.lang.Error that overrides >> fillInStackTrace() with 'return this', so stackless. Same instances stored >> in static final fields are always thrown. >> >> 3) Is the call stack depth about the same during the slow and fast throws? >> >> It is always exactly the same stack, as it happens in a loop. >> >> 4) Is the exception thrown frequently or infrequently? >> >> In the test where this was observed, the exception was thrown every 40ms, >> so, I guess, frequently. >> >> 5) Is there -XX:+PrintCompilation output available around the time when >> the slowdown is observed >> >> Unfortunately not, and at the moment, I cannot reproduce the behavior. >> >> I made a mockup-test (see attachment) that mimicks kind of what my >> generated code does and adds measurements before throw and after catch. >> Mostly, in my experiments with this, the throw takes on average 20 ns, >> but between a 1 and 3 per million iterations take longer than 1 ms, on >> average around 7 ms. It can be observed that doing a garbage collection in >> a different java process (namely my eclipse that started the test) greatly >> influences the frequency of the > 1 ms throws, so maybe it is simply a load >> issue. Using multiple threads in parallel also seems to heighten the >> >> Also, when introducing an additional sleep between iterations (like in my >> original scenario where I was waiting for a timer to timeout before >> executing the construct that used the throw), I cannot reproduce such >> outliers at all. >> >> >> >> So, my question is. Is this a known (and for some strange reason maybe >> even accepted) behavior or is this a bug that I should file with Oracle (or >> you guys). >> >> >> You really are not giving us anything to go on here. How are you observing >> this slowness? Exactly how do you throw? What exactly are you measuring? >> What's the execution context, the machine, processors etc etc etc. >> >> >> I was observing the slowness in a testcase where I wanted regular >> intervals of 40 ms between sending a message via UDP but frequently (every >> few seconds) got a lag of around 1.5 seconds (and a few days later 0.5 >> seconds) before the timer which timed out at the correct time was noted. >> >> >> When eliminating one kind of such thrown control exception, the next kind >> of thrown control exception (from a surrounding construct) was having the >> same effect which actually led me to the slow-throw-hypothesis. Eliminating >> all exceptions in the small example confirmed this hypothesis, but going >> that route would take a lot of effort for all cases and introduce other >> additional imperformances (as I basically have to mock the exception >> throwing via cascading return and if-checks on every level) which I would >> like to avoid. >> >> Regards, >> David >> >> >> >> >> >> Spirent Communications e-mail confidentiality. >> ------------------------------------------------------------------------ >> This e-mail contains confidential and / or privileged information >> belonging to Spirent Communications plc, its affiliates and / or >> subsidiaries. If you are not the intended recipient, you are hereby >> notified that any disclosure, copying, distribution and / or the taking of >> any action based upon reliance on the contents of this transmission is >> strictly forbidden. If you have received this message in error please >> notify the sender by return e-mail and delete it from your system. >> >> Spirent Communications plc >> Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United >> Kingdom. >> Tel No. +44 (0) 1293 767676 >> Fax No. +44 (0) 1293 767677 >> >> Registered in England Number 470893 >> Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 >> 9XN, United Kingdom. >> >> Or if within the US, >> >> Spirent Communications, >> 27349 Agoura Road, Calabasas, CA, 91301, USA. >> Tel No. 1-818-676- 2300 >> >> -- >> Sent from my phone >> From erik.helin at oracle.com Thu Mar 2 08:31:42 2017 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 2 Mar 2017 09:31:42 +0100 Subject: RFR(S): 8176012 - Remove unused groups in hotspot/test/TEST.groups In-Reply-To: <0b1701d2920b$784f3c80$68edb580$@oracle.com> References: <0b1701d2920b$784f3c80$68edb580$@oracle.com> Message-ID: On 02/28/2017 10:41 PM, Christian Tornqvist wrote: > Hi everyone, Hey Christian, > Please review this small change for JDK 10 that removes unused Hotspot test > groups from TEST.groups. Tested locally by running a small set of jtreg > tests to verify consistency of TEST.groups. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176012 > > Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8176012/webrev.00/ looks good, Reviewed. Thanks for cleaning this up! Thanks, Erik > Thanks, > > Christian From vitalyd at gmail.com Thu Mar 2 11:35:20 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 02 Mar 2017 11:35:20 +0000 Subject: Fw: throwing static exceptions sometimes VERY slow! In-Reply-To: <58B7B7BE.5050405@oracle.com> References: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com> <58B7B7BE.5050405@oracle.com> Message-ID: Ioi, Argh, you're right about the formatting. I'll paste below my last reply to Jacob. Instead of overriding fillInStacktrace, which is a synchronized method and you're sharing those instances between threads, can you construct the Continue and Break exceptions telling base class that you don't want writable stacktrace? The jitter you're seeing could be lock contention due to above, biased lock revocation, JIT deoptimizations (although in your Test I'm not sure if it'll deoptimize since the throws are not uncommon). You can add -XX:+PrintSafepointStatistics -XX:PrintSafepointStatisticsCount=1 to get reasons for safepoints printed to stdout. As mentioned, run your test with -XX:+PrintCompilation to see if JIT is doing something. On Thu, Mar 2, 2017 at 1:12 AM Ioi Lam wrote: > Vitaly, > > Thanks for forwarding the email. Unfortunately, the HTML formatting has > been stripped by the mailing list so it's hard to see who said what :-( > > > I made a mockup-test (see attachment) that mimicks kind of what my > > generated code does and adds measurements before throw and after catch. > > Mostly, in my experiments with this, the throw takes on average 20 ns, > > but between a 1 and 3 per million iterations take longer than 1 ms, on > > average around 7 ms. It can be observed that doing a garbage collection > in > > a different java process (namely my eclipse that started the test) > greatly > > influences the frequency of the > 1 ms throws, so maybe it is simply a > load > > issue. Using multiple threads in parallel also seems to heighten the > > Perhaps the slow down is caused by GC? > > One simple way to try is to run with different heap sizes, and use > different type of collectors. If you observe different frequency of the > outlier cases, then it's probably caused by GC pauses. > > Jacob, could you post the mockup test again? > > Thanks > - Ioi > > On 3/1/17 4:04 PM, Vitaly Davidovich wrote: > > +hotspot dev as it got dropped > > > > On Wed, Mar 1, 2017 at 7:03 PM Vitaly Davidovich > wrote: > > > >> Instead of overriding fillInStacktrace, which is a synchronized method > and > >> you're sharing those instances between threads, can you construct the > >> Continue and Break exceptions telling base class that you don't want > >> writable stacktrace? > >> > >> The jitter you're seeing could be lock contention due to above, biased > >> lock revocation, JIT deoptimizations (although in your Test I'm not > sure if > >> it'll deoptimize since the throws are not uncommon). > >> > >> You can add -XX:+PrintSafepointStatistics > >> -XX:PrintSafepointStatisticsCount=1 to get reasons for safepoints > printed > >> to stdout. > >> > >> As mentioned, run your test with -XX:+PrintCompilation to see if JIT is > >> doing something. > >> > >> On Wed, Mar 1, 2017 at 7:52 AM Wieland, Jacob < > Jacob.Wieland at spirent.com> > >> wrote: > >> > >> > >> > >> > >> sorry, forgot the attachment > >> > >> > >> BR > >> > >> > >> > >> *Dr. Jacob Wieland* > >> > >> *Senior Software Engineer* > >> > >> main +49.30.7261919.34 > >> > >> mobile +49.173.6446443 > >> > >> > >> jacob.wieland at spirent.com > >> > >> > >> www.spirent.com > >> > >> > >> > >> > >> Follow us on: > >> > >> > >> Spirent Communications > >> > >> > >> | > >> | > >> > >> > >> > >> Michaelkirchstra?e 17/18 > >> > >> 10179 Berlin, Germany > >> > >> > >> > >> *+++++ N E W S F L A S H +++++* > >> > >> > >> > >> Spirent Communications and Testing Technologies join forces to become > your > >> test automation power-house. Read more at http://conta.cc/1S62BEY. > >> > >> ------------------------------ > >> *From:* Wieland, Jacob > >> *Sent:* Wednesday, March 1, 2017 1:47:52 PM > >> *To:* Vitaly Davidovich; David Holmes > >> *Cc:* hotspot-dev at openjdk.java.net > >> > >> *Subject:* Re: throwing static exceptions sometimes VERY slow! > >> > >> Hi, > >> > >> > >> here are the general system infos: > >> > >> > >> uname -a: > >> > >> > >> Linux ber-28753 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 > >> (2016-01-17) x86_64 GNU/Linux > >> > >> > >> lscpu: > >> > >> > >> Architecture: x86_64 > >> CPU op-mode(s): 32-bit, 64-bit > >> Byte Order: Little Endian > >> CPU(s): 4 > >> On-line CPU(s) list: 0-3 > >> Thread(s) per core: 1 > >> Core(s) per socket: 4 > >> Socket(s): 1 > >> NUMA node(s): 1 > >> Vendor ID: GenuineIntel > >> CPU family: 6 > >> Model: 58 > >> Model name: Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz > >> Stepping: 9 > >> CPU MHz: 2094.558 > >> CPU max MHz: 3500.0000 > >> CPU min MHz: 1600.0000 > >> BogoMIPS: 6186.30 > >> Virtualization: VT-x > >> L1d cache: 32K > >> L1i cache: 32K > >> L2 cache: 256K > >> L3 cache: 6144K > >> NUMA node0 CPU(s): 0-3 > >> > >> java -version: > >> > >> > >> java version "1.8.0_102" > >> Java(TM) SE Runtime Environment (build 1.8.0_102-b14) > >> Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode) > >> > >> Rest of the answers inline. > >> > >> BR > >> > >> > >> > >> *Dr. Jacob Wieland* > >> > >> *Senior Software Engineer* > >> > >> main +49.30.7261919.34 > >> > >> mobile +49.173.6446443 > >> > >> > >> jacob.wieland at spirent.com > >> > >> > >> www.spirent.com > >> > >> > >> > >> > >> Follow us on: > >> > >> > >> Spirent Communications > >> > >> > >> | > >> | > >> > >> > >> > >> Michaelkirchstra?e 17/18 > >> > >> 10179 Berlin, Germany > >> > >> > >> > >> *+++++ N E W S F L A S H +++++* > >> > >> > >> > >> Spirent Communications and Testing Technologies join forces to become > your > >> test automation power-house. Read more at http://conta.cc/1S62BEY. > >> > >> ------------------------------ > >> *From:* Vitaly Davidovich > >> *Sent:* Tuesday, February 28, 2017 1:46:09 PM > >> *To:* David Holmes > >> *Cc:* Wieland, Jacob; hotspot-dev at openjdk.java.net > >> *Subject:* Re: throwing static exceptions sometimes VERY slow! > >> > >> > >> > >> On Tue, Feb 28, 2017 at 7:37 AM, David Holmes > >> wrote: > >> > >> Hi, > >> > >> On 28/02/2017 9:02 PM, Wieland, Jacob wrote: > >> > >> Hi, > >> > >> I am observing a very strange behavior. > >> > >> In our generated code (due to various reasons I won't go into here > unless > >> I have to, but trust me, they are legit), we throw static exceptions for > >> control flow purposes, This seems to work fine and without performance > loss > >> most of the time. However, we are observing now that every few seconds, > a > >> throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to > the > >> normal almost non-measurable time). > >> > >> Update: it seems that the observed behavior was kind of related with > some > >> extraneous circumstances (the computer seems to have been more lagged > than > >> usual at that point). By now, I can only observe throws that take up to > 20 > >> milliseconds (which opposed to the normal time still seems slow). > >> > >> It does not seem to be GC related, the only idea that I have is the > jitter. > >> > >> > >> What jitter? > >> > >> I'm assuming Jacob is referring to the JIT. But yes, he needs to > provide > >> more information. > >> > >> In particular, it would be good to know the following, as David > mentioned, > >> (off the top of my head): > >> 1) JVM cmdline flags > >> > >> -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=y > >> > >> 2) Type of exception thrown (with a stacktrace or not) > >> > >> Exception class derived from java.lang.Error that overrides > >> fillInStackTrace() with 'return this', so stackless. Same instances > stored > >> in static final fields are always thrown. > >> > >> 3) Is the call stack depth about the same during the slow and fast > throws? > >> > >> It is always exactly the same stack, as it happens in a loop. > >> > >> 4) Is the exception thrown frequently or infrequently? > >> > >> In the test where this was observed, the exception was thrown every > 40ms, > >> so, I guess, frequently. > >> > >> 5) Is there -XX:+PrintCompilation output available around the time when > >> the slowdown is observed > >> > >> Unfortunately not, and at the moment, I cannot reproduce the behavior. > >> > >> I made a mockup-test (see attachment) that mimicks kind of what my > >> generated code does and adds measurements before throw and after catch. > >> Mostly, in my experiments with this, the throw takes on average 20 ns, > >> but between a 1 and 3 per million iterations take longer than 1 ms, on > >> average around 7 ms. It can be observed that doing a garbage collection > in > >> a different java process (namely my eclipse that started the test) > greatly > >> influences the frequency of the > 1 ms throws, so maybe it is simply a > load > >> issue. Using multiple threads in parallel also seems to heighten the > >> > >> Also, when introducing an additional sleep between iterations (like in > my > >> original scenario where I was waiting for a timer to timeout before > >> executing the construct that used the throw), I cannot reproduce such > >> outliers at all. > >> > >> > >> > >> So, my question is. Is this a known (and for some strange reason maybe > >> even accepted) behavior or is this a bug that I should file with Oracle > (or > >> you guys). > >> > >> > >> You really are not giving us anything to go on here. How are you > observing > >> this slowness? Exactly how do you throw? What exactly are you measuring? > >> What's the execution context, the machine, processors etc etc etc. > >> > >> > >> I was observing the slowness in a testcase where I wanted regular > >> intervals of 40 ms between sending a message via UDP but frequently > (every > >> few seconds) got a lag of around 1.5 seconds (and a few days later 0.5 > >> seconds) before the timer which timed out at the correct time was noted. > >> > >> > >> When eliminating one kind of such thrown control exception, the next > kind > >> of thrown control exception (from a surrounding construct) was having > the > >> same effect which actually led me to the slow-throw-hypothesis. > Eliminating > >> all exceptions in the small example confirmed this hypothesis, but going > >> that route would take a lot of effort for all cases and introduce other > >> additional imperformances (as I basically have to mock the exception > >> throwing via cascading return and if-checks on every level) which I > would > >> like to avoid. > >> > >> Regards, > >> David > >> > >> > >> > >> > >> > >> Spirent Communications e-mail confidentiality. > >> ------------------------------------------------------------------------ > >> This e-mail contains confidential and / or privileged information > >> belonging to Spirent Communications plc, its affiliates and / or > >> subsidiaries. If you are not the intended recipient, you are hereby > >> notified that any disclosure, copying, distribution and / or the taking > of > >> any action based upon reliance on the contents of this transmission is > >> strictly forbidden. If you have received this message in error please > >> notify the sender by return e-mail and delete it from your system. > >> > >> Spirent Communications plc > >> Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United > >> Kingdom. > >> Tel No. +44 (0) 1293 767676 > >> Fax No. +44 (0) 1293 767677 > >> > >> Registered in England Number 470893 > >> Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 > >> 9XN, United Kingdom. > >> > >> Or if within the US, > >> > >> Spirent Communications, > >> 27349 Agoura Road, Calabasas, CA, 91301, USA. > >> Tel No. 1-818-676- 2300 > >> > >> -- > >> Sent from my phone > >> > > -- Sent from my phone From Jacob.Wieland at spirent.com Thu Mar 2 13:54:39 2017 From: Jacob.Wieland at spirent.com (Wieland, Jacob) Date: Thu, 2 Mar 2017 13:54:39 +0000 Subject: Fw: throwing static exceptions sometimes VERY slow! In-Reply-To: References: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com> <58B7B7BE.5050405@oracle.com> , Message-ID: 0: loops: 1000000000 66 1 java.lang.Object:: (1 bytes) 68 2 java.lang.Number:: (5 bytes) 69 3 n java.lang.System::nanoTime (native) (static) 84 4 java.lang.String::indexOf (70 bytes) 86 5 java.lang.String::hashCode (55 bytes) 0: Thu Mar 02 14:50:41 CET 2017: 178532 88 6 java.util.concurrent.atomic.AtomicLong::get (5 bytes) 88 8 n sun.misc.Unsafe::compareAndSwapLong (native) 88 7 java.util.concurrent.atomic.AtomicLong::compareAndSet (13 bytes) 88 9 java.util.Random::next (47 bytes) 88 10 java.math.BigInteger::add (178 bytes) 97 11 java.math.BigInteger::valueOf (62 bytes) 97 12 java.math.BigInteger:: (77 bytes) 98 13 java.math.BigInteger:: (24 bytes) 98 14 java.math.BigInteger::add (123 bytes) 98 15 java.lang.Math::random (17 bytes) 99 16 java.util.Random::nextDouble (24 bytes) 101 17 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ 70 (549 bytes) 123 17 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ -2 (549 bytes) made not entrant 124 18 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ 70 (549 bytes) 8802 18 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ -2 (549 bytes) made not entrant 0: Thu Mar 02 14:50:50 CET 2017: 620776 8803 19 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ 70 (549 bytes) 0: Thu Mar 02 14:50:56 CET 2017: 128566 21909 19 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ -2 (549 bytes) made not entrant 21909 14 java.math.BigInteger::add (123 bytes) made not entrant 21909 10 java.math.BigInteger::add (178 bytes) made not entrant 21911 20 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ 70 (549 bytes) 21914 21 java.math.BigInteger::add (178 bytes) 21921 22 java.math.BigInteger::add (123 bytes) 0: Thu Mar 02 14:51:04 CET 2017: 2974443 0: Thu Mar 02 14:51:05 CET 2017: 103290 44839 20 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ -2 (549 bytes) made not entrant 44840 23 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ 70 (549 bytes) 0: Thu Mar 02 14:51:35 CET 2017: 2076623 0: Thu Mar 02 14:51:36 CET 2017: 333400 0: Thu Mar 02 14:51:37 CET 2017: 240985 0: Thu Mar 02 14:51:39 CET 2017: 1360262 0: Thu Mar 02 14:51:40 CET 2017: 2993867 0: Thu Mar 02 14:51:48 CET 2017: 794752 67360 22 java.math.BigInteger::add (123 bytes) made not entrant 67362 24 java.math.BigInteger::add (123 bytes) 0: Thu Mar 02 14:52:03 CET 2017: 146861 0: Thu Mar 02 14:52:14 CET 2017: 312026 0: Thu Mar 02 14:52:23 CET 2017: 353730 104327 23 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure @ -2 (549 bytes) made not entrant 104327 11 java.math.BigInteger::valueOf (62 bytes) made not entrant 0: longThrows: 14 average time: 901293 0: shortThrows: 999999986 average time: 19 0: sum: 19999665505 0: average throw time: 19 That was run using nice -19 to minimize interaction with other processes. BR [X] Dr. Jacob Wieland Senior Software Engineer main +49.30.7261919.34 mobile +49.173.6446443 jacob.wieland at spirent.com www.spirent.com Follow us on: Spirent Communications [X]|[X]|[X] Michaelkirchstra?e 17/18 10179 Berlin, Germany +++++ N E W S F L A S H +++++ Spirent Communications and Testing Technologies join forces to become your test automation power-house. Read more at http://conta.cc/1S62BEY. ________________________________ From: Vitaly Davidovich Sent: Thursday, March 2, 2017 2:45:15 PM To: Wieland, Jacob Cc: Ioi Lam; hotspot-dev at openjdk.java.net Subject: Re: Fw: throwing static exceptions sometimes VERY slow! That's the safepoint stats, which is good to see - looks like a lot of GC activity. Is that also the case for your real application? GC activity like that could cause jitter but you mentioned in your original email that you ruled it out somehow? But, I was asking to see the PrintCompilation output in my last email :). There's no Deoptimize safepoint entry in the safepoint log here, but would still be interesting to see JITs compilation log. On Thu, Mar 2, 2017 at 8:36 AM Wieland, Jacob > wrote: here you are BR [X] Dr. Jacob Wieland Senior Software Engineer main +49.30.7261919.34 mobile +49.173.6446443 jacob.wieland at spirent.com www.spirent.com Follow us on: Spirent Communications [X]| [X] | [X] Michaelkirchstra?e 17/18 10179 Berlin, Germany +++++ N E W S F L A S H +++++ Spirent Communications and Testing Technologies join forces to become your test automation power-house. Read more at http://conta.cc/1S62BEY. ________________________________ From: Vitaly Davidovich > Sent: Thursday, March 2, 2017 1:45:43 PM To: Wieland, Jacob Cc: Ioi Lam; hotspot-dev at openjdk.java.net Subject: Re: Fw: throwing static exceptions sometimes VERY slow! So how about PrintCompilation output? On Thu, Mar 2, 2017 at 7:31 AM, Wieland, Jacob > wrote: I have modified the exception to not writable and disallow supression, but as I would have expected, no change in behavior was observed. Also, if only one thread is running and lock contention was an issue, should that not go away in that case (it doesn't). The other question if have regarding this is: if the exception is static, should fillInStackTrace() not be called only when creating that way before it is actually thrown? Why would that have any influence on the later throw? If it was just garbage collection, I would assume the GC output (if turned on) happening close to the long-throw output (it doesn't, as far as I can see). The most interesting observation about this (imho) is that the number of cases seems to be very reproducible, even over different threads, so there seems to be some non-random relationship (which of course could still be related to the underlying platform doing stuff at regular intervals or sth). Maybe I should measure also the distance between the long throws and see whether they are happening regularly or in clumps. My next experiment will be if the same behavior can be observed with a normal fast operation, but I doubt that as well, because then eliminating the throws from the original example where I observed this first should not have had any positive effect. Of course, another candidate could also be the System.nanoTime() itself which for some reason might contend on the system level for resources sometimes, but I have no idea how to measure that! [??] BR [X] Dr. Jacob Wieland Senior Software Engineer main +49.30.7261919.34 mobile +49.173.6446443 jacob.wieland at spirent.com www.spirent.com Follow us on: Spirent Communications [X]| [X] | [X] Michaelkirchstra?e 17/18 10179 Berlin, Germany +++++ N E W S F L A S H +++++ Spirent Communications and Testing Technologies join forces to become your test automation power-house. Read more at http://conta.cc/1S62BEY. ________________________________ From: Vitaly Davidovich > Sent: Thursday, March 2, 2017 12:35:20 PM To: Ioi Lam; Wieland, Jacob; hotspot-dev at openjdk.java.net Subject: Re: Fw: throwing static exceptions sometimes VERY slow! Ioi, Argh, you're right about the formatting. I'll paste below my last reply to Jacob. Instead of overriding fillInStacktrace, which is a synchronized method and you're sharing those instances between threads, can you construct the Continue and Break exceptions telling base class that you don't want writable stacktrace? The jitter you're seeing could be lock contention due to above, biased lock revocation, JIT deoptimizations (although in your Test I'm not sure if it'll deoptimize since the throws are not uncommon). You can add -XX:+PrintSafepointStatistics -XX:PrintSafepointStatisticsCount=1 to get reasons for safepoints printed to stdout. As mentioned, run your test with -XX:+PrintCompilation to see if JIT is doing something. On Thu, Mar 2, 2017 at 1:12 AM Ioi Lam > wrote: Vitaly, Thanks for forwarding the email. Unfortunately, the HTML formatting has been stripped by the mailing list so it's hard to see who said what :-( > I made a mockup-test (see attachment) that mimicks kind of what my > generated code does and adds measurements before throw and after catch. > Mostly, in my experiments with this, the throw takes on average 20 ns, > but between a 1 and 3 per million iterations take longer than 1 ms, on > average around 7 ms. It can be observed that doing a garbage collection in > a different java process (namely my eclipse that started the test) greatly > influences the frequency of the > 1 ms throws, so maybe it is simply a load > issue. Using multiple threads in parallel also seems to heighten the Perhaps the slow down is caused by GC? One simple way to try is to run with different heap sizes, and use different type of collectors. If you observe different frequency of the outlier cases, then it's probably caused by GC pauses. Jacob, could you post the mockup test again? Thanks - Ioi On 3/1/17 4:04 PM, Vitaly Davidovich wrote: > +hotspot dev as it got dropped > > On Wed, Mar 1, 2017 at 7:03 PM Vitaly Davidovich > wrote: > >> Instead of overriding fillInStacktrace, which is a synchronized method and >> you're sharing those instances between threads, can you construct the >> Continue and Break exceptions telling base class that you don't want >> writable stacktrace? >> >> The jitter you're seeing could be lock contention due to above, biased >> lock revocation, JIT deoptimizations (although in your Test I'm not sure if >> it'll deoptimize since the throws are not uncommon). >> >> You can add -XX:+PrintSafepointStatistics >> -XX:PrintSafepointStatisticsCount=1 to get reasons for safepoints printed >> to stdout. >> >> As mentioned, run your test with -XX:+PrintCompilation to see if JIT is >> doing something. >> >> On Wed, Mar 1, 2017 at 7:52 AM Wieland, Jacob > >> wrote: >> >> >> >> >> sorry, forgot the attachment >> >> >> BR >> >> >> >> *Dr. Jacob Wieland* >> >> *Senior Software Engineer* >> >> main +49.30.7261919.34 >> >> mobile +49.173.6446443 >> >> >> jacob.wieland at spirent.com >> >> >> www.spirent.com >> >> >> >> >> Follow us on: >> >> >> Spirent Communications >> >> >> | >> | >> >> >> >> Michaelkirchstra?e 17/18 >> >> 10179 Berlin, Germany >> >> >> >> *+++++ N E W S F L A S H +++++* >> >> >> >> Spirent Communications and Testing Technologies join forces to become your >> test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> ------------------------------ >> *From:* Wieland, Jacob >> *Sent:* Wednesday, March 1, 2017 1:47:52 PM >> *To:* Vitaly Davidovich; David Holmes >> *Cc:* hotspot-dev at openjdk.java.net >> >> *Subject:* Re: throwing static exceptions sometimes VERY slow! >> >> Hi, >> >> >> here are the general system infos: >> >> >> uname -a: >> >> >> Linux ber-28753 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 >> (2016-01-17) x86_64 GNU/Linux >> >> >> lscpu: >> >> >> Architecture: x86_64 >> CPU op-mode(s): 32-bit, 64-bit >> Byte Order: Little Endian >> CPU(s): 4 >> On-line CPU(s) list: 0-3 >> Thread(s) per core: 1 >> Core(s) per socket: 4 >> Socket(s): 1 >> NUMA node(s): 1 >> Vendor ID: GenuineIntel >> CPU family: 6 >> Model: 58 >> Model name: Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz >> Stepping: 9 >> CPU MHz: 2094.558 >> CPU max MHz: 3500.0000 >> CPU min MHz: 1600.0000 >> BogoMIPS: 6186.30 >> Virtualization: VT-x >> L1d cache: 32K >> L1i cache: 32K >> L2 cache: 256K >> L3 cache: 6144K >> NUMA node0 CPU(s): 0-3 >> >> java -version: >> >> >> java version "1.8.0_102" >> Java(TM) SE Runtime Environment (build 1.8.0_102-b14) >> Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode) >> >> Rest of the answers inline. >> >> BR >> >> >> >> *Dr. Jacob Wieland* >> >> *Senior Software Engineer* >> >> main +49.30.7261919.34 >> >> mobile +49.173.6446443 >> >> >> jacob.wieland at spirent.com >> >> >> www.spirent.com >> >> >> >> >> Follow us on: >> >> >> Spirent Communications >> >> >> | >> | >> >> >> >> Michaelkirchstra?e 17/18 >> >> 10179 Berlin, Germany >> >> >> >> *+++++ N E W S F L A S H +++++* >> >> >> >> Spirent Communications and Testing Technologies join forces to become your >> test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> ------------------------------ >> *From:* Vitaly Davidovich > >> *Sent:* Tuesday, February 28, 2017 1:46:09 PM >> *To:* David Holmes >> *Cc:* Wieland, Jacob; hotspot-dev at openjdk.java.net >> *Subject:* Re: throwing static exceptions sometimes VERY slow! >> >> >> >> On Tue, Feb 28, 2017 at 7:37 AM, David Holmes > >> wrote: >> >> Hi, >> >> On 28/02/2017 9:02 PM, Wieland, Jacob wrote: >> >> Hi, >> >> I am observing a very strange behavior. >> >> In our generated code (due to various reasons I won't go into here unless >> I have to, but trust me, they are legit), we throw static exceptions for >> control flow purposes, This seems to work fine and without performance loss >> most of the time. However, we are observing now that every few seconds, a >> throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to the >> normal almost non-measurable time). >> >> Update: it seems that the observed behavior was kind of related with some >> extraneous circumstances (the computer seems to have been more lagged than >> usual at that point). By now, I can only observe throws that take up to 20 >> milliseconds (which opposed to the normal time still seems slow). >> >> It does not seem to be GC related, the only idea that I have is the jitter. >> >> >> What jitter? >> >> I'm assuming Jacob is referring to the JIT. But yes, he needs to provide >> more information. >> >> In particular, it would be good to know the following, as David mentioned, >> (off the top of my head): >> 1) JVM cmdline flags >> >> -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=y >> >> 2) Type of exception thrown (with a stacktrace or not) >> >> Exception class derived from java.lang.Error that overrides >> fillInStackTrace() with 'return this', so stackless. Same instances stored >> in static final fields are always thrown. >> >> 3) Is the call stack depth about the same during the slow and fast throws? >> >> It is always exactly the same stack, as it happens in a loop. >> >> 4) Is the exception thrown frequently or infrequently? >> >> In the test where this was observed, the exception was thrown every 40ms, >> so, I guess, frequently. >> >> 5) Is there -XX:+PrintCompilation output available around the time when >> the slowdown is observed >> >> Unfortunately not, and at the moment, I cannot reproduce the behavior. >> >> I made a mockup-test (see attachment) that mimicks kind of what my >> generated code does and adds measurements before throw and after catch. >> Mostly, in my experiments with this, the throw takes on average 20 ns, >> but between a 1 and 3 per million iterations take longer than 1 ms, on >> average around 7 ms. It can be observed that doing a garbage collection in >> a different java process (namely my eclipse that started the test) greatly >> influences the frequency of the > 1 ms throws, so maybe it is simply a load >> issue. Using multiple threads in parallel also seems to heighten the >> >> Also, when introducing an additional sleep between iterations (like in my >> original scenario where I was waiting for a timer to timeout before >> executing the construct that used the throw), I cannot reproduce such >> outliers at all. >> >> >> >> So, my question is. Is this a known (and for some strange reason maybe >> even accepted) behavior or is this a bug that I should file with Oracle (or >> you guys). >> >> >> You really are not giving us anything to go on here. How are you observing >> this slowness? Exactly how do you throw? What exactly are you measuring? >> What's the execution context, the machine, processors etc etc etc. >> >> >> I was observing the slowness in a testcase where I wanted regular >> intervals of 40 ms between sending a message via UDP but frequently (every >> few seconds) got a lag of around 1.5 seconds (and a few days later 0.5 >> seconds) before the timer which timed out at the correct time was noted. >> >> >> When eliminating one kind of such thrown control exception, the next kind >> of thrown control exception (from a surrounding construct) was having the >> same effect which actually led me to the slow-throw-hypothesis. Eliminating >> all exceptions in the small example confirmed this hypothesis, but going >> that route would take a lot of effort for all cases and introduce other >> additional imperformances (as I basically have to mock the exception >> throwing via cascading return and if-checks on every level) which I would >> like to avoid. >> >> Regards, >> David >> >> >> >> >> >> Spirent Communications e-mail confidentiality. >> ------------------------------------------------------------------------ >> This e-mail contains confidential and / or privileged information >> belonging to Spirent Communications plc, its affiliates and / or >> subsidiaries. If you are not the intended recipient, you are hereby >> notified that any disclosure, copying, distribution and / or the taking of >> any action based upon reliance on the contents of this transmission is >> strictly forbidden. If you have received this message in error please >> notify the sender by return e-mail and delete it from your system. >> >> Spirent Communications plc >> Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United >> Kingdom. >> Tel No. +44 (0) 1293 767676 >> Fax No. +44 (0) 1293 767677 >> >> Registered in England Number 470893 >> Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 >> 9XN, United Kingdom. >> >> Or if within the US, >> >> Spirent Communications, >> 27349 Agoura Road, Calabasas, CA, 91301, USA. >> Tel No. 1-818-676- 2300 >> >> -- >> Sent from my phone >> -- Sent from my phone -- Sent from my phone From vitalyd at gmail.com Thu Mar 2 14:22:12 2017 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 2 Mar 2017 09:22:12 -0500 Subject: Fw: throwing static exceptions sometimes VERY slow! In-Reply-To: References: <15d597f0-d6ba-6fd6-6e28-49370353c29c@oracle.com> <58B7B7BE.5050405@oracle.com> Message-ID: Thanks. I now realize the compilation output was in your previous attachment as well, but I was looking at it on my phone and all I could see was the storm of allocation failures :). Can you try running your benchmark without using BigInteger (plain long)? It's creating GC/alloc noise, which would be good to remove from the equation. On Thu, Mar 2, 2017 at 8:54 AM, Wieland, Jacob wrote: > 0: loops: 1000000000 > 66 1 java.lang.Object:: (1 bytes) > 68 2 java.lang.Number:: (5 bytes) > 69 3 n java.lang.System::nanoTime (native) (static) > 84 4 java.lang.String::indexOf (70 bytes) > 86 5 java.lang.String::hashCode (55 bytes) > 0: Thu Mar 02 14:50:41 CET 2017: 178532 > 88 6 java.util.concurrent.atomic.AtomicLong::get (5 > bytes) > 88 8 n sun.misc.Unsafe::compareAndSwapLong (native) > 88 7 java.util.concurrent.atomic.AtomicLong::compareAndSet > (13 bytes) > 88 9 java.util.Random::next (47 bytes) > 88 10 java.math.BigInteger::add (178 bytes) > 97 11 java.math.BigInteger::valueOf (62 bytes) > 97 12 java.math.BigInteger:: (77 bytes) > 98 13 java.math.BigInteger:: (24 bytes) > 98 14 java.math.BigInteger::add (123 bytes) > 98 15 java.lang.Math::random (17 bytes) > 99 16 java.util.Random::nextDouble (24 bytes) > 101 17 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ 70 (549 bytes) > 123 17 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ -2 (549 bytes) made not entrant > 124 18 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ 70 (549 bytes) > 8802 18 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ -2 (549 bytes) made not entrant > 0: Thu Mar 02 14:50:50 CET 2017: 620776 > 8803 19 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ 70 (549 bytes) > 0: Thu Mar 02 14:50:56 CET 2017: 128566 > 21909 19 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ -2 (549 bytes) made not entrant > 21909 14 java.math.BigInteger::add (123 bytes) made not > entrant > 21909 10 java.math.BigInteger::add (178 bytes) made not > entrant > 21911 20 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ 70 (549 bytes) > 21914 21 java.math.BigInteger::add (178 bytes) > 21921 22 java.math.BigInteger::add (123 bytes) > 0: Thu Mar 02 14:51:04 CET 2017: 2974443 > 0: Thu Mar 02 14:51:05 CET 2017: 103290 > 44839 20 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ -2 (549 bytes) made not entrant > 44840 23 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ 70 (549 bytes) > 0: Thu Mar 02 14:51:35 CET 2017: 2076623 > 0: Thu Mar 02 14:51:36 CET 2017: 333400 > 0: Thu Mar 02 14:51:37 CET 2017: 240985 > 0: Thu Mar 02 14:51:39 CET 2017: 1360262 > 0: Thu Mar 02 14:51:40 CET 2017: 2993867 > 0: Thu Mar 02 14:51:48 CET 2017: 794752 > 67360 22 java.math.BigInteger::add (123 bytes) made not > entrant > 67362 24 java.math.BigInteger::add (123 bytes) > 0: Thu Mar 02 14:52:03 CET 2017: 146861 > 0: Thu Mar 02 14:52:14 CET 2017: 312026 > 0: Thu Mar 02 14:52:23 CET 2017: 353730 > 104327 23 % ! de.tu_berlin.cs.uebb.muttcn.runtime.Test::measure > @ -2 (549 bytes) made not entrant > 104327 11 java.math.BigInteger::valueOf (62 bytes) made > not entrant > 0: longThrows: 14 average time: 901293 > 0: shortThrows: 999999986 average time: 19 > 0: sum: 19999665505 > 0: average throw time: 19 > > That was run using nice -19 to minimize interaction with other processes. > > > BR > > > > *Dr. Jacob Wieland* > > *Senior Software Engineer* > > main +49.30.7261919.34 > > mobile +49.173.6446443 > > > jacob.wieland at spirent.com > > > www.spirent.com > > > > > Follow us on: > > > Spirent Communications > > > | > | > > > > Michaelkirchstra?e 17/18 > > 10179 Berlin, Germany > > > > *+++++ N E W S F L A S H +++++* > > > > Spirent Communications and Testing Technologies join forces to become your > test automation power-house. Read more at http://conta.cc/1S62BEY. > > ------------------------------ > *From:* Vitaly Davidovich > *Sent:* Thursday, March 2, 2017 2:45:15 PM > > *To:* Wieland, Jacob > *Cc:* Ioi Lam; hotspot-dev at openjdk.java.net > *Subject:* Re: Fw: throwing static exceptions sometimes VERY slow! > > That's the safepoint stats, which is good to see - looks like a lot of GC > activity. Is that also the case for your real application? GC activity > like that could cause jitter but you mentioned in your original email that > you ruled it out somehow? > > But, I was asking to see the PrintCompilation output in my last email :). > There's no Deoptimize safepoint entry in the safepoint log here, but would > still be interesting to see JITs compilation log. > > On Thu, Mar 2, 2017 at 8:36 AM Wieland, Jacob > wrote: > >> >> here you are >> >> >> BR >> >> >> >> *Dr. Jacob Wieland* >> >> *Senior Software Engineer* >> >> main +49.30.7261919.34 >> >> mobile +49.173.6446443 >> >> >> jacob.wieland at spirent.com >> >> >> www.spirent.com >> >> >> >> >> Follow us on: >> >> >> Spirent Communications >> >> >> | >> | >> >> >> >> Michaelkirchstra?e 17/18 >> >> 10179 Berlin, Germany >> >> >> >> *+++++ N E W S F L A S H +++++* >> >> >> >> Spirent Communications and Testing Technologies join forces to become >> your test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> ------------------------------ >> *From:* Vitaly Davidovich >> *Sent:* Thursday, March 2, 2017 1:45:43 PM >> *To:* Wieland, Jacob >> *Cc:* Ioi Lam; hotspot-dev at openjdk.java.net >> >> *Subject:* Re: Fw: throwing static exceptions sometimes VERY slow! >> So how about PrintCompilation output? >> >> On Thu, Mar 2, 2017 at 7:31 AM, Wieland, Jacob > > wrote: >> >> I have modified the exception to not writable and disallow supression, >> but as I would have expected, no change in behavior was observed. Also, if >> only one thread is running and lock contention was an issue, should that >> not go away in that case (it doesn't). The other question if have regarding >> this is: if the exception is static, should fillInStackTrace() not be >> called only when creating that way before it is actually thrown? Why would >> that have any influence on the later throw? >> >> >> If it was just garbage collection, I would assume the GC output (if >> turned on) happening close to the long-throw output (it doesn't, as far as >> I can see). >> >> >> The most interesting observation about this (imho) is that the number of >> cases seems to be very reproducible, even over different threads, so there >> seems to be some non-random relationship (which of course could still be >> related to the underlying platform doing stuff at regular intervals or >> sth). Maybe I should measure also the distance between the long throws and >> see whether they are happening regularly or in clumps. >> >> >> My next experiment will be if the same behavior can be observed with a >> normal fast operation, but I doubt that as well, because then eliminating >> the throws from the original example where I observed this first should not >> have had any positive effect. >> >> >> Of course, another candidate could also be the System.nanoTime() itself >> which for some reason might contend on the system level for resources >> sometimes, but I have no idea how to measure that! [image: ??] >> >> >> BR >> >> >> >> *Dr. Jacob Wieland* >> >> *Senior Software Engineer* >> >> main +49.30.7261919.34 >> >> mobile +49.173.6446443 >> >> >> jacob.wieland at spirent.com >> >> >> www.spirent.com >> >> >> >> >> Follow us on: >> >> >> Spirent Communications >> >> >> | >> | >> >> >> >> Michaelkirchstra?e 17/18 >> >> 10179 Berlin, Germany >> >> >> >> *+++++ N E W S F L A S H +++++* >> >> >> >> Spirent Communications and Testing Technologies join forces to become >> your test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> ------------------------------ >> *From:* Vitaly Davidovich >> *Sent:* Thursday, March 2, 2017 12:35:20 PM >> *To:* Ioi Lam; Wieland, Jacob; hotspot-dev at openjdk.java.net >> *Subject:* Re: Fw: throwing static exceptions sometimes VERY slow! >> >> Ioi, >> >> Argh, you're right about the formatting. I'll paste below my last reply >> to Jacob. >> >> Instead of overriding fillInStacktrace, which is a synchronized method >> and you're sharing those instances between threads, can you construct the >> Continue and Break exceptions telling base class that you don't want >> writable stacktrace? >> >> The jitter you're seeing could be lock contention due to above, biased >> lock revocation, JIT deoptimizations (although in your Test I'm not sure if >> it'll deoptimize since the throws are not uncommon). >> >> You can add -XX:+PrintSafepointStatistics -XX: >> PrintSafepointStatisticsCount=1 to get reasons for safepoints printed >> to stdout. >> >> As mentioned, run your test with -XX:+PrintCompilation to see if JIT is >> doing something. >> >> On Thu, Mar 2, 2017 at 1:12 AM Ioi Lam wrote: >> >> Vitaly, >> >> Thanks for forwarding the email. Unfortunately, the HTML formatting has >> been stripped by the mailing list so it's hard to see who said what :-( >> >> > I made a mockup-test (see attachment) that mimicks kind of what my >> > generated code does and adds measurements before throw and after catch. >> > Mostly, in my experiments with this, the throw takes on average 20 ns, >> > but between a 1 and 3 per million iterations take longer than 1 ms, on >> > average around 7 ms. It can be observed that doing a garbage collection >> in >> > a different java process (namely my eclipse that started the test) >> greatly >> > influences the frequency of the > 1 ms throws, so maybe it is simply a >> load >> > issue. Using multiple threads in parallel also seems to heighten the >> >> Perhaps the slow down is caused by GC? >> >> One simple way to try is to run with different heap sizes, and use >> different type of collectors. If you observe different frequency of the >> outlier cases, then it's probably caused by GC pauses. >> >> Jacob, could you post the mockup test again? >> >> Thanks >> - Ioi >> >> On 3/1/17 4:04 PM, Vitaly Davidovich wrote: >> > +hotspot dev as it got dropped >> > >> > On Wed, Mar 1, 2017 at 7:03 PM Vitaly Davidovich >> wrote: >> > >> >> Instead of overriding fillInStacktrace, which is a synchronized method >> and >> >> you're sharing those instances between threads, can you construct the >> >> Continue and Break exceptions telling base class that you don't want >> >> writable stacktrace? >> >> >> >> The jitter you're seeing could be lock contention due to above, biased >> >> lock revocation, JIT deoptimizations (although in your Test I'm not >> sure if >> >> it'll deoptimize since the throws are not uncommon). >> >> >> >> You can add -XX:+PrintSafepointStatistics >> >> -XX:PrintSafepointStatisticsCount=1 to get reasons for safepoints >> printed >> >> to stdout. >> >> >> >> As mentioned, run your test with -XX:+PrintCompilation to see if JIT is >> >> doing something. >> >> >> >> On Wed, Mar 1, 2017 at 7:52 AM Wieland, Jacob < >> Jacob.Wieland at spirent.com> >> >> wrote: >> >> >> >> >> >> >> >> >> >> sorry, forgot the attachment >> >> >> >> >> >> BR >> >> >> >> >> >> >> >> *Dr. Jacob Wieland* >> >> >> >> *Senior Software Engineer* >> >> >> >> main +49.30.7261919.34 <+49%2030%20726191934> >> >> >> >> mobile +49.173.6446443 <+49%20173%206446443> >> >> >> >> >> >> jacob.wieland at spirent.com >> >> >> >> >> >> www.spirent.com >> >> >> >> >> >> >> >> >> >> Follow us on: >> >> >> >> >> >> Spirent Communications >> >> >> >> >> >> | >> >> | >> >> >> >> >> >> >> >> Michaelkirchstra?e 17/18 >> >> >> >> 10179 Berlin, Germany >> >> >> >> >> >> >> >> *+++++ N E W S F L A S H +++++* >> >> >> >> >> >> >> >> Spirent Communications and Testing Technologies join forces to become >> your >> >> test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> >> >> ------------------------------ >> >> *From:* Wieland, Jacob >> >> *Sent:* Wednesday, March 1, 2017 1:47:52 PM >> >> *To:* Vitaly Davidovich; David Holmes >> >> *Cc:* hotspot-dev at openjdk.java.net >> >> >> >> *Subject:* Re: throwing static exceptions sometimes VERY slow! >> >> >> >> Hi, >> >> >> >> >> >> here are the general system infos: >> >> >> >> >> >> uname -a: >> >> >> >> >> >> Linux ber-28753 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 >> >> (2016-01-17) x86_64 GNU/Linux >> >> >> >> >> >> lscpu: >> >> >> >> >> >> Architecture: x86_64 >> >> CPU op-mode(s): 32-bit, 64-bit >> >> Byte Order: Little Endian >> >> CPU(s): 4 >> >> On-line CPU(s) list: 0-3 >> >> Thread(s) per core: 1 >> >> Core(s) per socket: 4 >> >> Socket(s): 1 >> >> NUMA node(s): 1 >> >> Vendor ID: GenuineIntel >> >> CPU family: 6 >> >> Model: 58 >> >> Model name: Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz >> >> Stepping: 9 >> >> CPU MHz: 2094.558 >> >> CPU max MHz: 3500.0000 >> >> CPU min MHz: 1600.0000 >> >> BogoMIPS: 6186.30 >> >> Virtualization: VT-x >> >> L1d cache: 32K >> >> L1i cache: 32K >> >> L2 cache: 256K >> >> L3 cache: 6144K >> >> NUMA node0 CPU(s): 0-3 >> >> >> >> java -version: >> >> >> >> >> >> java version "1.8.0_102" >> >> Java(TM) SE Runtime Environment (build 1.8.0_102-b14) >> >> Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode) >> >> >> >> Rest of the answers inline. >> >> >> >> BR >> >> >> >> >> >> >> >> *Dr. Jacob Wieland* >> >> >> >> *Senior Software Engineer* >> >> >> >> main +49.30.7261919.34 <+49%2030%20726191934> >> >> >> >> mobile +49.173.6446443 <+49%20173%206446443> >> >> >> >> >> >> jacob.wieland at spirent.com >> >> >> >> >> >> www.spirent.com >> >> >> >> >> >> >> >> >> >> Follow us on: >> >> >> >> >> >> Spirent Communications >> >> >> >> >> >> | >> >> | >> >> >> >> >> >> >> >> Michaelkirchstra?e 17/18 >> >> >> >> 10179 Berlin, Germany >> >> >> >> >> >> >> >> *+++++ N E W S F L A S H +++++* >> >> >> >> >> >> >> >> Spirent Communications and Testing Technologies join forces to become >> your >> >> test automation power-house. Read more at http://conta.cc/1S62BEY. >> >> >> >> ------------------------------ >> >> *From:* Vitaly Davidovich >> >> *Sent:* Tuesday, February 28, 2017 1:46:09 PM >> >> *To:* David Holmes >> >> *Cc:* Wieland, Jacob; hotspot-dev at openjdk.java.net >> >> *Subject:* Re: throwing static exceptions sometimes VERY slow! >> >> >> >> >> >> >> >> On Tue, Feb 28, 2017 at 7:37 AM, David Holmes > > >> >> wrote: >> >> >> >> Hi, >> >> >> >> On 28/02/2017 9:02 PM, Wieland, Jacob wrote: >> >> >> >> Hi, >> >> >> >> I am observing a very strange behavior. >> >> >> >> In our generated code (due to various reasons I won't go into here >> unless >> >> I have to, but trust me, they are legit), we throw static exceptions >> for >> >> control flow purposes, This seems to work fine and without performance >> loss >> >> most of the time. However, we are observing now that every few >> seconds, a >> >> throw sometimes takes up between 1,5 and 2.5 seconds! (in contrast to >> the >> >> normal almost non-measurable time). >> >> >> >> Update: it seems that the observed behavior was kind of related with >> some >> >> extraneous circumstances (the computer seems to have been more lagged >> than >> >> usual at that point). By now, I can only observe throws that take up >> to 20 >> >> milliseconds (which opposed to the normal time still seems slow). >> >> >> >> It does not seem to be GC related, the only idea that I have is the >> jitter. >> >> >> >> >> >> What jitter? >> >> >> >> I'm assuming Jacob is referring to the JIT. But yes, he needs to >> provide >> >> more information. >> >> >> >> In particular, it would be good to know the following, as David >> mentioned, >> >> (off the top of my head): >> >> 1) JVM cmdline flags >> >> >> >> -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=y >> >> >> >> 2) Type of exception thrown (with a stacktrace or not) >> >> >> >> Exception class derived from java.lang.Error that overrides >> >> fillInStackTrace() with 'return this', so stackless. Same instances >> stored >> >> in static final fields are always thrown. >> >> >> >> 3) Is the call stack depth about the same during the slow and fast >> throws? >> >> >> >> It is always exactly the same stack, as it happens in a loop. >> >> >> >> 4) Is the exception thrown frequently or infrequently? >> >> >> >> In the test where this was observed, the exception was thrown every >> 40ms, >> >> so, I guess, frequently. >> >> >> >> 5) Is there -XX:+PrintCompilation output available around the time when >> >> the slowdown is observed >> >> >> >> Unfortunately not, and at the moment, I cannot reproduce the behavior. >> >> >> >> I made a mockup-test (see attachment) that mimicks kind of what my >> >> generated code does and adds measurements before throw and after catch. >> >> Mostly, in my experiments with this, the throw takes on average 20 ns, >> >> but between a 1 and 3 per million iterations take longer than 1 ms, on >> >> average around 7 ms. It can be observed that doing a garbage >> collection in >> >> a different java process (namely my eclipse that started the test) >> greatly >> >> influences the frequency of the > 1 ms throws, so maybe it is simply a >> load >> >> issue. Using multiple threads in parallel also seems to heighten the >> >> >> >> Also, when introducing an additional sleep between iterations (like in >> my >> >> original scenario where I was waiting for a timer to timeout before >> >> executing the construct that used the throw), I cannot reproduce such >> >> outliers at all. >> >> >> >> >> >> >> >> So, my question is. Is this a known (and for some strange reason maybe >> >> even accepted) behavior or is this a bug that I should file with >> Oracle (or >> >> you guys). >> >> >> >> >> >> You really are not giving us anything to go on here. How are you >> observing >> >> this slowness? Exactly how do you throw? What exactly are you >> measuring? >> >> What's the execution context, the machine, processors etc etc etc. >> >> >> >> >> >> I was observing the slowness in a testcase where I wanted regular >> >> intervals of 40 ms between sending a message via UDP but frequently >> (every >> >> few seconds) got a lag of around 1.5 seconds (and a few days later 0.5 >> >> seconds) before the timer which timed out at the correct time was >> noted. >> >> >> >> >> >> When eliminating one kind of such thrown control exception, the next >> kind >> >> of thrown control exception (from a surrounding construct) was having >> the >> >> same effect which actually led me to the slow-throw-hypothesis. >> Eliminating >> >> all exceptions in the small example confirmed this hypothesis, but >> going >> >> that route would take a lot of effort for all cases and introduce other >> >> additional imperformances (as I basically have to mock the exception >> >> throwing via cascading return and if-checks on every level) which I >> would >> >> like to avoid. >> >> >> >> Regards, >> >> David >> >> >> >> >> >> >> >> >> >> >> >> Spirent Communications e-mail confidentiality. >> >> ------------------------------------------------------------ >> ------------ >> >> This e-mail contains confidential and / or privileged information >> >> belonging to Spirent Communications plc, its affiliates and / or >> >> subsidiaries. If you are not the intended recipient, you are hereby >> >> notified that any disclosure, copying, distribution and / or the >> taking of >> >> any action based upon reliance on the contents of this transmission is >> >> strictly forbidden. If you have received this message in error please >> >> notify the sender by return e-mail and delete it from your system. >> >> >> >> Spirent Communications plc >> >> Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United >> >> Kingdom. >> >> Tel No. +44 (0) 1293 767676 <+44%201293%20767676> >> >> Fax No. +44 (0) 1293 767677 <+44%201293%20767677> >> >> >> >> Registered in England Number 470893 >> >> Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 >> >> 9XN, United Kingdom. >> >> >> >> Or if within the US, >> >> >> >> Spirent Communications, >> >> 27349 Agoura Road, Calabasas, CA, 91301, USA. >> >> Tel No. 1-818-676- 2300 <(818)%20676-2300> >> >> >> >> -- >> >> Sent from my phone >> >> >> >> -- >> Sent from my phone >> >> >> -- > Sent from my phone > From ioi.lam at oracle.com Fri Mar 3 03:37:27 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 02 Mar 2017 19:37:27 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> Message-ID: <58B8E4F7.5050301@oracle.com> Hi Coleen, Thanks for the comments. I have updated the webrev. See in-line for responses. http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ On 3/2/17 8:48 AM, coleen.phillimore at oracle.com wrote: > > Ioi > I like the concept of this a lot but have some stylistic comments to > help people reading this code later. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html > > s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't save many > characters, especially in CppVtableInfo/Testers > Done. > + // Start at slot 1, because slot 0 may be RTTI (on Solaris/Sparc) > + int i; > + for (i=1; ; i++) { > Since you're using 'i' later, can you rename it to something > descriptive. Or have another variable "vtable_length" to use later. > This looks like an old style for loop. > Done > Can the functions for CppVtableInfo be declared outside of the class > declaration? They don't need to be inline and then the debug code for > testing the vtable size can be not in the middle of the class > declaration. Then you can move the Tester classes to inside the same > #ifndef PRODUCT block. > > Can you put #endif // PRODUCT when the ifdef covers several lines of code? > Done > vtab_of could be more descriptive, like cpp_vtable_for(). > I changed to vtable_of(). Because the class name is already CppVtableCloner, repeating the word "cpp" seems repetitive to me. > Was PrintSharedSpaces was never converted to UL? > Right. I've filed https://bugs.openjdk.java.net/browse/JDK-8176132 (-XX:+PrintSharedSpaces should be converted to use Unified Logging.) > + int n = MAX_VTABLE_SIZE; > > Can you propagate MAX_VTABLE_SIZE to the places where it's used. n > isn't descriptive. This starts out with max_vtable_size and then > changes the size. Reusing 'n' makes this really hard to follow. Not > having a comment that we only allocate enough slots for the vtable > makes it hard too. > > + // allocate CppVtableInfo in the MD section > + _info = (CppVtabInfo*)md_top; > + _info->set_vtab_size(n); // initially set to max_vtable_size > + > + // allocate temporary local instance of the metadata type T > + T tmp; > + intptr_t* srcvtab = vtab_of(tmp); > + intptr_t* dstvtab = _info->vtab(); > + Fixed. > Something like that for comments. dstvtab is the destination_vtable > in the MD section. I've dropped the md_ prefix from the functions that deal with the vtables, since they shouldn't care whether it's the "MD" section or not. Now it looks like this: // Allocate and initialize the C++ vtables, starting from top, but do not go past end. intptr_t* MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, intptr_t* end) { assert(DumpSharedSpaces, "dump-time only"); // Layout (each slot is a intptr_t): // [number of slots in the first vtable = n1] // [ slots for the first vtable] // [number of slots in the first second = n2] // [ slots for the second vtable] // ... // The order of the vtables is the same as the CPP_VTAB_PATCH_TYPES_DO macro. CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); return top; } > + for (int i=0; i + const intptr_t bad = intptr_t(0xdeadbeef); > + intptr_t num = SafeFetchN(&srcvtab[i], bad); > + if (num == bad > + // || i > 120 /* uncomment this line to test */ > + ) { > + _info->set_vtab_size(i-1); > + break; > + } > + dstvtab[i] = num; > + } > > I dont understand this code. You get deadbeef for a bad value if > SafeFetchN gets a fault but why would it get a fault at the end of the > metadata's vtable? Couldn't it just run onto the next vtable? I > think your original way of counting vtable entries might be better > (sorry I didn't have time to study that thread). > I've modified the comments to this. Does it make sense to you? // It is not always safe to call memcpy(), because srcvtable might be shorter than // MAX_VTABLE_SIZE, and the C++ linker might have placed the vtable at the very // end of the last page of libjvm.so. Crossing over to the next page might // cause a page fault. My fear is the JVM would suddenly start crashing because the order of .o files have changed on the linker's command line, or if you enable some special linker optimization flags. It's better safe than sorry. > Would be nice to have comments here too!! > > + intptr_t* start = md_top; > > This doesn't do anything (?) Fixed. This was left over code. > > + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); > > Why not zero the destination vtable in allocate? Or does patching > the vtable pointers call virtual functions? You could prevent that so > you don't need this code. > I added this comment: // During patching, some virtual methods may be called, so at this point // the vtables must contain valid methods (as filled in by CppVtableCloner::allocate). MetaspaceShared::patch_cpp_vtable_pointers(); // The vtable clones contain addresses of the current process. // We don't want to write these addresses into the archive. MetaspaceShared::zero_cpp_vtable_clones_for_writing(); > + // Restore the vtable in case we invoke any virtual methods. > + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); > Can this be restore_cpp_vtables since that's what it's doing. The > first is after the dump and the second call is at UseSharedSpaces. A > couple of comments in this clone_cpp_vtables --> restore_cpp_vtables > would be nice. eg: > I prefer to use the word clone. Otherwise when you just say "vtable" it's not clear whether you're talking about the original one (made by the c++ linker), or the cloned one in the CDS archive. > + static intptr_t* clone_vtable(const char* name, intptr_t* p) { > + T tmp; // Allocate temporary dummy metadata object to get vtable initialized > + CppVtabInfo* info = (CppVtabInfo*)p; > + int n = info->vtab_size(); > + intptr_t* srcvtab = vtab_of(tmp); > + intptr_t* dstvtab = info->vtab(); > + > + // We already checked (and, if necessary, adjusted n) when the vtables were allocated, so we are > + // safe to do memcpy. > + if (PrintSharedSpaces) { > + tty->print_cr("%s copying %d vtable entries", name, n); > + } > + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); > + return dstvtab + n; > + } > Done. I changed the wording T tmp; // Allocate temporary dummy metadata object to get to the original vtable. As we are not really "initializing a vtable" here. > Same with 'patch'. It'd be so much faster and easier to read this > code with more comments please. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html > > Why are these testers here? > I updated the comment: // Used by CDS. These classes need to access the private ConstantPool() constructor. template friend class CppVtableTesterA; template friend class CppVtableTesterB; template friend class CppVtableCloner; Thanks - Ioi >> >>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>> >>>> Hi, >>>> >>>> This is the official review (follow up of the "Determining the size of C++ vtables" thread onhotspot-dev at openjdk.java.net). >>>> >>>> The new code has the same assumption as the existing code in JDK 10: for a C++ object that contains virtual methods (e.g., ConstantPool), we assume the first intptr_t slot of the object is a _vptr, which points to a vtable, which consists of no more than 150 intptr_t's. >>>> >>>> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >>>> [ field #0 ] [ vtable slot 1 ] >>>> [ field #1 ] [ vtable slot 2 ] >>>> [ field #2 ] [ vtable slot 3 ] >>>> [ .... ] [ vtable slot 4] >>>> [ vtable slot 5 ] >>>> [ ... ] >>>> >>>> + In the existing code, we were pointing the vtable slots to >>>> code that's generated by HotSpot. >>>> >>>> + In the new code, we copy the vtable slots from an existing >>>> vtable (generated by the C++ linker). >>>> >>>> Per Thomas St?fe's advice, I don't try to determine the size of the vtable (as that would add one more compiler requirement where new virtual methods added by a subclass must be placed at a higher offset in the vtable). >>>> >>>> Instead, I have added code in non-product builds to ensure that the vtables are no longer than 150 entries. You can run with "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size of the vtables for your particular platform: >>>> >>>> ConstantPool has 12 virtual methods >>>> InstanceKlass has 113 virtual methods >>>> InstanceClassLoaderKlass has 113 virtual methods >>>> InstanceMirrorKlass has 113 virtual methods >>>> InstanceRefKlass has 113 virtual methods >>>> Method has 12 virtual methods >>>> ObjArrayKlass has 114 virtual methods >>>> TypeArrayKlass has 114 virtual methods >>>> >>>> As mentioned in the code comments, if you have an esoteric C++ compiler, the verify_sufficient_size() function will probably fail, but hopefully that would give you some hints for porting this code. >>>> >>>> To avoid accidentally touching an unmapped page, the code uses SafeFetchN for copying the vtable contents, and would shrink the vtable to less than 150 entries if necessary. I can't test this for real, but I've added some code to simulate an error: >>>> >>>> for (int i=0; i>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>> if (num == bad >>>> // || i > 120 /* uncomment this line to test */ >>>> ) { >>>> _info->set_vtab_size(i-1); >>>> break; >>>> } >>>> dstvtab[i] = num; >>>> } >>>> >>>> Results: >>>> >>>> + Removed 850 lines of CPU-dependent code >>>> >>>> + CDS image is about 50K smaller >>>> >>>> + Previously Metadata objects must live in the read-write section in the CDS >>>> archive, because their _vptr was updated at run time. Now _vptr is no longer >>>> updated, so ConstantPool can be moved to the read-only section (see JDK-8171392). >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> > From ioi.lam at oracle.com Fri Mar 3 03:40:49 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 02 Mar 2017 19:40:49 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> Message-ID: <58B8E5C1.6000901@oracle.com> On 3/1/17 2:22 AM, Thomas St?fe wrote: > Hi Ioi, > > I did not yet look closely at your change yet, just a small nit: To > prevent the copying SafeFetch coding from accidentally tripping over a > real "deadbeef" value which may be a valid part of the vtable - > however completely unlikely this is - one could call SafeFetch twice > with two different bad values,eg: > > const intptr_t bad1 = intptr_t(0xdeadbeef); > const intptr_t bad2 = intptr_t(0xbeefdead); > if (SafeFetchN(ptr, bad1) == bad1 && SafeFetchN(ptr, bad2) == bad2) { > ... break out ... } > > Kind Regards, Thomas > Hi Thomas, thanks for the suggestion. Now the code looks like this: const intptr_t bad1 = intptr_t(0xdeadbeef); const intptr_t bad2 = intptr_t(0xbaadf00d); intptr_t num = SafeFetchN(&srcvtable[i], bad1); if ((num == bad1 && SafeFetchN(&srcvtable[i], bad2) == bad2) // || i > 120 /* uncomment this line to test */ ) { _info->set_vtable_size(i-1); break; } I've updated the webrev with this change and other recommendations by Coleen: http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ Thanks - Ioi > > On Wed, Mar 1, 2017 at 3:25 AM, Ioi Lam > wrote: > > https://bugs.openjdk.java.net/browse/JDK-8005165 > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ > > > Hi, > > This is the official review (follow up of the "Determining the > size of C++ vtables" thread on hotspot-dev at openjdk.java.net > ). > > The new code has the same assumption as the existing code in JDK > 10: for a C++ object that contains virtual methods (e.g., > ConstantPool), we assume the first intptr_t slot of the object is > a _vptr, which points to a vtable, which consists of no more than > 150 intptr_t's. > > ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] > [ field #0 ] [ vtable slot 1 ] > [ field #1 ] [ vtable slot 2 ] > [ field #2 ] [ vtable slot 3 ] > [ .... ] [ vtable slot 4] > [ vtable slot 5 ] > [ ... ] > > + In the existing code, we were pointing the vtable slots to > code that's generated by HotSpot. > > + In the new code, we copy the vtable slots from an existing > vtable (generated by the C++ linker). > > Per Thomas St?fe's advice, I don't try to determine the size of > the vtable (as that would add one more compiler requirement where > new virtual methods added by a subclass must be placed at a higher > offset in the vtable). > > Instead, I have added code in non-product builds to ensure that > the vtables are no longer than 150 entries. You can run with > "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size > of the vtables for your particular platform: > > ConstantPool has 12 virtual methods > InstanceKlass has 113 virtual methods > InstanceClassLoaderKlass has 113 virtual methods > InstanceMirrorKlass has 113 virtual methods > InstanceRefKlass has 113 virtual methods > Method has 12 virtual methods > ObjArrayKlass has 114 virtual methods > TypeArrayKlass has 114 virtual methods > > As mentioned in the code comments, if you have an esoteric C++ > compiler, the verify_sufficient_size() function will probably > fail, but hopefully that would give you some hints for porting > this code. > > To avoid accidentally touching an unmapped page, the code uses > SafeFetchN for copying the vtable contents, and would shrink the > vtable to less than 150 entries if necessary. I can't test this > for real, but I've added some code to simulate an error: > > for (int i=0; i const intptr_t bad = intptr_t(0xdeadbeef); > intptr_t num = SafeFetchN(&srcvtab[i], bad); > if (num == bad > // || i > 120 /* uncomment this line to test */ > ) { > _info->set_vtab_size(i-1); > break; > } > dstvtab[i] = num; > } > > Results: > > + Removed 850 lines of CPU-dependent code > > + CDS image is about 50K smaller > > + Previously Metadata objects must live in the read-write section > in the CDS > archive, because their _vptr was updated at run time. Now _vptr > is no longer > updated, so ConstantPool can be moved to the read-only section > (see JDK-8171392). > > Thanks > - Ioi > > > > From christian.tornqvist at oracle.com Fri Mar 3 15:41:42 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Fri, 3 Mar 2017 10:41:42 -0500 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* Message-ID: <169f01d29434$a84a7d70$f8df7850$@oracle.com> Hi everyone, Please review this small change that renames the hotspot_fast* test groups to hotspot_tier1, also renamed the hotspot_runtime_tier* to follow the same naming standard. Tested by running the change through JPRT. Webrevs: http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8176102 Thanks, Christian From george.triantafillou at oracle.com Fri Mar 3 16:19:28 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Fri, 3 Mar 2017 11:19:28 -0500 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* In-Reply-To: <169f01d29434$a84a7d70$f8df7850$@oracle.com> References: <169f01d29434$a84a7d70$f8df7850$@oracle.com> Message-ID: <6d286ef6-10d6-9c65-2d93-07c7835cea6b@oracle.com> Hi Christian, Looks good. -George On 3/3/2017 10:41 AM, Christian Tornqvist wrote: > Hi everyone, > > > > Please review this small change that renames the hotspot_fast* test groups > to hotspot_tier1, also renamed the hotspot_runtime_tier* to follow the same > naming standard. > > Tested by running the change through JPRT. > > > > Webrevs: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ > > http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8176102 > > > > Thanks, > > Christian > From mikhailo.seledtsov at oracle.com Fri Mar 3 17:45:31 2017 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Fri, 03 Mar 2017 09:45:31 -0800 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* In-Reply-To: <6d286ef6-10d6-9c65-2d93-07c7835cea6b@oracle.com> References: <169f01d29434$a84a7d70$f8df7850$@oracle.com> <6d286ef6-10d6-9c65-2d93-07c7835cea6b@oracle.com> Message-ID: <58B9ABBB.5070408@oracle.com> Changes look good to me, Misha On 3/3/17, 8:19 AM, George Triantafillou wrote: > Hi Christian, > > Looks good. > > -George > > On 3/3/2017 10:41 AM, Christian Tornqvist wrote: >> Hi everyone, >> >> >> Please review this small change that renames the hotspot_fast* test >> groups >> to hotspot_tier1, also renamed the hotspot_runtime_tier* to follow >> the same >> naming standard. >> >> Tested by running the change through JPRT. >> >> >> Webrevs: >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00/ >> >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8176102 >> >> >> Thanks, >> >> Christian >> > From jiangli.zhou at oracle.com Sat Mar 4 00:05:29 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 3 Mar 2017 16:05:29 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58B8E5C1.6000901@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <58B8E5C1.6000901@oracle.com> Message-ID: <910E2C1B-0D44-4F05-AC8B-12206190F48B@oracle.com> Hi Ioi, This is great! I think it might be more beneficial to not zero out the cloned vtables before writing into the archive. If the runtime libjvm.so is loaded at the same based address, then you could avoid re-cloning the vtables at runtime completely with just one quick comparison of the libjvm base address. In that case, the MD would be read-only also. That would improve both the memory and startup efficiency a bit further. 883 // The vtable clones contain addresses of the current process. 884 // We don't want to write these addresses into the archive. 885 MetaspaceShared::zero_cpp_vtable_clones_for_writing(); Thanks, Jiangli > On Mar 2, 2017, at 7:40 PM, Ioi Lam wrote: > > > > On 3/1/17 2:22 AM, Thomas St?fe wrote: >> Hi Ioi, >> >> I did not yet look closely at your change yet, just a small nit: To prevent the copying SafeFetch coding from accidentally tripping over a real "deadbeef" value which may be a valid part of the vtable - however completely unlikely this is - one could call SafeFetch twice with two different bad values,eg: >> >> const intptr_t bad1 = intptr_t(0xdeadbeef); >> const intptr_t bad2 = intptr_t(0xbeefdead); >> if (SafeFetchN(ptr, bad1) == bad1 && SafeFetchN(ptr, bad2) == bad2) { ... break out ... } >> >> Kind Regards, Thomas >> > > Hi Thomas, thanks for the suggestion. Now the code looks like this: > > const intptr_t bad1 = intptr_t(0xdeadbeef); > const intptr_t bad2 = intptr_t(0xbaadf00d); > intptr_t num = SafeFetchN(&srcvtable[i], bad1); > if ((num == bad1 && SafeFetchN(&srcvtable[i], bad2) == bad2) > // || i > 120 /* uncomment this line to test */ > ) { > _info->set_vtable_size(i-1); > break; > } > > I've updated the webrev with this change and other recommendations by Coleen: > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ > > Thanks > - Ioi > >> >> On Wed, Mar 1, 2017 at 3:25 AM, Ioi Lam >> wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8005165 >> > >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >> > >> >> Hi, >> >> This is the official review (follow up of the "Determining the >> size of C++ vtables" thread on hotspot-dev at openjdk.java.net >> >). >> >> The new code has the same assumption as the existing code in JDK >> 10: for a C++ object that contains virtual methods (e.g., >> ConstantPool), we assume the first intptr_t slot of the object is >> a _vptr, which points to a vtable, which consists of no more than >> 150 intptr_t's. >> >> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >> [ field #0 ] [ vtable slot 1 ] >> [ field #1 ] [ vtable slot 2 ] >> [ field #2 ] [ vtable slot 3 ] >> [ .... ] [ vtable slot 4] >> [ vtable slot 5 ] >> [ ... ] >> >> + In the existing code, we were pointing the vtable slots to >> code that's generated by HotSpot. >> >> + In the new code, we copy the vtable slots from an existing >> vtable (generated by the C++ linker). >> >> Per Thomas St?fe's advice, I don't try to determine the size of >> the vtable (as that would add one more compiler requirement where >> new virtual methods added by a subclass must be placed at a higher >> offset in the vtable). >> >> Instead, I have added code in non-product builds to ensure that >> the vtables are no longer than 150 entries. You can run with >> "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size >> of the vtables for your particular platform: >> >> ConstantPool has 12 virtual methods >> InstanceKlass has 113 virtual methods >> InstanceClassLoaderKlass has 113 virtual methods >> InstanceMirrorKlass has 113 virtual methods >> InstanceRefKlass has 113 virtual methods >> Method has 12 virtual methods >> ObjArrayKlass has 114 virtual methods >> TypeArrayKlass has 114 virtual methods >> >> As mentioned in the code comments, if you have an esoteric C++ >> compiler, the verify_sufficient_size() function will probably >> fail, but hopefully that would give you some hints for porting >> this code. >> >> To avoid accidentally touching an unmapped page, the code uses >> SafeFetchN for copying the vtable contents, and would shrink the >> vtable to less than 150 entries if necessary. I can't test this >> for real, but I've added some code to simulate an error: >> >> for (int i=0; i> const intptr_t bad = intptr_t(0xdeadbeef); >> intptr_t num = SafeFetchN(&srcvtab[i], bad); >> if (num == bad >> // || i > 120 /* uncomment this line to test */ >> ) { >> _info->set_vtab_size(i-1); >> break; >> } >> dstvtab[i] = num; >> } >> >> Results: >> >> + Removed 850 lines of CPU-dependent code >> >> + CDS image is about 50K smaller >> >> + Previously Metadata objects must live in the read-write section >> in the CDS >> archive, because their _vptr was updated at run time. Now _vptr >> is no longer >> updated, so ConstantPool can be moved to the read-only section >> (see JDK-8171392). >> >> Thanks >> - Ioi From david.holmes at oracle.com Sat Mar 4 02:21:28 2017 From: david.holmes at oracle.com (David Holmes) Date: Sat, 4 Mar 2017 12:21:28 +1000 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* In-Reply-To: <169f01d29434$a84a7d70$f8df7850$@oracle.com> References: <169f01d29434$a84a7d70$f8df7850$@oracle.com> Message-ID: <86ccedc2-37e1-9360-f0d5-5ed2338b0c8d@oracle.com> Hi Christian, Rename looks good. Should we also rename the now oddly named hotspot_not_fast_compiler? Thanks, David On 4/03/2017 1:41 AM, Christian Tornqvist wrote: > Hi everyone, > > > > Please review this small change that renames the hotspot_fast* test groups > to hotspot_tier1, also renamed the hotspot_runtime_tier* to follow the same > naming standard. > > Tested by running the change through JPRT. > > > > Webrevs: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ > > http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8176102 > > > > Thanks, > > Christian > From ioi.lam at oracle.com Sat Mar 4 05:43:06 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 03 Mar 2017 21:43:06 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <910E2C1B-0D44-4F05-AC8B-12206190F48B@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <58B8E5C1.6000901@oracle.com> <910E2C1B-0D44-4F05-AC8B-12206190F48B@oracle.com> Message-ID: <58BA53EA.10807@oracle.com> Hi Jiangli, Thanks for the suggestion. I tried implemented the code like this: static intptr_t* clone_vtable(const char* name, intptr_t* p) { .... for (int i=0; iprint_cr("Updated: " INTPTR_FORMAT " -> " INTPTR_FORMAT, q, p); } } } but it turns out with Address Space Layout Randomization, which is now enabled on most (all?) OS for security reasons, libjvm.so/libjvm.dll will always be loaded at a different address: [run1] Updated: 0x00007f040f27e7dc -> 0x00007fdddf0ac7dc [run2] Updated: 0x00007f040f27e7dc -> 0x00007fba31c367dc [run3] Updated: 0x00007f040f27e7dc -> 0x00007f585342d7dc Also, we are writing just a small amount of data (9600 bytes on 64-bit, 4800 bytes on 32-bit), so I think the unlikely space saving is not worth this extra effort. Zeroing the vtables inside the archive is also for security reasons. It makes sure that even if the VM fails to initialize the vtables due to a bug, we will never jump to an address that might contain unexpected code. Thanks - Ioi On 3/3/17 4:05 PM, Jiangli Zhou wrote: > Hi Ioi, > > This is great! > > I think it might be more beneficial to not zero out the cloned vtables > before writing into the archive. If the runtime libjvm.so is loaded at > the same based address, then you could avoid re-cloning the vtables at > runtime completely with just one quick comparison of the libjvm base > address. In that case, the MD would be read-only also. That would > improve both the memory and startup efficiency a bit further. > 883 // The vtable clones contain addresses of the current process. > 884 // We don't want to write these addresses into the archive. > 885 MetaspaceShared::zero_cpp_vtable_clones_for_writing(); > Thanks, > Jiangli > >> On Mar 2, 2017, at 7:40 PM, Ioi Lam > > wrote: >> >> >> >> On 3/1/17 2:22 AM, Thomas St?fe wrote: >>> Hi Ioi, >>> >>> I did not yet look closely at your change yet, just a small nit: To >>> prevent the copying SafeFetch coding from accidentally tripping over >>> a real "deadbeef" value which may be a valid part of the vtable - >>> however completely unlikely this is - one could call SafeFetch twice >>> with two different bad values,eg: >>> >>> const intptr_t bad1 = intptr_t(0xdeadbeef); >>> const intptr_t bad2 = intptr_t(0xbeefdead); >>> if (SafeFetchN(ptr, bad1) == bad1 && SafeFetchN(ptr, bad2) == bad2) >>> { ... break out ... } >>> >>> Kind Regards, Thomas >>> >> >> Hi Thomas, thanks for the suggestion. Now the code looks like this: >> >> const intptr_t bad1 = intptr_t(0xdeadbeef); >> const intptr_t bad2 = intptr_t(0xbaadf00d); >> intptr_t num = SafeFetchN(&srcvtable[i], bad1); >> if ((num == bad1 && SafeFetchN(&srcvtable[i], bad2) == bad2) >> // || i > 120 /* uncomment this line to test */ >> ) { >> _info->set_vtable_size(i-1); >> break; >> } >> >> I've updated the webrev with this change and other recommendations by >> Coleen: >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >> >> >> Thanks >> - Ioi >> >>> >>> On Wed, Mar 1, 2017 at 3:25 AM, Ioi Lam >> > wrote: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>> >>> >>> >>> Hi, >>> >>> This is the official review (follow up of the "Determining the >>> size of C++ vtables" thread onhotspot-dev at openjdk.java.net >>> >>> ). >>> >>> The new code has the same assumption as the existing code in JDK >>> 10: for a C++ object that contains virtual methods (e.g., >>> ConstantPool), we assume the first intptr_t slot of the object is >>> a _vptr, which points to a vtable, which consists of no more than >>> 150 intptr_t's. >>> >>> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >>> [ field #0 ] [ vtable slot 1 ] >>> [ field #1 ] [ vtable slot 2 ] >>> [ field #2 ] [ vtable slot 3 ] >>> [ .... ] [ vtable slot 4] >>> [ vtable slot 5 ] >>> [ ... ] >>> >>> + In the existing code, we were pointing the vtable slots to >>> code that's generated by HotSpot. >>> >>> + In the new code, we copy the vtable slots from an existing >>> vtable (generated by the C++ linker). >>> >>> Per Thomas St?fe's advice, I don't try to determine the size of >>> the vtable (as that would add one more compiler requirement where >>> new virtual methods added by a subclass must be placed at a higher >>> offset in the vtable). >>> >>> Instead, I have added code in non-product builds to ensure that >>> the vtables are no longer than 150 entries. You can run with >>> "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size >>> of the vtables for your particular platform: >>> >>> ConstantPool has 12 virtual methods >>> InstanceKlass has 113 virtual methods >>> InstanceClassLoaderKlass has 113 virtual methods >>> InstanceMirrorKlass has 113 virtual methods >>> InstanceRefKlass has 113 virtual methods >>> Method has 12 virtual methods >>> ObjArrayKlass has 114 virtual methods >>> TypeArrayKlass has 114 virtual methods >>> >>> As mentioned in the code comments, if you have an esoteric C++ >>> compiler, the verify_sufficient_size() function will probably >>> fail, but hopefully that would give you some hints for porting >>> this code. >>> >>> To avoid accidentally touching an unmapped page, the code uses >>> SafeFetchN for copying the vtable contents, and would shrink the >>> vtable to less than 150 entries if necessary. I can't test this >>> for real, but I've added some code to simulate an error: >>> >>> for (int i=0; i>> const intptr_t bad = intptr_t(0xdeadbeef); >>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>> if (num == bad >>> // || i > 120 /* uncomment this line to test */ >>> ) { >>> _info->set_vtab_size(i-1); >>> break; >>> } >>> dstvtab[i] = num; >>> } >>> >>> Results: >>> >>> + Removed 850 lines of CPU-dependent code >>> >>> + CDS image is about 50K smaller >>> >>> + Previously Metadata objects must live in the read-write section >>> in the CDS >>> archive, because their _vptr was updated at run time. Now _vptr >>> is no longer >>> updated, so ConstantPool can be moved to the read-only section >>> (see JDK-8171392). >>> >>> Thanks >>> - Ioi > From jiangli.zhou at oracle.com Sat Mar 4 05:59:25 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 3 Mar 2017 21:59:25 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BA53EA.10807@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <58B8E5C1.6000901@oracle.com> <910E2C1B-0D44-4F05-AC8B-12206190F48B@oracle.com> <58BA53EA.10807@oracle.com> Message-ID: <225FFA19-676A-46AB-9836-86EAE511B4DC@oracle.com> > On Mar 3, 2017, at 9:43 PM, Ioi Lam wrote: > > Hi Jiangli, > > Thanks for the suggestion. I tried implemented the code like this: > > static intptr_t* clone_vtable(const char* name, intptr_t* p) { > .... > for (int i=0; i intptr_t p = srcvtable[i]; > intptr_t q = dstvtable[i]; > if (p != q) { > dstvtable[i] = p; > if (i == 0) { > tty->print_cr("Updated: " INTPTR_FORMAT " -> " INTPTR_FORMAT, q, p); > } > } > } > > but it turns out with Address Space Layout Randomization, which is now enabled on most (all?) OS for security reasons, libjvm.so/libjvm.dll will always be loaded at a different address: > > [run1] Updated: 0x00007f040f27e7dc -> 0x00007fdddf0ac7dc > [run2] Updated: 0x00007f040f27e7dc -> 0x00007fba31c367dc > [run3] Updated: 0x00007f040f27e7dc -> 0x00007f585342d7dc > > Also, we are writing just a small amount of data (9600 bytes on 64-bit, 4800 bytes on 32-bit), so I think the unlikely space saving is not worth this extra effort. > > Zeroing the vtables inside the archive is also for security reasons. It makes sure that even if the VM fails to initialize the vtables due to a bug, we will never jump to an address that might contain unexpected code. Ok. Thanks for trying that out. Jiangli > > Thanks > - Ioi > > > On 3/3/17 4:05 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> This is great! >> >> I think it might be more beneficial to not zero out the cloned vtables before writing into the archive. If the runtime libjvm.so is loaded at the same based address, then you could avoid re-cloning the vtables at runtime completely with just one quick comparison of the libjvm base address. In that case, the MD would be read-only also. That would improve both the memory and startup efficiency a bit further. >> 883 // The vtable clones contain addresses of the current process. >> 884 // We don't want to write these addresses into the archive. >> 885 MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >> Thanks, >> Jiangli >> >>> On Mar 2, 2017, at 7:40 PM, Ioi Lam > wrote: >>> >>> >>> >>> On 3/1/17 2:22 AM, Thomas St?fe wrote: >>>> Hi Ioi, >>>> >>>> I did not yet look closely at your change yet, just a small nit: To prevent the copying SafeFetch coding from accidentally tripping over a real "deadbeef" value which may be a valid part of the vtable - however completely unlikely this is - one could call SafeFetch twice with two different bad values,eg: >>>> >>>> const intptr_t bad1 = intptr_t(0xdeadbeef); >>>> const intptr_t bad2 = intptr_t(0xbeefdead); >>>> if (SafeFetchN(ptr, bad1) == bad1 && SafeFetchN(ptr, bad2) == bad2) { ... break out ... } >>>> >>>> Kind Regards, Thomas >>>> >>> >>> Hi Thomas, thanks for the suggestion. Now the code looks like this: >>> >>> const intptr_t bad1 = intptr_t(0xdeadbeef); >>> const intptr_t bad2 = intptr_t(0xbaadf00d); >>> intptr_t num = SafeFetchN(&srcvtable[i], bad1); >>> if ((num == bad1 && SafeFetchN(&srcvtable[i], bad2) == bad2) >>> // || i > 120 /* uncomment this line to test */ >>> ) { >>> _info->set_vtable_size(i-1); >>> break; >>> } >>> >>> I've updated the webrev with this change and other recommendations by Coleen: >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>> >>> Thanks >>> - Ioi >>> >>>> >>>> On Wed, Mar 1, 2017 at 3:25 AM, Ioi Lam >> wrote: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>> > >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>> > >>>> >>>> Hi, >>>> >>>> This is the official review (follow up of the "Determining the >>>> size of C++ vtables" thread on hotspot-dev at openjdk.java.net >>>> >). >>>> >>>> The new code has the same assumption as the existing code in JDK >>>> 10: for a C++ object that contains virtual methods (e.g., >>>> ConstantPool), we assume the first intptr_t slot of the object is >>>> a _vptr, which points to a vtable, which consists of no more than >>>> 150 intptr_t's. >>>> >>>> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >>>> [ field #0 ] [ vtable slot 1 ] >>>> [ field #1 ] [ vtable slot 2 ] >>>> [ field #2 ] [ vtable slot 3 ] >>>> [ .... ] [ vtable slot 4] >>>> [ vtable slot 5 ] >>>> [ ... ] >>>> >>>> + In the existing code, we were pointing the vtable slots to >>>> code that's generated by HotSpot. >>>> >>>> + In the new code, we copy the vtable slots from an existing >>>> vtable (generated by the C++ linker). >>>> >>>> Per Thomas St?fe's advice, I don't try to determine the size of >>>> the vtable (as that would add one more compiler requirement where >>>> new virtual methods added by a subclass must be placed at a higher >>>> offset in the vtable). >>>> >>>> Instead, I have added code in non-product builds to ensure that >>>> the vtables are no longer than 150 entries. You can run with >>>> "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size >>>> of the vtables for your particular platform: >>>> >>>> ConstantPool has 12 virtual methods >>>> InstanceKlass has 113 virtual methods >>>> InstanceClassLoaderKlass has 113 virtual methods >>>> InstanceMirrorKlass has 113 virtual methods >>>> InstanceRefKlass has 113 virtual methods >>>> Method has 12 virtual methods >>>> ObjArrayKlass has 114 virtual methods >>>> TypeArrayKlass has 114 virtual methods >>>> >>>> As mentioned in the code comments, if you have an esoteric C++ >>>> compiler, the verify_sufficient_size() function will probably >>>> fail, but hopefully that would give you some hints for porting >>>> this code. >>>> >>>> To avoid accidentally touching an unmapped page, the code uses >>>> SafeFetchN for copying the vtable contents, and would shrink the >>>> vtable to less than 150 entries if necessary. I can't test this >>>> for real, but I've added some code to simulate an error: >>>> >>>> for (int i=0; i>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>> if (num == bad >>>> // || i > 120 /* uncomment this line to test */ >>>> ) { >>>> _info->set_vtab_size(i-1); >>>> break; >>>> } >>>> dstvtab[i] = num; >>>> } >>>> >>>> Results: >>>> >>>> + Removed 850 lines of CPU-dependent code >>>> >>>> + CDS image is about 50K smaller >>>> >>>> + Previously Metadata objects must live in the read-write section >>>> in the CDS >>>> archive, because their _vptr was updated at run time. Now _vptr >>>> is no longer >>>> updated, so ConstantPool can be moved to the read-only section >>>> (see JDK-8171392). >>>> >>>> Thanks >>>> - Ioi >> > From coleen.phillimore at oracle.com Sun Mar 5 15:17:46 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Sun, 5 Mar 2017 10:17:46 -0500 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58B8E4F7.5050301@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> Message-ID: <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> Ioi, Some comments inline (where no comments, insert "ok") :) On 3/2/17 10:37 PM, Ioi Lam wrote: > Hi Coleen, > > Thanks for the comments. I have updated the webrev. See in-line for > responses. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ > > > On 3/2/17 8:48 AM, coleen.phillimore at oracle.com wrote: >> >> Ioi >> I like the concept of this a lot but have some stylistic comments to >> help people reading this code later. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >> >> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't save many >> characters, especially in CppVtableInfo/Testers >> > Done. >> + // Start at slot 1, because slot 0 may be RTTI (on Solaris/Sparc) >> + int i; >> + for (i=1; ; i++) { >> Since you're using 'i' later, can you rename it to something >> descriptive. Or have another variable "vtable_length" to use >> later. This looks like an old style for loop. >> > Done >> Can the functions for CppVtableInfo be declared outside of the class >> declaration? They don't need to be inline and then the debug code >> for testing the vtable size can be not in the middle of the class >> declaration. Then you can move the Tester classes to inside the >> same #ifndef PRODUCT block. >> >> Can you put #endif // PRODUCT when the ifdef covers several lines of >> code? >> > Done >> vtab_of could be more descriptive, like cpp_vtable_for(). >> > I changed to vtable_of(). Because the class name is already > CppVtableCloner, repeating the word "cpp" seems repetitive to me. > >> Was PrintSharedSpaces was never converted to UL? >> > Right. I've filed https://bugs.openjdk.java.net/browse/JDK-8176132 > (-XX:+PrintSharedSpaces should be converted to use Unified Logging.) >> + int n = MAX_VTABLE_SIZE; >> >> Can you propagate MAX_VTABLE_SIZE to the places where it's used. n >> isn't descriptive. This starts out with max_vtable_size and then >> changes the size. Reusing 'n' makes this really hard to follow. Not >> having a comment that we only allocate enough slots for the vtable >> makes it hard too. >> >> + // allocate CppVtableInfo in the MD section + _info = >> (CppVtabInfo*)md_top; >> + _info->set_vtab_size(n); // initially set to max_vtable_size >> + + // allocate temporary local instance of the metadata type T + T tmp; >> + intptr_t* srcvtab = vtab_of(tmp); >> + intptr_t* dstvtab = _info->vtab(); >> + > Fixed. >> Something like that for comments. dstvtab is the destination_vtable >> in the MD section. > > I've dropped the md_ prefix from the functions that deal with the > vtables, since they shouldn't care whether it's the "MD" section or > not. Now it looks like this: > > // Allocate and initialize the C++ vtables, starting from top, but do > not go past end. > intptr_t* MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, > intptr_t* end) { > assert(DumpSharedSpaces, "dump-time only"); > // Layout (each slot is a intptr_t): > // [number of slots in the first vtable = n1] > // [ slots for the first vtable] > // [number of slots in the first second = n2] > // [ slots for the second vtable] > // ... > // The order of the vtables is the same as the > CPP_VTAB_PATCH_TYPES_DO macro. > CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); > return top; > } > >> + for (int i=0; i> + const intptr_t bad = intptr_t(0xdeadbeef); >> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >> + if (num == bad >> + // || i > 120 /* uncomment this line to test */ >> + ) { >> + _info->set_vtab_size(i-1); >> + break; >> + } >> + dstvtab[i] = num; >> + } >> I dont understand this code. You get deadbeef for a bad value if >> SafeFetchN gets a fault but why would it get a fault at the end of >> the metadata's vtable? Couldn't it just run onto the next vtable? >> I think your original way of counting vtable entries might be better >> (sorry I didn't have time to study that thread). >> > I've modified the comments to this. Does it make sense to you? > > // It is not always safe to call memcpy(), because srcvtable might > be shorter than > // MAX_VTABLE_SIZE, and the C++ linker might have placed the > vtable at the very > // end of the last page of libjvm.so. Crossing over to the next > page might > // cause a page fault. > > My fear is the JVM would suddenly start crashing because the order of > .o files have changed on the linker's command line, or if you enable > some special linker optimization flags. It's better safe than sorry. This wasn't exactly what I was not understanding. I didn't see that you are copying 120 entries from the old vtable and junk memory beyond the old vtable, unless you get a segv, in which case you copy less. I don't think you should copy random memory into the vtable in the archive. This doesn't seem secure, even with the segv protection. Since we already have assumptions about C++ vtable layout in the code and it's mostly specified by various ABIs, and you have the assert code, I think I would prefer that you copy only the vtable entries into the archive. I guess Thomas Stuefe had a different opinion. I've read the original thread. Two points: If new C++ compiler implementations add a discontigous vtable, both the SafeFetchN and subclass additional virtual function at end implementation will fail. I don't think C++ implementations would do this and a contiguous vtable as first in the instance has been standard for years. If our metadata adds multiple inheritance, the same issue would be a problem for both implementations, as well as for the implementation we have before Ioi's fix. Ioi's subclass adding virtual function method would work for any esoteric C++ implementations in my memory, except the vptr for the old DECC++ compiler was after the nonstatic data members (which would fail with all of our implementations). Since the code is there anyway for debug purposes, we're not saving code by implementing SafeFetchN. The SafeFetchN implementation isn't obvious at all what it's doing, and requires better comments, especially if you don't know already what SafeFetchN does. It looks really cryptic. The poisoned values also bothered me in that they overload other poisoned values in other parts of the jvm. Ioi, could you make all methods of CppVtableCloner out of line? The other changes look good, although I might have more requests for comments. Thanks, Coleen >> Would be nice to have comments here too!! >> >> + intptr_t* start = md_top; >> >> This doesn't do anything (?) > > Fixed. This was left over code. >> >> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >> >> Why not zero the destination vtable in allocate? Or does patching >> the vtable pointers call virtual functions? You could prevent that >> so you don't need this code. >> > I added this comment: > > // During patching, some virtual methods may be called, so at this point > // the vtables must contain valid methods (as filled in by > CppVtableCloner::allocate). > MetaspaceShared::patch_cpp_vtable_pointers(); > > // The vtable clones contain addresses of the current process. > // We don't want to write these addresses into the archive. > MetaspaceShared::zero_cpp_vtable_clones_for_writing(); > >> + // Restore the vtable in case we invoke any virtual methods. >> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >> Can this be restore_cpp_vtables since that's what it's doing. The >> first is after the dump and the second call is at UseSharedSpaces. >> A couple of comments in this clone_cpp_vtables --> >> restore_cpp_vtables would be nice. eg: >> > I prefer to use the word clone. Otherwise when you just say "vtable" > it's not clear whether you're talking about the original one (made by > the c++ linker), or the cloned one in the CDS archive. >> + static intptr_t* clone_vtable(const char* name, intptr_t* p) { >> + T tmp; // Allocate temporary dummy metadata object to get vtable initialized >> + CppVtabInfo* info = (CppVtabInfo*)p; >> + int n = info->vtab_size(); >> + intptr_t* srcvtab = vtab_of(tmp); >> + intptr_t* dstvtab = info->vtab(); >> + >> + // We already checked (and, if necessary, adjusted n) when the >> vtables were allocated, so we are >> + // safe to do memcpy. >> + if (PrintSharedSpaces) { >> + tty->print_cr("%s copying %d vtable entries", name, n); >> + } >> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >> + return dstvtab + n; >> + } >> > Done. I changed the wording > > T tmp; // Allocate temporary dummy metadata object to get to the > original vtable. > > As we are not really "initializing a vtable" here. > >> Same with 'patch'. It'd be so much faster and easier to read this >> code with more comments please. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >> >> Why are these testers here? >> > > I updated the comment: > > // Used by CDS. These classes need to access the private > ConstantPool() constructor. > template friend class CppVtableTesterA; > template friend class CppVtableTesterB; > template friend class CppVtableCloner; > > > Thanks > - Ioi > >>> >>>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>>> >>>>> Hi, >>>>> >>>>> This is the official review (follow up of the "Determining the size of C++ vtables" thread onhotspot-dev at openjdk.java.net). >>>>> >>>>> The new code has the same assumption as the existing code in JDK 10: for a C++ object that contains virtual methods (e.g., ConstantPool), we assume the first intptr_t slot of the object is a _vptr, which points to a vtable, which consists of no more than 150 intptr_t's. >>>>> >>>>> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >>>>> [ field #0 ] [ vtable slot 1 ] >>>>> [ field #1 ] [ vtable slot 2 ] >>>>> [ field #2 ] [ vtable slot 3 ] >>>>> [ .... ] [ vtable slot 4] >>>>> [ vtable slot 5 ] >>>>> [ ... ] >>>>> >>>>> + In the existing code, we were pointing the vtable slots to >>>>> code that's generated by HotSpot. >>>>> >>>>> + In the new code, we copy the vtable slots from an existing >>>>> vtable (generated by the C++ linker). >>>>> >>>>> Per Thomas St?fe's advice, I don't try to determine the size of the vtable (as that would add one more compiler requirement where new virtual methods added by a subclass must be placed at a higher offset in the vtable). >>>>> >>>>> Instead, I have added code in non-product builds to ensure that the vtables are no longer than 150 entries. You can run with "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size of the vtables for your particular platform: >>>>> >>>>> ConstantPool has 12 virtual methods >>>>> InstanceKlass has 113 virtual methods >>>>> InstanceClassLoaderKlass has 113 virtual methods >>>>> InstanceMirrorKlass has 113 virtual methods >>>>> InstanceRefKlass has 113 virtual methods >>>>> Method has 12 virtual methods >>>>> ObjArrayKlass has 114 virtual methods >>>>> TypeArrayKlass has 114 virtual methods >>>>> >>>>> As mentioned in the code comments, if you have an esoteric C++ compiler, the verify_sufficient_size() function will probably fail, but hopefully that would give you some hints for porting this code. >>>>> >>>>> To avoid accidentally touching an unmapped page, the code uses SafeFetchN for copying the vtable contents, and would shrink the vtable to less than 150 entries if necessary. I can't test this for real, but I've added some code to simulate an error: >>>>> >>>>> for (int i=0; i>>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>> if (num == bad >>>>> // || i > 120 /* uncomment this line to test */ >>>>> ) { >>>>> _info->set_vtab_size(i-1); >>>>> break; >>>>> } >>>>> dstvtab[i] = num; >>>>> } >>>>> >>>>> Results: >>>>> >>>>> + Removed 850 lines of CPU-dependent code >>>>> >>>>> + CDS image is about 50K smaller >>>>> >>>>> + Previously Metadata objects must live in the read-write section in the CDS >>>>> archive, because their _vptr was updated at run time. Now _vptr is no longer >>>>> updated, so ConstantPool can be moved to the read-only section (see JDK-8171392). >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >> > From ioi.lam at oracle.com Sun Mar 5 22:35:52 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 05 Mar 2017 14:35:52 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> Message-ID: <58BC92C8.40105@oracle.com> On 3/5/17 7:17 AM, coleen.phillimore at oracle.com wrote: > > Ioi, Some comments inline (where no comments, insert "ok") :) > > On 3/2/17 10:37 PM, Ioi Lam wrote: >> Hi Coleen, >> >> Thanks for the comments. I have updated the webrev. See in-line for >> responses. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >> >> >> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com wrote: >>> >>> Ioi >>> I like the concept of this a lot but have some stylistic comments to >>> help people reading this code later. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>> >>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't save many >>> characters, especially in CppVtableInfo/Testers >>> >> Done. >>> + // Start at slot 1, because slot 0 may be RTTI (on Solaris/Sparc) >>> + int i; >>> + for (i=1; ; i++) { >>> Since you're using 'i' later, can you rename it to something >>> descriptive. Or have another variable "vtable_length" to use >>> later. This looks like an old style for loop. >>> >> Done >>> Can the functions for CppVtableInfo be declared outside of the class >>> declaration? They don't need to be inline and then the debug code >>> for testing the vtable size can be not in the middle of the class >>> declaration. Then you can move the Tester classes to inside the >>> same #ifndef PRODUCT block. >>> >>> Can you put #endif // PRODUCT when the ifdef covers several lines of >>> code? >>> >> Done >>> vtab_of could be more descriptive, like cpp_vtable_for(). >>> >> I changed to vtable_of(). Because the class name is already >> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >> >>> Was PrintSharedSpaces was never converted to UL? >>> >> Right. I've filed https://bugs.openjdk.java.net/browse/JDK-8176132 >> (-XX:+PrintSharedSpaces should be converted to use Unified Logging.) >>> + int n = MAX_VTABLE_SIZE; >>> >>> Can you propagate MAX_VTABLE_SIZE to the places where it's used. n >>> isn't descriptive. This starts out with max_vtable_size and then >>> changes the size. Reusing 'n' makes this really hard to follow. >>> Not having a comment that we only allocate enough slots for the >>> vtable makes it hard too. >>> >>> + // allocate CppVtableInfo in the MD section >>> + _info = (CppVtabInfo*)md_top; >>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>> + >>> + // allocate temporary local instance of the metadata type T >>> + T tmp; >>> + intptr_t* srcvtab = vtab_of(tmp); >>> + intptr_t* dstvtab = _info->vtab(); >>> + >> Fixed. >>> Something like that for comments. dstvtab is the destination_vtable >>> in the MD section. >> >> I've dropped the md_ prefix from the functions that deal with the >> vtables, since they shouldn't care whether it's the "MD" section or >> not. Now it looks like this: >> >> // Allocate and initialize the C++ vtables, starting from top, but do >> not go past end. >> intptr_t* MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >> intptr_t* end) { >> assert(DumpSharedSpaces, "dump-time only"); >> // Layout (each slot is a intptr_t): >> // [number of slots in the first vtable = n1] >> // [ slots for the first vtable] >> // [number of slots in the first second = n2] >> // [ slots for the second vtable] >> // ... >> // The order of the vtables is the same as the >> CPP_VTAB_PATCH_TYPES_DO macro. >> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >> return top; >> } >> >>> + for (int i=0; i>> + const intptr_t bad = intptr_t(0xdeadbeef); >>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>> + if (num == bad >>> + // || i > 120 /* uncomment this line to test */ >>> + ) { >>> + _info->set_vtab_size(i-1); >>> + break; >>> + } >>> + dstvtab[i] = num; >>> + } >>> >>> I dont understand this code. You get deadbeef for a bad value if >>> SafeFetchN gets a fault but why would it get a fault at the end of >>> the metadata's vtable? Couldn't it just run onto the next vtable? >>> I think your original way of counting vtable entries might be better >>> (sorry I didn't have time to study that thread). >>> >> I've modified the comments to this. Does it make sense to you? >> >> // It is not always safe to call memcpy(), because srcvtable >> might be shorter than >> // MAX_VTABLE_SIZE, and the C++ linker might have placed the >> vtable at the very >> // end of the last page of libjvm.so. Crossing over to the next >> page might >> // cause a page fault. >> >> My fear is the JVM would suddenly start crashing because the order of >> .o files have changed on the linker's command line, or if you enable >> some special linker optimization flags. It's better safe than sorry. > > This wasn't exactly what I was not understanding. I didn't see that > you are copying 120 entries from the old vtable and junk memory beyond > the old vtable, unless you get a segv, in which case you copy less. > I don't think you should copy random memory into the vtable in the > archive. This doesn't seem secure, even with the segv protection. > > Since we already have assumptions about C++ vtable layout in the code > and it's mostly specified by various ABIs, and you have the assert > code, I think I would prefer that you copy only the vtable entries > into the archive. I guess Thomas Stuefe had a different opinion. > I've read the original thread. Two points: > > If new C++ compiler implementations add a discontigous vtable, both > the SafeFetchN and subclass additional virtual function at end > implementation will fail. I don't think C++ implementations would do > this and a contiguous vtable as first in the instance has been > standard for years. If our metadata adds multiple inheritance, the > same issue would be a problem for both implementations, as well as for > the implementation we have before Ioi's fix. > > Ioi's subclass adding virtual function method would work for any > esoteric C++ implementations in my memory, except the vptr for the old > DECC++ compiler was after the nonstatic data members (which would fail > with all of our implementations). > > Since the code is there anyway for debug purposes, we're not saving > code by implementing SafeFetchN. The SafeFetchN implementation isn't > obvious at all what it's doing, and requires better comments, > especially if you don't know already what SafeFetchN does. It looks > really cryptic. The poisoned values also bothered me in that they > overload other poisoned values in other parts of the jvm. > I can go with either implementation, although I like the original one that doesn't use SafeFetch. > Ioi, could you make all methods of CppVtableCloner out of line? > Is it for debugging purposes? I am using a recent version of gdb and I have no problems setting break points or stepping into the code. I can move the bigger methods out, but for clarity I think it's better to leave the small methods inside the class declaration. Thanks - Ioi > The other changes look good, although I might have more requests for > comments. > > Thanks, > Coleen > >>> Would be nice to have comments here too!! >>> >>> + intptr_t* start = md_top; >>> >>> This doesn't do anything (?) >> >> Fixed. This was left over code. >>> >>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>> >>> Why not zero the destination vtable in allocate? Or does patching >>> the vtable pointers call virtual functions? You could prevent that >>> so you don't need this code. >>> >> I added this comment: >> >> // During patching, some virtual methods may be called, so at this >> point >> // the vtables must contain valid methods (as filled in by >> CppVtableCloner::allocate). >> MetaspaceShared::patch_cpp_vtable_pointers(); >> >> // The vtable clones contain addresses of the current process. >> // We don't want to write these addresses into the archive. >> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >> >>> + // Restore the vtable in case we invoke any virtual methods. >>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>> Can this be restore_cpp_vtables since that's what it's doing. The >>> first is after the dump and the second call is at UseSharedSpaces. >>> A couple of comments in this clone_cpp_vtables --> >>> restore_cpp_vtables would be nice. eg: >>> >> I prefer to use the word clone. Otherwise when you just say "vtable" >> it's not clear whether you're talking about the original one (made by >> the c++ linker), or the cloned one in the CDS archive. >>> + static intptr_t* clone_vtable(const char* name, intptr_t* p) { >>> + T tmp; // Allocate temporary dummy metadata object to get vtable initialized >>> + CppVtabInfo* info = (CppVtabInfo*)p; >>> + int n = info->vtab_size(); >>> + intptr_t* srcvtab = vtab_of(tmp); >>> + intptr_t* dstvtab = info->vtab(); >>> + >>> + // We already checked (and, if necessary, adjusted n) when the vtables were allocated, so we are >>> + // safe to do memcpy. >>> + if (PrintSharedSpaces) { >>> + tty->print_cr("%s copying %d vtable entries", name, n); >>> + } >>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>> + return dstvtab + n; >>> + } >>> >> Done. I changed the wording >> >> T tmp; // Allocate temporary dummy metadata object to get to the >> original vtable. >> >> As we are not really "initializing a vtable" here. >> >>> Same with 'patch'. It'd be so much faster and easier to read this >>> code with more comments please. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>> >>> Why are these testers here? >>> >> >> I updated the comment: >> >> // Used by CDS. These classes need to access the private >> ConstantPool() constructor. >> template friend class CppVtableTesterA; >> template friend class CppVtableTesterB; >> template friend class CppVtableCloner; >> >> >> Thanks >> - Ioi >> >>>> >>>>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>>>> >>>>>> Hi, >>>>>> >>>>>> This is the official review (follow up of the "Determining the size of C++ vtables" thread onhotspot-dev at openjdk.java.net). >>>>>> >>>>>> The new code has the same assumption as the existing code in JDK 10: for a C++ object that contains virtual methods (e.g., ConstantPool), we assume the first intptr_t slot of the object is a _vptr, which points to a vtable, which consists of no more than 150 intptr_t's. >>>>>> >>>>>> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >>>>>> [ field #0 ] [ vtable slot 1 ] >>>>>> [ field #1 ] [ vtable slot 2 ] >>>>>> [ field #2 ] [ vtable slot 3 ] >>>>>> [ .... ] [ vtable slot 4] >>>>>> [ vtable slot 5 ] >>>>>> [ ... ] >>>>>> >>>>>> + In the existing code, we were pointing the vtable slots to >>>>>> code that's generated by HotSpot. >>>>>> >>>>>> + In the new code, we copy the vtable slots from an existing >>>>>> vtable (generated by the C++ linker). >>>>>> >>>>>> Per Thomas St?fe's advice, I don't try to determine the size of the vtable (as that would add one more compiler requirement where new virtual methods added by a subclass must be placed at a higher offset in the vtable). >>>>>> >>>>>> Instead, I have added code in non-product builds to ensure that the vtables are no longer than 150 entries. You can run with "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size of the vtables for your particular platform: >>>>>> >>>>>> ConstantPool has 12 virtual methods >>>>>> InstanceKlass has 113 virtual methods >>>>>> InstanceClassLoaderKlass has 113 virtual methods >>>>>> InstanceMirrorKlass has 113 virtual methods >>>>>> InstanceRefKlass has 113 virtual methods >>>>>> Method has 12 virtual methods >>>>>> ObjArrayKlass has 114 virtual methods >>>>>> TypeArrayKlass has 114 virtual methods >>>>>> >>>>>> As mentioned in the code comments, if you have an esoteric C++ compiler, the verify_sufficient_size() function will probably fail, but hopefully that would give you some hints for porting this code. >>>>>> >>>>>> To avoid accidentally touching an unmapped page, the code uses SafeFetchN for copying the vtable contents, and would shrink the vtable to less than 150 entries if necessary. I can't test this for real, but I've added some code to simulate an error: >>>>>> >>>>>> for (int i=0; i>>>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>> if (num == bad >>>>>> // || i > 120 /* uncomment this line to test */ >>>>>> ) { >>>>>> _info->set_vtab_size(i-1); >>>>>> break; >>>>>> } >>>>>> dstvtab[i] = num; >>>>>> } >>>>>> >>>>>> Results: >>>>>> >>>>>> + Removed 850 lines of CPU-dependent code >>>>>> >>>>>> + CDS image is about 50K smaller >>>>>> >>>>>> + Previously Metadata objects must live in the read-write section in the CDS >>>>>> archive, because their _vptr was updated at run time. Now _vptr is no longer >>>>>> updated, so ConstantPool can be moved to the read-only section (see JDK-8171392). >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> >>> >> > From david.holmes at oracle.com Mon Mar 6 01:09:47 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 6 Mar 2017 11:09:47 +1000 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> Message-ID: <1d2586eb-0257-a330-2ff6-5028de510b4b@oracle.com> On 3/03/2017 6:59 PM, Zhu Yong wrote: > Thank you very much for helping. > > Today I tried to debug the test code, but without luck, still can't > figure out why. > > First, I followed the instruction > http://stackoverflow.com/questions/29916995/intellij-idea-remotely-debug-java-console-program, > changed suspend=y because the test code is short, it will exit before > debug start if not suspend first. Unfortunately, with remote debug, I > was not able to reproduce the issue. Same class file, directly run, will > produce the issue, remote debug won't. > > Then, I tried *jdb *from my OpenJDK9 client vm, run directly on my > target system to debug the code. If I set breakpoint, program run > without issue, however, if don't set breakpoint, just "run" in jdb, > problem reproduced. This suggests to me that a floating-point register is not being preserved when needed. When you run with the breakpoint you will get full save/restore of all the register state. Just a theory. > Next, I tried to use *javap *to decompile the output class file, run > OpenJDK9 client vm javap and host side javap on same class file, the > output is same for getTime function. It is a native code issue not a bytecode issue, so javap wont help here. > One interesting point I noticed today is the output like below when > problem happen, 2nd call of getTime() has value of 1st call result / > 1000. Looks like 2nd call getTime(), double variable q was not updated > startTime: 284092.063000 endTime: 284.092063 runTime: -283807.970937 > > What should I do so that I can come a conclusion that it's due to > compiler issue? Can you get access to a different gcc to build with? David ----- > > On Thu, Mar 2, 2017 at 12:32 PM, David Holmes > wrote: > > I built your sample program with latest JDK 9 for arm-vfp-sflt and > ran on an emulator I have available, and I had no issues. > > So I still have concerns this may be a build/compiler issue. > > David > > > On 2/03/2017 12:51 PM, David Holmes wrote: > > I've moved this discussion over to hotspot-dev as this seems no > longer a > build issue but a C1 soft-float issue. > > David > > On 2/03/2017 12:35 PM, Zhu Yong wrote: > > If run with -Xint, it works. > > I have created simplified FP test by remove all non-related > code from > Whetstone test code. > The test code is here: > https://gist.github.com/yongzhy/d65c26d39fe5d549d1b406c7c84eacd4 > > > I am not sure if the test code can help or not. The > behaviour is weird : > - If I print both itime and q, program will run OK > - inside condition block if(q<1.0f), if I use exit code > 2,3,4,5, problem > appears, however, if I use exit code >=6, program run OK. > > I always get exit code 5, the line q = (double)itime is wrong? > > public static double getTime() > { > double q; > long itime; > itime = System.currentTimeMillis(); > if(itime<1000000) { > System.exit(1); > } > //System.out.printf("time long value %d\n", itime); > q = (double) itime; > if(q < 1.0f) { > System.exit(5); // I always get exit code 5 > } > //System.out.printf("time float value %f\n", q); > return q / 1000.0; > } > > > > On Wed, Mar 1, 2017 at 5:31 PM, David Holmes > > >> wrote: > > On 1/03/2017 6:45 PM, Zhu Yong wrote: > > OK, for the Whetstone Benchmark, I added debug > printing and > found it's > due to function getTime(), it only get good value on > 1st call, > all > following calls get 0s. > Debug printing of itime value is correct. So reason > should be > the return > division line. Could it because toolchain's floating > point > operation??? > But why server VM Ok? > > > Server and client implement FP logic differently, and > particularly > as this is soft-fp, they may well not be in sync. Does > -Xint work? > > Can you isolate to a simple FP test? > > David > > public static double getTime() > { > double q; > long itime; > itime = System.currentTimeMillis(); > q = (double) itime; > return q / 1000.0; > } > > On Wed, Mar 1, 2017 at 3:14 PM, David Holmes > > > > > >>> wrote: > > On 1/03/2017 4:26 PM, Zhu Yong wrote: > > We use armv7-marvell-linux-gnueabi-softfp > toolchain > > gcc version 4.6.4 (Marvell GCC release > 20150204-c4af733b 64K > MAXPAGESIZE > ALIGN CVE-2015-0235) > > Is this toolchain too old? I am asking > because I saw > other strange > issues as well: > > > We last used gcc 4.7.2. I can't really say if > 4.6.4 is "too > old". > > I have successfully build server, client and > minimal VM, > when I run > Dhrystone benchmark file (the same jar file from > http://www.okayan.jp/DhrystoneApplet/ > > > > > >>), the performance from > server VM > is much better than client and minimal VM. > (minimal: 1629852, client: 1615508, server: > 2338871 ) > > > That's expected. The server VM is high performance. > > And when run Whetstone Benchmark > from > > http://www.roylongbottom.org.uk/online/whetjava2.html > > > > > > > > >>, > server VM > finished with good result, client and > minimal VM not > able to finish > (program stuck there forever after print 1st > line of > output). > > > That needs investigating. You need to try and > generate a > stackdump > to see where things are stuck. > > David > > > On Wed, Mar 1, 2017 at 1:56 PM, David Holmes > > > > > >> > > > > > >>>> wrote: > > On 1/03/2017 3:39 PM, Zhu Yong wrote: > > Thank you for the information, I > managed to make > minimal > build > pass now. > > Initially I though JVM_EXCLUDE_FILES > need to add > additional 3 > generated > files (they appeared > in _BUILD_LIBJVM_objectfilenames.txt) : > bytecodeInterpreterWithChecks.cpp > jvmtiEnter.cpp > jvmtiEnterTrace.cpp > But build still fail with same error > message. > > Finally I figured out it's due to > those 8 doit() > functions > implementation in jvmtiEnvBase.hpp > file. After > move all > those 8 > doit() > implementations to jvmtiEnvBase.cpp, > build of > minimal VM > passed > without > any issue. > > > That seems to be a compiler issue. > jvmtiEnvBase.hpp is > unconditionally included in a couple of > places > because we > have to > have enough of the JVMTI code to say > JVMTI is not > available. > Those > doit() implementations, though arguably > could be > conditional on > INCLUDE_JVMTI, are not referenced by any > called code > and so > should > not linked in. But in your case it seems > they are. > > What toolchain are you using? > > David > ----- > > Changes appended > ============= > > --- > .../src/share/vm/prims/jvmtiEnvBase.cpp > | 74 > ++++++++++++++++++++++ > .../src/share/vm/prims/jvmtiEnvBase.hpp > | 68 > +++----------------- > 2 files changed, 82 insertions(+), 60 > deletions(-) > mode change 100644 => 100755 > > hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > mode change 100644 => 100755 > > hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > > diff --git > > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > old mode 100644 > new mode 100755 > index dd241a0..e5832ba > --- > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > +++ > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > @@ -1283,6 +1283,80 @@ > > VM_GetMultipleStackTraces::allocate_and_fill_stacks(jint > thread_count) { > "the last copied frame > info must be > the last > record"); > } > > +void > +VM_UpdateForPopTopFrame::doit() { > + JavaThread* jt = > _state->get_thread(); > + if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() != > NULL) { > + _state->update_for_pop_top_frame(); > + } else { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + } > +} > + > +void > +VM_SetFramePop::doit() { > + JavaThread* jt = > _state->get_thread(); > + if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() != > NULL) { > + int frame_number = > _state->count_frames() - > _depth; > + > > > > _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_pop(frame_number); > > + } else { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + } > +} > + > +void > +VM_GetOwnedMonitorInfo::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + if > (Threads::includes(_java_thread) && > !_java_thread->is_exiting() > + > && > _java_thread->threadObj() != > NULL) { > + _result = ((JvmtiEnvBase > > *)_env)->get_owned_monitors(_calling_thread, > _java_thread, > + > _owned_monitors_list); > + } > +} > + > +void > +VM_GetObjectMonitorUsage::doit() { > + _result = ((JvmtiEnvBase*) > > _env)->get_object_monitor_usage(_calling_thread, > _object, > _info_ptr); > +} > + > +void > +VM_GetCurrentContendedMonitor::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + if > (Threads::includes(_java_thread) && > !_java_thread->is_exiting() && > + _java_thread->threadObj() != > NULL) { > + _result = ((JvmtiEnvBase > > > > *)_env)->get_current_contended_monitor(_calling_thread,_java_thread,_owned_monitor_ptr); > > + } > +} > + > +void > +VM_GetStackTrace::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + if > (Threads::includes(_java_thread) && > !_java_thread->is_exiting() > + > && > _java_thread->threadObj() != > NULL) { > + _result = ((JvmtiEnvBase > *)_env)->get_stack_trace(_java_thread, > + > _start_depth, > _max_count, > + > _frame_buffer, > _count_ptr); > + } > +} > + > +void > +VM_GetFrameCount::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + JavaThread* jt = > _state->get_thread(); > + if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() != > NULL) { > + _result = > ((JvmtiEnvBase*)_env)->get_frame_count(_state, > _count_ptr); > + } > +} > + > +void > +VM_GetFrameLocation::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + if > (Threads::includes(_java_thread) && > !_java_thread->is_exiting() && > + _java_thread->threadObj() != > NULL) { > + _result = > > ((JvmtiEnvBase*)_env)->get_frame_location(_java_thread, > _depth, > + > _method_ptr, > _location_ptr); > + } > +} > > void > VM_GetThreadListStackTraces::doit() { > diff --git > > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > old mode 100644 > new mode 100755 > index 04e6869..00b9890 > --- > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > +++ > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > @@ -359,14 +359,7 @@ public: > } > VMOp_Type type() const { return > VMOp_UpdateForPopTopFrame; } > jvmtiError result() { return > _result; } > - void doit() { > - JavaThread* jt = > _state->get_thread(); > - if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - > _state->update_for_pop_top_frame(); > - } else { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - } > - } > + void doit(); > }; > > // VM operation to set frame pop. > @@ -389,15 +382,7 @@ public: > bool allow_nested_vm_operations() > const { > return true; } > VMOp_Type type() const { return > VMOp_SetFramePop; } > jvmtiError result() { return > _result; } > - void doit() { > - JavaThread* jt = > _state->get_thread(); > - if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - int frame_number = > _state->count_frames() > - _depth; > - > > > > _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_pop(frame_number); > > - } else { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - } > - } > + void doit(); > }; > > > @@ -421,14 +406,7 @@ public: > _result = JVMTI_ERROR_NONE; > } > VMOp_Type type() const { return > VMOp_GetOwnedMonitorInfo; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - if > (Threads::includes(_java_thread) && > !_java_thread->is_exiting() > - > && > _java_thread->threadObj() != > NULL) { > - _result = ((JvmtiEnvBase > > *)_env)->get_owned_monitors(_calling_thread, > _java_thread, > - > _owned_monitors_list); > - } > - } > + void doit(); > jvmtiError result() { return > _result; } > }; > > @@ -451,10 +429,7 @@ public: > } > VMOp_Type type() const { return > VMOp_GetObjectMonitorUsage; } > jvmtiError result() { return > _result; } > - void doit() { > - _result = ((JvmtiEnvBase*) > > _env)->get_object_monitor_usage(_calling_thread, > _object, > _info_ptr); > - } > - > + void doit(); > }; > > // VM operation to get current > contended > monitor. > @@ -475,13 +450,7 @@ public: > } > VMOp_Type type() const { return > VMOp_GetCurrentContendedMonitor; } > jvmtiError result() { return > _result; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - if > (Threads::includes(_java_thread) && > !_java_thread->is_exiting() && > - _java_thread->threadObj() > != NULL) { > - _result = ((JvmtiEnvBase > > > > *)_env)->get_current_contended_monitor(_calling_thread,_java_thread,_owned_monitor_ptr); > > - } > - } > + void doit(); > }; > > // VM operation to get stack trace > at safepoint. > @@ -508,15 +477,7 @@ public: > } > jvmtiError result() { return > _result; } > VMOp_Type type() const { return > VMOp_GetStackTrace; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - if > (Threads::includes(_java_thread) && > !_java_thread->is_exiting() > - > && > _java_thread->threadObj() != > NULL) { > - _result = ((JvmtiEnvBase > *)_env)->get_stack_trace(_java_thread, > - > _start_depth, > _max_count, > - > _frame_buffer, > _count_ptr); > - } > - } > + void doit(); > }; > > // forward declaration > @@ -606,13 +567,7 @@ public: > } > VMOp_Type type() const { return > VMOp_GetFrameCount; } > jvmtiError result() { return > _result; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - JavaThread* jt = > _state->get_thread(); > - if (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - _result = > ((JvmtiEnvBase*)_env)->get_frame_count(_state, > _count_ptr); > - } > - } > + void doit(); > }; > > // VM operation to frame location > at safepoint. > @@ -636,14 +591,7 @@ public: > } > VMOp_Type type() const { return > VMOp_GetFrameLocation; } > jvmtiError result() { return > _result; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - if > (Threads::includes(_java_thread) && > !_java_thread->is_exiting() && > - _java_thread->threadObj() > != NULL) { > - _result = > > ((JvmtiEnvBase*)_env)->get_frame_location(_java_thread, > _depth, > - > _method_ptr, > _location_ptr); > - } > - } > + void doit(); > }; > > > -- > 2.1.4 > > > On Tue, Feb 28, 2017 at 6:11 PM, > David Holmes > > > > > >> > > > > > >>> > > > > > >> > > > > > > >>>>> wrote: > > On 28/02/2017 7:35 PM, David > Holmes wrote: > > On 28/02/2017 6:51 PM, Zhu > Yong wrote: > > Dear All, > > I am testing cross compile > OpenJDK-9+158 for our > embedded > system using > buildroot, I can build > server and > client > variants > successfully, but the > output compact1 profile > file size > is too > big, then I > tried > to build > minimal > variant, failed when linking > libjvm.so. > > I checked > > > > > build/linux-arm-normal-minimal-release/hotspot/variant-minimal/ > > libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt, > jvmtiEnvBase.o > and > jvmtiEnvThreadState.o are not > listed in > the file > (which > is required > from the error message > below). > > > Right - JVM TI is not part > of the > Minimal VM. At the > moment it looks > like some service has either > been > enabled when > it should not > have been, > or has not been correctly > if'def in the > source. > > > As far as I can see your error > messages are > complaining about > missing functions that are > called from the > same > sources as the > functions that are missing! ie. > > undefined reference to > > > `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, > JavaThread*, > _jobject**)' > /tmp/cc27HS5M.ltrans0.ltrans.o: > In function > `VM_GetOwnedMonitorInfo::doit() > > but VM_GetOwnedMonitorInfo is > defined in > jvmtiEnv.cpp which > should > be included or excluded the same as > jvmtiEnBase.cpp. > Here's the > logic in the makefile that > controls this: > > ifneq ($(call check-jvm-feature, > jvmti), > true) > JVM_CFLAGS_FEATURES += > -DINCLUDE_JVMTI=0 > JVM_EXCLUDE_FILES += > jvmtiGetLoadedClasses.cpp > jvmtiThreadState.cpp > jvmtiExtensions.cpp \ > jvmtiImpl.cpp > jvmtiManageCapabilities.cpp > jvmtiRawMonitor.cpp > jvmtiUtil.cpp jvmtiTrace.cpp \ > jvmtiCodeBlobEvents.cpp > jvmtiEnv.cpp > jvmtiRedefineClasses.cpp > jvmtiEnvBase.cpp > jvmtiEnvThreadState.cpp \ > jvmtiTagMap.cpp > jvmtiEventController.cpp > evmCompat.cpp > jvmtiEnter.xsl jvmtiExport.cpp \ > > jvmtiClassFileReconstituter.cpp > endif > > Can you run with LOG=trace so > that each > compiled file is > listed in > the build log, then see if there > are any > jvmti* > files listed > there. > > Thanks, > David > > > > > Can you provide the lines > from your > spec.gmk > that define the > JVM_FEATURES_* variables please. > > Thanks, > David > ------ > > > === Output from failing > command(s) > repeated > here === > * For target > > > hotspot_variant-minimal_libjvm_objs_BUILD_LIBJVM_link: > > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_GetStackTrace::doit() > [clone .local.640]': > > cc27HS5M.ltrans0.o:(.text+0xce5e): > undefined > reference to > > `JvmtiEnvBase::get_stack_trace(JavaThread*, > int, int, > _jvmtiFrameInfo*, > int*)' > > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > > `VM_GetCurrentContendedMonitor::doit() > [clone .local.639]': > > cc27HS5M.ltrans0.o:(.text+0xcec2): > undefined > reference to > > > > `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, > JavaThread*, > _jobject**)' > > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > > `VM_GetOwnedMonitorInfo::doit() > [clone .local.638]': > > cc27HS5M.ltrans0.o:(.text+0xcf26): > undefined > reference to > > `JvmtiEnvBase::get_owned_monitors(JavaThread*, > JavaThread*, > GrowableArray<_ > > jvmtiMonitorStackDepthInfo*>*)' > > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_GetFrameCount::doit() > [clone .local.637]': > > cc27HS5M.ltrans0.o:(.text+0xcf8a): > undefined > reference to > > > `JvmtiEnvBase::get_frame_count(JvmtiThreadState*, int*)' > > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_SetFramePop::doit() > [clone > .local.636]': > > cc27HS5M.ltrans0.o:(.text+0xcfea): > undefined > reference to > > `JvmtiThreadState::count_frames()' > > cc27HS5M.ltrans0.o:(.text+0xd030): > undefined > reference to > > `JvmtiEnvThreadState::set_frame_pop(int)' > > /tmp/cc27HS5M.ltrans0.ltrans.o: In > function > `VM_GetFrameLocation::doit() > [clone .local.641]': > ... (rest of output > omitted) > > > My configuration: > > --with-jdk-variant=normal \ > > --with-jvm-variants=minimal \ > --with-debug-level=release \ > > --disable-warnings-as-errors \ > --disable-hotspot-gtest \ > > --with-abi-profile=arm-vfp-sflt \ > > --openjdk-target=$(GNU_TARGET_NAME) \ > > --with-sys-root=$(STAGING_DIR) \ > > --with-tools-dir=$(HOST_DIR) \ > > > --with-freetype-include=$(STAGING_DIR)/usr/include \ > > --with-freetype-lib=$(STAGING_DIR)/usr/lib \ > > --with-freetype=$(STAGING_DIR)/usr/ \ > > --with-alsa-include=$(STAGING_DIR)/usr/include \ > > --with-alsa-lib=$(STAGING_DIR)/usr/lib \ > > --with-alsa=$(STAGING_DIR)/usr/ \ > > --with-cups=$(STAGING_DIR)/usr/ \ > > --with-x=$(STAGING_DIR)/usr/include \ > > --with-extra-ldflags=--sysroot=$(STAGING_DIR) \ > --enable-headless-only \ > > --disable-freetype-bundling \ > --enable-unlimited-crypto \ > > --with-boot-jdk=/opt/java/jdk1.8.0_102 > > > > > > From zhuyong.me at gmail.com Mon Mar 6 04:30:12 2017 From: zhuyong.me at gmail.com (Zhu Yong) Date: Mon, 6 Mar 2017 12:30:12 +0800 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: <1d2586eb-0257-a330-2ff6-5028de510b4b@oracle.com> References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> <1d2586eb-0257-a330-2ff6-5028de510b4b@oracle.com> Message-ID: Sometime later this week, I might get newer version of Marvell toolchain, I will do a build and test again then. I will update this thread when I get result. Thank you very much for helping. On Mon, Mar 6, 2017 at 9:09 AM, David Holmes wrote: > On 3/03/2017 6:59 PM, Zhu Yong wrote: > >> Thank you very much for helping. >> >> Today I tried to debug the test code, but without luck, still can't >> figure out why. >> >> First, I followed the instruction >> http://stackoverflow.com/questions/29916995/intellij-idea- >> remotely-debug-java-console-program, >> changed suspend=y because the test code is short, it will exit before >> debug start if not suspend first. Unfortunately, with remote debug, I >> was not able to reproduce the issue. Same class file, directly run, will >> produce the issue, remote debug won't. >> >> Then, I tried *jdb *from my OpenJDK9 client vm, run directly on my >> target system to debug the code. If I set breakpoint, program run >> without issue, however, if don't set breakpoint, just "run" in jdb, >> problem reproduced. >> > > This suggests to me that a floating-point register is not being preserved > when needed. When you run with the breakpoint you will get full > save/restore of all the register state. Just a theory. > > Next, I tried to use *javap *to decompile the output class file, run >> OpenJDK9 client vm javap and host side javap on same class file, the >> output is same for getTime function. >> > > It is a native code issue not a bytecode issue, so javap wont help here. > > One interesting point I noticed today is the output like below when >> problem happen, 2nd call of getTime() has value of 1st call result / >> 1000. Looks like 2nd call getTime(), double variable q was not updated >> startTime: 284092.063000 endTime: 284.092063 runTime: -283807.970937 >> >> What should I do so that I can come a conclusion that it's due to >> compiler issue? >> > > Can you get access to a different gcc to build with? > > David > ----- > > >> On Thu, Mar 2, 2017 at 12:32 PM, David Holmes > > wrote: >> >> I built your sample program with latest JDK 9 for arm-vfp-sflt and >> ran on an emulator I have available, and I had no issues. >> >> So I still have concerns this may be a build/compiler issue. >> >> David >> >> >> On 2/03/2017 12:51 PM, David Holmes wrote: >> >> I've moved this discussion over to hotspot-dev as this seems no >> longer a >> build issue but a C1 soft-float issue. >> >> David >> >> On 2/03/2017 12:35 PM, Zhu Yong wrote: >> >> If run with -Xint, it works. >> >> I have created simplified FP test by remove all non-related >> code from >> Whetstone test code. >> The test code is here: >> https://gist.github.com/yongzhy/d65c26d39fe5d549d1b406c7c84e >> acd4 >> > eacd4> >> >> I am not sure if the test code can help or not. The >> behaviour is weird : >> - If I print both itime and q, program will run OK >> - inside condition block if(q<1.0f), if I use exit code >> 2,3,4,5, problem >> appears, however, if I use exit code >=6, program run OK. >> >> I always get exit code 5, the line q = (double)itime is wrong? >> >> public static double getTime() >> { >> double q; >> long itime; >> itime = System.currentTimeMillis(); >> if(itime<1000000) { >> System.exit(1); >> } >> //System.out.printf("time long value %d\n", >> itime); >> q = (double) itime; >> if(q < 1.0f) { >> System.exit(5); // I always get exit code 5 >> } >> //System.out.printf("time float value %f\n", q); >> return q / 1000.0; >> } >> >> >> >> On Wed, Mar 1, 2017 at 5:31 PM, David Holmes >> >> > >> wrote: >> >> On 1/03/2017 6:45 PM, Zhu Yong wrote: >> >> OK, for the Whetstone Benchmark, I added debug >> printing and >> found it's >> due to function getTime(), it only get good value on >> 1st call, >> all >> following calls get 0s. >> Debug printing of itime value is correct. So reason >> should be >> the return >> division line. Could it because toolchain's floating >> point >> operation??? >> But why server VM Ok? >> >> >> Server and client implement FP logic differently, and >> particularly >> as this is soft-fp, they may well not be in sync. Does >> -Xint work? >> >> Can you isolate to a simple FP test? >> >> David >> >> public static double getTime() >> { >> double q; >> long itime; >> itime = System.currentTimeMillis(); >> q = (double) itime; >> return q / 1000.0; >> } >> >> On Wed, Mar 1, 2017 at 3:14 PM, David Holmes >> > >> > > >> > >> > >>> wrote: >> >> On 1/03/2017 4:26 PM, Zhu Yong wrote: >> >> We use armv7-marvell-linux-gnueabi-softfp >> toolchain >> >> gcc version 4.6.4 (Marvell GCC release >> 20150204-c4af733b 64K >> MAXPAGESIZE >> ALIGN CVE-2015-0235) >> >> Is this toolchain too old? I am asking >> because I saw >> other strange >> issues as well: >> >> >> We last used gcc 4.7.2. I can't really say if >> 4.6.4 is "too >> old". >> >> I have successfully build server, client and >> minimal VM, >> when I run >> Dhrystone benchmark file (the same jar file >> from >> http://www.okayan.jp/DhrystoneApplet/ >> >> > > >> > >> > >>), the performance >> from >> server VM >> is much better than client and minimal VM. >> (minimal: 1629852, client: 1615508, server: >> 2338871 ) >> >> >> That's expected. The server VM is high >> performance. >> >> And when run Whetstone Benchmark >> from >> >> http://www.roylongbottom.org.uk/online/whetjava2.html >> >> >> > > >> >> > >> >> > >>, >> server VM >> finished with good result, client and >> minimal VM not >> able to finish >> (program stuck there forever after print 1st >> line of >> output). >> >> >> That needs investigating. You need to try and >> generate a >> stackdump >> to see where things are stuck. >> >> David >> >> >> On Wed, Mar 1, 2017 at 1:56 PM, David Holmes >> > >> > > >> > david.holmes at oracle.com> >> > >> >> > >> > > >> > >> > >>>> wrote: >> >> On 1/03/2017 3:39 PM, Zhu Yong wrote: >> >> Thank you for the information, I >> managed to make >> minimal >> build >> pass now. >> >> Initially I though JVM_EXCLUDE_FILES >> need to add >> additional 3 >> generated >> files (they appeared >> in _BUILD_LIBJVM_objectfilenames.txt) >> : >> bytecodeInterpreterWithChecks.cpp >> jvmtiEnter.cpp >> jvmtiEnterTrace.cpp >> But build still fail with same error >> message. >> >> Finally I figured out it's due to >> those 8 doit() >> functions >> implementation in jvmtiEnvBase.hpp >> file. After >> move all >> those 8 >> doit() >> implementations to jvmtiEnvBase.cpp, >> build of >> minimal VM >> passed >> without >> any issue. >> >> >> That seems to be a compiler issue. >> jvmtiEnvBase.hpp is >> unconditionally included in a couple of >> places >> because we >> have to >> have enough of the JVMTI code to say >> JVMTI is not >> available. >> Those >> doit() implementations, though arguably >> could be >> conditional on >> INCLUDE_JVMTI, are not referenced by any >> called code >> and so >> should >> not linked in. But in your case it seems >> they are. >> >> What toolchain are you using? >> >> David >> ----- >> >> Changes appended >> ============= >> >> --- >> .../src/share/vm/prims/jvmtiE >> nvBase.cpp >> | 74 >> ++++++++++++++++++++++ >> .../src/share/vm/prims/jvmtiE >> nvBase.hpp >> | 68 >> +++----------------- >> 2 files changed, 82 insertions(+), 60 >> deletions(-) >> mode change 100644 => 100755 >> >> hotspot-9211c2e89c1c/src/share >> /vm/prims/jvmtiEnvBase.cpp >> mode change 100644 => 100755 >> >> hotspot-9211c2e89c1c/src/share >> /vm/prims/jvmtiEnvBase.hpp >> >> diff --git >> >> >> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> >> >> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> old mode 100644 >> new mode 100755 >> index dd241a0..e5832ba >> --- >> >> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> +++ >> >> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >> @@ -1283,6 +1283,80 @@ >> >> VM_GetMultipleStackTraces::all >> ocate_and_fill_stacks(jint >> thread_count) { >> "the last copied frame >> info must be >> the last >> record"); >> } >> >> +void >> +VM_UpdateForPopTopFrame::doit() { >> + JavaThread* jt = >> _state->get_thread(); >> + if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + _state->update_for_pop_top_fra >> me(); >> + } else { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + } >> +} >> + >> +void >> +VM_SetFramePop::doit() { >> + JavaThread* jt = >> _state->get_thread(); >> + if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + int frame_number = >> _state->count_frames() - >> _depth; >> + >> >> >> >> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ >> pop(frame_number); >> >> + } else { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + } >> +} >> + >> +void >> +VM_GetOwnedMonitorInfo::doit() { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if >> (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() >> + >> && >> _java_thread->threadObj() != >> NULL) { >> + _result = ((JvmtiEnvBase >> >> *)_env)->get_owned_monitors(_calling_thread, >> _java_thread, >> + >> _owned_monitors_list); >> + } >> +} >> + >> +void >> +VM_GetObjectMonitorUsage::doit() { >> + _result = ((JvmtiEnvBase*) >> >> _env)->get_object_monitor_usage(_calling_thread, >> _object, >> _info_ptr); >> +} >> + >> +void >> +VM_GetCurrentContendedMonitor::doit() >> { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if >> (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() && >> + _java_thread->threadObj() != >> NULL) { >> + _result = ((JvmtiEnvBase >> >> >> >> *)_env)->get_current_contended_monitor(_calling_thread,_ >> java_thread,_owned_monitor_ptr); >> >> + } >> +} >> + >> +void >> +VM_GetStackTrace::doit() { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if >> (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() >> + >> && >> _java_thread->threadObj() != >> NULL) { >> + _result = ((JvmtiEnvBase >> *)_env)->get_stack_trace(_java_thread, >> + >> _start_depth, >> _max_count, >> + >> _frame_buffer, >> _count_ptr); >> + } >> +} >> + >> +void >> +VM_GetFrameCount::doit() { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + JavaThread* jt = >> _state->get_thread(); >> + if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + _result = >> ((JvmtiEnvBase*)_env)->get_fra >> me_count(_state, >> _count_ptr); >> + } >> +} >> + >> +void >> +VM_GetFrameLocation::doit() { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if >> (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() && >> + _java_thread->threadObj() != >> NULL) { >> + _result = >> >> ((JvmtiEnvBase*)_env)->get_fra >> me_location(_java_thread, >> _depth, >> + >> _method_ptr, >> _location_ptr); >> + } >> +} >> >> void >> VM_GetThreadListStackTraces::doit() >> { >> diff --git >> >> >> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> >> >> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> old mode 100644 >> new mode 100755 >> index 04e6869..00b9890 >> --- >> >> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> +++ >> >> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >> @@ -359,14 +359,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_UpdateForPopTopFrame; } >> jvmtiError result() { return >> _result; } >> - void doit() { >> - JavaThread* jt = >> _state->get_thread(); >> - if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() >> != NULL) { >> - >> _state->update_for_pop_top_frame(); >> - } else { >> - _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> - } >> - } >> + void doit(); >> }; >> >> // VM operation to set frame pop. >> @@ -389,15 +382,7 @@ public: >> bool allow_nested_vm_operations() >> const { >> return true; } >> VMOp_Type type() const { return >> VMOp_SetFramePop; } >> jvmtiError result() { return >> _result; } >> - void doit() { >> - JavaThread* jt = >> _state->get_thread(); >> - if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() >> != NULL) { >> - int frame_number = >> _state->count_frames() >> - _depth; >> - >> >> >> >> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ >> pop(frame_number); >> >> - } else { >> - _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> - } >> - } >> + void doit(); >> }; >> >> >> @@ -421,14 +406,7 @@ public: >> _result = JVMTI_ERROR_NONE; >> } >> VMOp_Type type() const { return >> VMOp_GetOwnedMonitorInfo; } >> - void doit() { >> - _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> - if >> (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() >> - >> && >> _java_thread->threadObj() != >> NULL) { >> - _result = ((JvmtiEnvBase >> >> *)_env)->get_owned_monitors(_calling_thread, >> _java_thread, >> - >> _owned_monitors_list); >> - } >> - } >> + void doit(); >> jvmtiError result() { return >> _result; } >> }; >> >> @@ -451,10 +429,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_GetObjectMonitorUsage; } >> jvmtiError result() { return >> _result; } >> - void doit() { >> - _result = ((JvmtiEnvBase*) >> >> _env)->get_object_monitor_usage(_calling_thread, >> _object, >> _info_ptr); >> - } >> - >> + void doit(); >> }; >> >> // VM operation to get current >> contended >> monitor. >> @@ -475,13 +450,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_GetCurrentContendedMonitor; } >> jvmtiError result() { return >> _result; } >> - void doit() { >> - _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> - if >> (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() && >> - _java_thread->threadObj() >> != NULL) { >> - _result = ((JvmtiEnvBase >> >> >> >> *)_env)->get_current_contended_monitor(_calling_thread,_ >> java_thread,_owned_monitor_ptr); >> >> - } >> - } >> + void doit(); >> }; >> >> // VM operation to get stack trace >> at safepoint. >> @@ -508,15 +477,7 @@ public: >> } >> jvmtiError result() { return >> _result; } >> VMOp_Type type() const { return >> VMOp_GetStackTrace; } >> - void doit() { >> - _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> - if >> (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() >> - >> && >> _java_thread->threadObj() != >> NULL) { >> - _result = ((JvmtiEnvBase >> *)_env)->get_stack_trace(_java >> _thread, >> - >> _start_depth, >> _max_count, >> - >> _frame_buffer, >> _count_ptr); >> - } >> - } >> + void doit(); >> }; >> >> // forward declaration >> @@ -606,13 +567,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_GetFrameCount; } >> jvmtiError result() { return >> _result; } >> - void doit() { >> - _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> - JavaThread* jt = >> _state->get_thread(); >> - if (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() >> != NULL) { >> - _result = >> ((JvmtiEnvBase*)_env)->get_fra >> me_count(_state, >> _count_ptr); >> - } >> - } >> + void doit(); >> }; >> >> // VM operation to frame location >> at safepoint. >> @@ -636,14 +591,7 @@ public: >> } >> VMOp_Type type() const { return >> VMOp_GetFrameLocation; } >> jvmtiError result() { return >> _result; } >> - void doit() { >> - _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> - if >> (Threads::includes(_java_thread) && >> !_java_thread->is_exiting() && >> - _java_thread->threadObj() >> != NULL) { >> - _result = >> >> ((JvmtiEnvBase*)_env)->get_fra >> me_location(_java_thread, >> _depth, >> - >> _method_ptr, >> _location_ptr); >> - } >> - } >> + void doit(); >> }; >> >> >> -- >> 2.1.4 >> >> >> On Tue, Feb 28, 2017 at 6:11 PM, >> David Holmes >> > >> > > >> > >> > >> >> > >> > > >> > >> > >>> >> > >> > > >> > >> > >> >> >> > >> > > >> > >> > >>>>> wrote: >> >> On 28/02/2017 7:35 PM, David >> Holmes wrote: >> >> On 28/02/2017 6:51 PM, Zhu >> Yong wrote: >> >> Dear All, >> >> I am testing cross compile >> OpenJDK-9+158 for our >> embedded >> system using >> buildroot, I can build >> server and >> client >> variants >> successfully, but the >> output compact1 profile >> file size >> is too >> big, then I >> tried >> to build >> minimal >> variant, failed when >> linking >> libjvm.so. >> >> I checked >> >> >> >> >> build/linux-arm-normal-minimal-release/hotspot/variant- >> minimal/ >> >> libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt, >> jvmtiEnvBase.o >> and >> jvmtiEnvThreadState.o are not >> listed in >> the file >> (which >> is required >> from the error message >> below). >> >> >> Right - JVM TI is not part >> of the >> Minimal VM. At the >> moment it looks >> like some service has either >> been >> enabled when >> it should not >> have been, >> or has not been correctly >> if'def in the >> source. >> >> >> As far as I can see your error >> messages are >> complaining about >> missing functions that are >> called from the >> same >> sources as the >> functions that are missing! ie. >> >> undefined reference to >> >> >> `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, >> JavaThread*, >> _jobject**)' >> /tmp/cc27HS5M.ltrans0.ltrans.o: >> In function >> `VM_GetOwnedMonitorInfo::doit() >> >> but VM_GetOwnedMonitorInfo is >> defined in >> jvmtiEnv.cpp which >> should >> be included or excluded the same >> as >> jvmtiEnBase.cpp. >> Here's the >> logic in the makefile that >> controls this: >> >> ifneq ($(call check-jvm-feature, >> jvmti), >> true) >> JVM_CFLAGS_FEATURES += >> -DINCLUDE_JVMTI=0 >> JVM_EXCLUDE_FILES += >> jvmtiGetLoadedClasses.cpp >> jvmtiThreadState.cpp >> jvmtiExtensions.cpp \ >> jvmtiImpl.cpp >> jvmtiManageCapabilities.cpp >> jvmtiRawMonitor.cpp >> jvmtiUtil.cpp jvmtiTrace.cpp \ >> jvmtiCodeBlobEvents.cpp >> jvmtiEnv.cpp >> jvmtiRedefineClasses.cpp >> jvmtiEnvBase.cpp >> jvmtiEnvThreadState.cpp \ >> jvmtiTagMap.cpp >> jvmtiEventController.cpp >> evmCompat.cpp >> jvmtiEnter.xsl jvmtiExport.cpp \ >> >> jvmtiClassFileReconstituter.cpp >> endif >> >> Can you run with LOG=trace so >> that each >> compiled file is >> listed in >> the build log, then see if there >> are any >> jvmti* >> files listed >> there. >> >> Thanks, >> David >> >> >> >> >> Can you provide the lines >> from your >> spec.gmk >> that define the >> JVM_FEATURES_* variables >> please. >> >> Thanks, >> David >> ------ >> >> >> === Output from failing >> command(s) >> repeated >> here === >> * For target >> >> >> hotspot_variant-minimal_libjvm_objs_BUILD_LIBJVM_link: >> >> /tmp/cc27HS5M.ltrans0.ltrans.o: In >> function >> `VM_GetStackTrace::doit() >> [clone .local.640]': > > From thomas.schatzl at oracle.com Mon Mar 6 11:45:45 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 06 Mar 2017 12:45:45 +0100 Subject: RFR (XXS): JDK-8138610 add assert to ThreadLocalAllocBuffer::clear_before_allocation to check the storage above top In-Reply-To: References: Message-ID: <1488800745.3964.60.camel@oracle.com> Hi Joe, On Mon, 2017-02-13 at 13:57 -0500, Joseph Provino wrote: > Please review this one line change: > > CR: JDK-8138610 > add assert to ThreadLocalAllocBuffer::clear_before_allocation to > check the storage above top > > Webrev: http://cr.openjdk.java.net/~jprovino/8138610/webrev.00 > > Tests:??Passes JPRT. ? I am not sure that the assert is correct. ? 45???assert(!ZapUnusedHeapArea || _reserve_for_allocation_prefetch > 0 ||? ? 46?????top() == NULL || *(intptr_t*)top() != 0, "overzeroing detected");? Maybe I do not understand the related code here, but some questions: ?- did you run jprt with -XX:-ZapUnusedHeapArea? Because this is a develop option that is true in debug mode, i.e. that assert will never be checked without explicitly disabling the ZapUnusedHeapArea flag. ?- if ZeroTLAB (another option that is disabled by default) is enabled, that assert will likely fail because it, as the switch indicates, zeros the entire tlab. I.e. if that code is called before top() reaches end(), the contents are guaranteed to be zero. ?- it is not clear to me too why if _reserve_for_allocation_prefetch is greater than zero, we do not want to do this check. Even with _reserve_for_allocation_prefetch == 0, there is some alignment reserve at the end of the TLAB that could be "overzeroed". ?- neither it is clear to me that the heap after the last object, e.g. in the alignment reserve always contains non-zero data (e.g. if calling this with top() == end()). ?- related to this, I looked at the CR, and it has a cryptic comment that adds another question: what does "uninitialized" mean? (We typically do not initialize the TLAB contents at all). ?- why does the value at top() should not be zero, and not for example badHeapWordVal? (Which the TLAB is initialized with -XX:+ZapUnusedHeapArea). I.e. could that check be made more specific? ?- why does the message of the assert talk about "overzeroing"? Does this refer to the object initialization, or -XX:+ZeroTLAB? ?- jprt only testing is not sufficient due to resman. The patch as is seems to be some random debug assert that just worked for the specific case the original author of the change had been investigating at that time. I recommend asking the original author for feedback on these questions before continuing. Thanks, ? Thomas From coleen.phillimore at oracle.com Mon Mar 6 12:27:28 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 6 Mar 2017 07:27:28 -0500 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BC92C8.40105@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BC92C8.40105@oracle.com> Message-ID: On 3/5/17 5:35 PM, Ioi Lam wrote: > > > On 3/5/17 7:17 AM, coleen.phillimore at oracle.com wrote: >> >> Ioi, Some comments inline (where no comments, insert "ok") :) >> >> On 3/2/17 10:37 PM, Ioi Lam wrote: >>> Hi Coleen, >>> >>> Thanks for the comments. I have updated the webrev. See in-line for >>> responses. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>> >>> >>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> Ioi >>>> I like the concept of this a lot but have some stylistic comments >>>> to help people reading this code later. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>>> >>>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't save many >>>> characters, especially in CppVtableInfo/Testers >>>> >>> Done. >>>> + // Start at slot 1, because slot 0 may be RTTI (on Solaris/Sparc) >>>> + int i; >>>> + for (i=1; ; i++) { >>>> Since you're using 'i' later, can you rename it to something >>>> descriptive. Or have another variable "vtable_length" to use >>>> later. This looks like an old style for loop. >>>> >>> Done >>>> Can the functions for CppVtableInfo be declared outside of the >>>> class declaration? They don't need to be inline and then the debug >>>> code for testing the vtable size can be not in the middle of the >>>> class declaration. Then you can move the Tester classes to inside >>>> the same #ifndef PRODUCT block. >>>> >>>> Can you put #endif // PRODUCT when the ifdef covers several lines >>>> of code? >>>> >>> Done >>>> vtab_of could be more descriptive, like cpp_vtable_for(). >>>> >>> I changed to vtable_of(). Because the class name is already >>> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >>> >>>> Was PrintSharedSpaces was never converted to UL? >>>> >>> Right. I've filed https://bugs.openjdk.java.net/browse/JDK-8176132 >>> (-XX:+PrintSharedSpaces should be converted to use Unified Logging.) >>>> + int n = MAX_VTABLE_SIZE; >>>> >>>> Can you propagate MAX_VTABLE_SIZE to the places where it's used. n >>>> isn't descriptive. This starts out with max_vtable_size and then >>>> changes the size. Reusing 'n' makes this really hard to follow. >>>> Not having a comment that we only allocate enough slots for the >>>> vtable makes it hard too. >>>> >>>> + // allocate CppVtableInfo in the MD section + _info = >>>> (CppVtabInfo*)md_top; >>>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>>> + + // allocate temporary local instance of the metadata type T + T >>>> tmp; >>>> + intptr_t* srcvtab = vtab_of(tmp); >>>> + intptr_t* dstvtab = _info->vtab(); >>>> + >>> Fixed. >>>> Something like that for comments. dstvtab is the >>>> destination_vtable in the MD section. >>> >>> I've dropped the md_ prefix from the functions that deal with the >>> vtables, since they shouldn't care whether it's the "MD" section or >>> not. Now it looks like this: >>> >>> // Allocate and initialize the C++ vtables, starting from top, but >>> do not go past end. >>> intptr_t* MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>> intptr_t* end) { >>> assert(DumpSharedSpaces, "dump-time only"); >>> // Layout (each slot is a intptr_t): >>> // [number of slots in the first vtable = n1] >>> // [ slots for the first vtable] >>> // [number of slots in the first second = n2] >>> // [ slots for the second vtable] >>> // ... >>> // The order of the vtables is the same as the >>> CPP_VTAB_PATCH_TYPES_DO macro. >>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>> return top; >>> } >>> >>>> + for (int i=0; i>>> + const intptr_t bad = intptr_t(0xdeadbeef); >>>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>> + if (num == bad >>>> + // || i > 120 /* uncomment this line to test */ >>>> + ) { >>>> + _info->set_vtab_size(i-1); >>>> + break; >>>> + } >>>> + dstvtab[i] = num; >>>> + } >>>> I dont understand this code. You get deadbeef for a bad value if >>>> SafeFetchN gets a fault but why would it get a fault at the end of >>>> the metadata's vtable? Couldn't it just run onto the next >>>> vtable? I think your original way of counting vtable entries might >>>> be better (sorry I didn't have time to study that thread). >>>> >>> I've modified the comments to this. Does it make sense to you? >>> >>> // It is not always safe to call memcpy(), because srcvtable >>> might be shorter than >>> // MAX_VTABLE_SIZE, and the C++ linker might have placed the >>> vtable at the very >>> // end of the last page of libjvm.so. Crossing over to the next >>> page might >>> // cause a page fault. >>> >>> My fear is the JVM would suddenly start crashing because the order >>> of .o files have changed on the linker's command line, or if you >>> enable some special linker optimization flags. It's better safe than >>> sorry. >> >> This wasn't exactly what I was not understanding. I didn't see that >> you are copying 120 entries from the old vtable and junk memory >> beyond the old vtable, unless you get a segv, in which case you copy >> less. I don't think you should copy random memory into the vtable >> in the archive. This doesn't seem secure, even with the segv protection. >> >> Since we already have assumptions about C++ vtable layout in the code >> and it's mostly specified by various ABIs, and you have the assert >> code, I think I would prefer that you copy only the vtable entries >> into the archive. I guess Thomas Stuefe had a different opinion. >> I've read the original thread. Two points: >> >> If new C++ compiler implementations add a discontigous vtable, both >> the SafeFetchN and subclass additional virtual function at end >> implementation will fail. I don't think C++ implementations would do >> this and a contiguous vtable as first in the instance has been >> standard for years. If our metadata adds multiple inheritance, the >> same issue would be a problem for both implementations, as well as >> for the implementation we have before Ioi's fix. >> >> Ioi's subclass adding virtual function method would work for any >> esoteric C++ implementations in my memory, except the vptr for the >> old DECC++ compiler was after the nonstatic data members (which would >> fail with all of our implementations). >> >> Since the code is there anyway for debug purposes, we're not saving >> code by implementing SafeFetchN. The SafeFetchN implementation isn't >> obvious at all what it's doing, and requires better comments, >> especially if you don't know already what SafeFetchN does. It looks >> really cryptic. The poisoned values also bothered me in that they >> overload other poisoned values in other parts of the jvm. >> > > I can go with either implementation, although I like the original one > that doesn't use SafeFetch. >> Ioi, could you make all methods of CppVtableCloner out of line? >> > Is it for debugging purposes? I am using a recent version of gdb and I > have no problems setting break points or stepping into the code. I can > move the bigger methods out, but for clarity I think it's better to > leave the small methods inside the class declaration. Yes, only the bigger methods need to be out of line. I've never had a problem with gdb and separating declaration and implementation (details!) make it much easier to read, or skip, depending on what you're looking for. thanks! Coleen > > Thanks > - Ioi > >> The other changes look good, although I might have more requests for >> comments. >> >> Thanks, >> Coleen >> >>>> Would be nice to have comments here too!! >>>> >>>> + intptr_t* start = md_top; >>>> >>>> This doesn't do anything (?) >>> >>> Fixed. This was left over code. >>>> >>>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>> >>>> Why not zero the destination vtable in allocate? Or does patching >>>> the vtable pointers call virtual functions? You could prevent that >>>> so you don't need this code. >>>> >>> I added this comment: >>> >>> // During patching, some virtual methods may be called, so at this >>> point >>> // the vtables must contain valid methods (as filled in by >>> CppVtableCloner::allocate). >>> MetaspaceShared::patch_cpp_vtable_pointers(); >>> >>> // The vtable clones contain addresses of the current process. >>> // We don't want to write these addresses into the archive. >>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>> >>>> + // Restore the vtable in case we invoke any virtual methods. >>>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>>> Can this be restore_cpp_vtables since that's what it's doing. The >>>> first is after the dump and the second call is at >>>> UseSharedSpaces. A couple of comments in this clone_cpp_vtables >>>> --> restore_cpp_vtables would be nice. eg: >>>> >>> I prefer to use the word clone. Otherwise when you just say "vtable" >>> it's not clear whether you're talking about the original one (made >>> by the c++ linker), or the cloned one in the CDS archive. >>>> + static intptr_t* clone_vtable(const char* name, intptr_t* p) { >>>> + T tmp; // Allocate temporary dummy metadata object to get vtable initialized >>>> + CppVtabInfo* info = (CppVtabInfo*)p; >>>> + int n = info->vtab_size(); >>>> + intptr_t* srcvtab = vtab_of(tmp); >>>> + intptr_t* dstvtab = info->vtab(); >>>> + >>>> + // We already checked (and, if necessary, adjusted n) when the >>>> vtables were allocated, so we are >>>> + // safe to do memcpy. >>>> + if (PrintSharedSpaces) { >>>> + tty->print_cr("%s copying %d vtable entries", name, n); >>>> + } >>>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>>> + return dstvtab + n; >>>> + } >>>> >>> Done. I changed the wording >>> >>> T tmp; // Allocate temporary dummy metadata object to get to the >>> original vtable. >>> >>> As we are not really "initializing a vtable" here. >>> >>>> Same with 'patch'. It'd be so much faster and easier to read this >>>> code with more comments please. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>>> >>>> Why are these testers here? >>>> >>> >>> I updated the comment: >>> >>> // Used by CDS. These classes need to access the private >>> ConstantPool() constructor. >>> template friend class CppVtableTesterA; >>> template friend class CppVtableTesterB; >>> template friend class CppVtableCloner; >>> >>> >>> Thanks >>> - Ioi >>> >>>>> >>>>>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This is the official review (follow up of the "Determining the size of C++ vtables" thread onhotspot-dev at openjdk.java.net). >>>>>>> >>>>>>> The new code has the same assumption as the existing code in JDK 10: for a C++ object that contains virtual methods (e.g., ConstantPool), we assume the first intptr_t slot of the object is a _vptr, which points to a vtable, which consists of no more than 150 intptr_t's. >>>>>>> >>>>>>> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >>>>>>> [ field #0 ] [ vtable slot 1 ] >>>>>>> [ field #1 ] [ vtable slot 2 ] >>>>>>> [ field #2 ] [ vtable slot 3 ] >>>>>>> [ .... ] [ vtable slot 4] >>>>>>> [ vtable slot 5 ] >>>>>>> [ ... ] >>>>>>> >>>>>>> + In the existing code, we were pointing the vtable slots to >>>>>>> code that's generated by HotSpot. >>>>>>> >>>>>>> + In the new code, we copy the vtable slots from an existing >>>>>>> vtable (generated by the C++ linker). >>>>>>> >>>>>>> Per Thomas St?fe's advice, I don't try to determine the size of the vtable (as that would add one more compiler requirement where new virtual methods added by a subclass must be placed at a higher offset in the vtable). >>>>>>> >>>>>>> Instead, I have added code in non-product builds to ensure that the vtables are no longer than 150 entries. You can run with "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size of the vtables for your particular platform: >>>>>>> >>>>>>> ConstantPool has 12 virtual methods >>>>>>> InstanceKlass has 113 virtual methods >>>>>>> InstanceClassLoaderKlass has 113 virtual methods >>>>>>> InstanceMirrorKlass has 113 virtual methods >>>>>>> InstanceRefKlass has 113 virtual methods >>>>>>> Method has 12 virtual methods >>>>>>> ObjArrayKlass has 114 virtual methods >>>>>>> TypeArrayKlass has 114 virtual methods >>>>>>> >>>>>>> As mentioned in the code comments, if you have an esoteric C++ compiler, the verify_sufficient_size() function will probably fail, but hopefully that would give you some hints for porting this code. >>>>>>> >>>>>>> To avoid accidentally touching an unmapped page, the code uses SafeFetchN for copying the vtable contents, and would shrink the vtable to less than 150 entries if necessary. I can't test this for real, but I've added some code to simulate an error: >>>>>>> >>>>>>> for (int i=0; i>>>>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>>>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>>> if (num == bad >>>>>>> // || i > 120 /* uncomment this line to test */ >>>>>>> ) { >>>>>>> _info->set_vtab_size(i-1); >>>>>>> break; >>>>>>> } >>>>>>> dstvtab[i] = num; >>>>>>> } >>>>>>> >>>>>>> Results: >>>>>>> >>>>>>> + Removed 850 lines of CPU-dependent code >>>>>>> >>>>>>> + CDS image is about 50K smaller >>>>>>> >>>>>>> + Previously Metadata objects must live in the read-write section in the CDS >>>>>>> archive, because their _vptr was updated at run time. Now _vptr is no longer >>>>>>> updated, so ConstantPool can be moved to the read-only section (see JDK-8171392). >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> >>>> >>> >> > From christian.tornqvist at oracle.com Mon Mar 6 15:36:10 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Mon, 6 Mar 2017 10:36:10 -0500 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* In-Reply-To: <86ccedc2-37e1-9360-f0d5-5ed2338b0c8d@oracle.com> References: <169f01d29434$a84a7d70$f8df7850$@oracle.com> <86ccedc2-37e1-9360-f0d5-5ed2338b0c8d@oracle.com> Message-ID: <310c01d2968f$61e0d520$25a27f60$@oracle.com> Hi David, Not sure what that name would be though, the compiler is using this group with flag rotation in multiple tiers. I'll leave the renaming of this group to the compiler team to sort out. Thanks, Christioan -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Friday, March 3, 2017 9:21 PM To: Christian Tornqvist ; hotspot-dev at openjdk.java.net Subject: Re: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* Hi Christian, Rename looks good. Should we also rename the now oddly named hotspot_not_fast_compiler? Thanks, David On 4/03/2017 1:41 AM, Christian Tornqvist wrote: > Hi everyone, > > > > Please review this small change that renames the hotspot_fast* test > groups to hotspot_tier1, also renamed the hotspot_runtime_tier* to > follow the same naming standard. > > Tested by running the change through JPRT. > > > > Webrevs: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ > > http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8176102 > > > > Thanks, > > Christian > From thomas.stuefe at gmail.com Mon Mar 6 16:51:50 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 6 Mar 2017 17:51:50 +0100 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> Message-ID: Hi Coleen and Ioi, I had to port C++ code to platforms with terrible compilers for a time in my life, that is why I like code to be as portable as possible. That said, you are right in your argumentation, the SafeFetch solution is not terribly elegant and Ioi's original way of determining the vtable size is cleaner. I did some checks on some of our architectures with a test similar to Ioi's and on a first glance it seems to work for simple cases (single and public inheritance) on ppc (AIX) and Linux ia64. Although the vtables seemed to me to contain function descriptors, not real pointers to code, so this is something to keep in mind. But if the live vtable are copied, the function descriptors they contain should point to valid code too, so it should not matter. Just to remember to not expect every slot in the array to be a valid code pointer. So, in short, I remove my objection to Ioi's original solution, as far as that matters. I still think we rely on a lot here: Contiguous vtable containing absolute memory addresses, vtable pointer at start of object and vtable entries to be ordered from base->derived class. So I wonder how much effort it would be (now or in the future as a separate change) to have a fallback where - at loading time - instead of copying vtables the vtable pointers in the objects were fixed up to point to the new live vtables? I know this would be more expensive and potentially defeat the point of shared classes. But maybe not, it depends on how many objects are there, no? Kind Regards, Thomas On Sun, Mar 5, 2017 at 4:17 PM, wrote: > > Ioi, Some comments inline (where no comments, insert "ok") :) > > > On 3/2/17 10:37 PM, Ioi Lam wrote: > >> Hi Coleen, >> >> Thanks for the comments. I have updated the webrev. See in-line for >> responses. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind >> ependent-cds-vtable.v03/ >> >> >> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com wrote: >> >>> >>> Ioi >>> I like the concept of this a lot but have some stylistic comments to >>> help people reading this code later. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind >>> ependent-cds-vtable.v02/src/share/vm/memory/metaspaceShare >>> d.cpp.udiff.html >>> >>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't save many >>> characters, especially in CppVtableInfo/Testers >>> >>> Done. >> >>> + // Start at slot 1, because slot 0 may be RTTI (on Solaris/Sparc) >>> + int i; >>> + for (i=1; ; i++) { >>> Since you're using 'i' later, can you rename it to something >>> descriptive. Or have another variable "vtable_length" to use later. This >>> looks like an old style for loop. >>> >>> Done >> >>> Can the functions for CppVtableInfo be declared outside of the class >>> declaration? They don't need to be inline and then the debug code for >>> testing the vtable size can be not in the middle of the class declaration. >>> Then you can move the Tester classes to inside the same #ifndef PRODUCT >>> block. >>> >>> Can you put #endif // PRODUCT when the ifdef covers several lines of >>> code? >>> >>> Done >> >>> vtab_of could be more descriptive, like cpp_vtable_for(). >>> >>> I changed to vtable_of(). Because the class name is already >> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >> >> Was PrintSharedSpaces was never converted to UL? >>> >>> Right. I've filed https://bugs.openjdk.java.net/browse/JDK-8176132 >> (-XX:+PrintSharedSpaces should be converted to use Unified Logging.) >> >>> + int n = MAX_VTABLE_SIZE; >>> >>> Can you propagate MAX_VTABLE_SIZE to the places where it's used. n >>> isn't descriptive. This starts out with max_vtable_size and then changes >>> the size. Reusing 'n' makes this really hard to follow. Not having a >>> comment that we only allocate enough slots for the vtable makes it hard too. >>> >>> + // allocate CppVtableInfo in the MD section + _info = >>> (CppVtabInfo*)md_top; >>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>> + + // allocate temporary local instance of the metadata type T + T tmp; >>> + intptr_t* srcvtab = vtab_of(tmp); >>> + intptr_t* dstvtab = _info->vtab(); >>> + >>> >> Fixed. >> >>> Something like that for comments. dstvtab is the destination_vtable in >>> the MD section. >>> >> >> I've dropped the md_ prefix from the functions that deal with the >> vtables, since they shouldn't care whether it's the "MD" section or not. >> Now it looks like this: >> >> // Allocate and initialize the C++ vtables, starting from top, but do not >> go past end. >> intptr_t* MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >> intptr_t* end) { >> assert(DumpSharedSpaces, "dump-time only"); >> // Layout (each slot is a intptr_t): >> // [number of slots in the first vtable = n1] >> // [ slots for the first vtable] >> // [number of slots in the first second = n2] >> // [ slots for the second vtable] >> // ... >> // The order of the vtables is the same as the CPP_VTAB_PATCH_TYPES_DO >> macro. >> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >> return top; >> } >> >> + for (int i=0; i>> + const intptr_t bad = intptr_t(0xdeadbeef); >>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>> + if (num == bad >>> + // || i > 120 /* uncomment this line to test */ >>> + ) { >>> + _info->set_vtab_size(i-1); >>> + break; >>> + } >>> + dstvtab[i] = num; >>> + } >>> I dont understand this code. You get deadbeef for a bad value if >>> SafeFetchN gets a fault but why would it get a fault at the end of the >>> metadata's vtable? Couldn't it just run onto the next vtable? I think >>> your original way of counting vtable entries might be better (sorry I >>> didn't have time to study that thread). >>> >>> I've modified the comments to this. Does it make sense to you? >> >> // It is not always safe to call memcpy(), because srcvtable might be >> shorter than >> // MAX_VTABLE_SIZE, and the C++ linker might have placed the vtable >> at the very >> // end of the last page of libjvm.so. Crossing over to the next page >> might >> // cause a page fault. >> >> My fear is the JVM would suddenly start crashing because the order of .o >> files have changed on the linker's command line, or if you enable some >> special linker optimization flags. It's better safe than sorry. >> > > This wasn't exactly what I was not understanding. I didn't see that you > are copying 120 entries from the old vtable and junk memory beyond the old > vtable, unless you get a segv, in which case you copy less. I don't think > you should copy random memory into the vtable in the archive. This doesn't > seem secure, even with the segv protection. > > Since we already have assumptions about C++ vtable layout in the code and > it's mostly specified by various ABIs, and you have the assert code, I > think I would prefer that you copy only the vtable entries into the > archive. I guess Thomas Stuefe had a different opinion. I've read the > original thread. Two points: > > If new C++ compiler implementations add a discontigous vtable, both the > SafeFetchN and subclass additional virtual function at end implementation > will fail. I don't think C++ implementations would do this and a > contiguous vtable as first in the instance has been standard for years. > If our metadata adds multiple inheritance, the same issue would be a > problem for both implementations, as well as for the implementation we have > before Ioi's fix. > > Ioi's subclass adding virtual function method would work for any esoteric > C++ implementations in my memory, except the vptr for the old DECC++ > compiler was after the nonstatic data members (which would fail with all of > our implementations). > > Since the code is there anyway for debug purposes, we're not saving code > by implementing SafeFetchN. The SafeFetchN implementation isn't obvious at > all what it's doing, and requires better comments, especially if you don't > know already what SafeFetchN does. It looks really cryptic. The poisoned > values also bothered me in that they overload other poisoned values in > other parts of the jvm. > > Ioi, could you make all methods of CppVtableCloner out of line? > > The other changes look good, although I might have more requests for > comments. > > Thanks, > Coleen > > > Would be nice to have comments here too!! >>> >>> + intptr_t* start = md_top; >>> >>> This doesn't do anything (?) >>> >> >> Fixed. This was left over code. >> >>> >>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>> >>> Why not zero the destination vtable in allocate? Or does patching the >>> vtable pointers call virtual functions? You could prevent that so you >>> don't need this code. >>> >>> I added this comment: >> >> // During patching, some virtual methods may be called, so at this point >> // the vtables must contain valid methods (as filled in by >> CppVtableCloner::allocate). >> MetaspaceShared::patch_cpp_vtable_pointers(); >> >> // The vtable clones contain addresses of the current process. >> // We don't want to write these addresses into the archive. >> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >> >> + // Restore the vtable in case we invoke any virtual methods. >>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>> Can this be restore_cpp_vtables since that's what it's doing. The first >>> is after the dump and the second call is at UseSharedSpaces. A couple of >>> comments in this clone_cpp_vtables --> restore_cpp_vtables would be nice. >>> eg: >>> >>> I prefer to use the word clone. Otherwise when you just say "vtable" >> it's not clear whether you're talking about the original one (made by the >> c++ linker), or the cloned one in the CDS archive. >> >>> + static intptr_t* clone_vtable(const char* name, intptr_t* p) { >>> + T tmp; // Allocate temporary dummy metadata object to get vtable >>> initialized >>> + CppVtabInfo* info = (CppVtabInfo*)p; >>> + int n = info->vtab_size(); >>> + intptr_t* srcvtab = vtab_of(tmp); >>> + intptr_t* dstvtab = info->vtab(); >>> + >>> + // We already checked (and, if necessary, adjusted n) when the vtables >>> were allocated, so we are >>> + // safe to do memcpy. >>> + if (PrintSharedSpaces) { >>> + tty->print_cr("%s copying %d vtable entries", name, n); >>> + } >>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>> + return dstvtab + n; >>> + } >>> >>> Done. I changed the wording >> >> T tmp; // Allocate temporary dummy metadata object to get to the >> original vtable. >> >> As we are not really "initializing a vtable" here. >> >> Same with 'patch'. It'd be so much faster and easier to read this code >>> with more comments please. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind >>> ependent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>> >>> Why are these testers here? >>> >>> >> I updated the comment: >> >> // Used by CDS. These classes need to access the private ConstantPool() >> constructor. >> template friend class CppVtableTesterA; >> template friend class CppVtableTesterB; >> template friend class CppVtableCloner; >> >> >> Thanks >> - Ioi >> >> >>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind >>>>>> ependent-cds-vtable.v02/ >>>>>> Hi, >>>>>> >>>>>> This is the official review (follow up of the "Determining the size >>>>>> of C++ vtables" thread onhotspot-dev at openjdk.java.net). >>>>>> >>>>>> The new code has the same assumption as the existing code in JDK 10: >>>>>> for a C++ object that contains virtual methods (e.g., ConstantPool), we >>>>>> assume the first intptr_t slot of the object is a _vptr, which points to a >>>>>> vtable, which consists of no more than 150 intptr_t's. >>>>>> >>>>>> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >>>>>> [ field #0 ] [ vtable slot 1 ] >>>>>> [ field #1 ] [ vtable slot 2 ] >>>>>> [ field #2 ] [ vtable slot 3 ] >>>>>> [ .... ] [ vtable slot 4] >>>>>> [ vtable slot 5 ] >>>>>> [ ... ] >>>>>> >>>>>> + In the existing code, we were pointing the vtable slots to >>>>>> code that's generated by HotSpot. >>>>>> >>>>>> + In the new code, we copy the vtable slots from an existing >>>>>> vtable (generated by the C++ linker). >>>>>> >>>>>> Per Thomas St?fe's advice, I don't try to determine the size of the >>>>>> vtable (as that would add one more compiler requirement where new virtual >>>>>> methods added by a subclass must be placed at a higher offset in the >>>>>> vtable). >>>>>> >>>>>> Instead, I have added code in non-product builds to ensure that the >>>>>> vtables are no longer than 150 entries. You can run with >>>>>> "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size of the >>>>>> vtables for your particular platform: >>>>>> >>>>>> ConstantPool has 12 virtual methods >>>>>> InstanceKlass has 113 virtual methods >>>>>> InstanceClassLoaderKlass has 113 virtual methods >>>>>> InstanceMirrorKlass has 113 virtual methods >>>>>> InstanceRefKlass has 113 virtual methods >>>>>> Method has 12 virtual methods >>>>>> ObjArrayKlass has 114 virtual methods >>>>>> TypeArrayKlass has 114 virtual methods >>>>>> >>>>>> As mentioned in the code comments, if you have an esoteric C++ >>>>>> compiler, the verify_sufficient_size() function will probably fail, but >>>>>> hopefully that would give you some hints for porting this code. >>>>>> >>>>>> To avoid accidentally touching an unmapped page, the code uses >>>>>> SafeFetchN for copying the vtable contents, and would shrink the vtable to >>>>>> less than 150 entries if necessary. I can't test this for real, but I've >>>>>> added some code to simulate an error: >>>>>> >>>>>> for (int i=0; i>>>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>> if (num == bad >>>>>> // || i > 120 /* uncomment this line to test */ >>>>>> ) { >>>>>> _info->set_vtab_size(i-1); >>>>>> break; >>>>>> } >>>>>> dstvtab[i] = num; >>>>>> } >>>>>> >>>>>> Results: >>>>>> >>>>>> + Removed 850 lines of CPU-dependent code >>>>>> >>>>>> + CDS image is about 50K smaller >>>>>> >>>>>> + Previously Metadata objects must live in the read-write section in >>>>>> the CDS >>>>>> archive, because their _vptr was updated at run time. Now _vptr is >>>>>> no longer >>>>>> updated, so ConstantPool can be moved to the read-only section (see >>>>>> JDK-8171392). >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> >>>>>> >>> >> > From vladimir.kozlov at oracle.com Mon Mar 6 16:59:12 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 6 Mar 2017 08:59:12 -0800 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* In-Reply-To: <310c01d2968f$61e0d520$25a27f60$@oracle.com> References: <169f01d29434$a84a7d70$f8df7850$@oracle.com> <86ccedc2-37e1-9360-f0d5-5ed2338b0c8d@oracle.com> <310c01d2968f$61e0d520$25a27f60$@oracle.com> Message-ID: Hi Christian, As I remember hotspot_not_fast_compiler is used in later tiers. Can we just name it accordingly, for example: hotspot_tier2_compiler? Thanks, Vladimir On 3/6/17 7:36 AM, Christian Tornqvist wrote: > Hi David, > > Not sure what that name would be though, the compiler is using this group > with flag rotation in multiple tiers. I'll leave the renaming of this group > to the compiler team to sort out. > > Thanks, > Christioan > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Friday, March 3, 2017 9:21 PM > To: Christian Tornqvist ; > hotspot-dev at openjdk.java.net > Subject: Re: RFR(S): 8176102 - Rename hotspot_fast* test groups to > hotspot_tier1* > > Hi Christian, > > Rename looks good. > > Should we also rename the now oddly named hotspot_not_fast_compiler? > > Thanks, > David > > On 4/03/2017 1:41 AM, Christian Tornqvist wrote: >> Hi everyone, >> >> >> >> Please review this small change that renames the hotspot_fast* test >> groups to hotspot_tier1, also renamed the hotspot_runtime_tier* to >> follow the same naming standard. >> >> Tested by running the change through JPRT. >> >> >> >> Webrevs: >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00/ >> >> >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8176102 >> >> >> >> Thanks, >> >> Christian >> > From zhuyong.me at gmail.com Fri Mar 3 08:59:23 2017 From: zhuyong.me at gmail.com (Zhu Yong) Date: Fri, 3 Mar 2017 16:59:23 +0800 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> Message-ID: Thank you very much for helping. Today I tried to debug the test code, but without luck, still can't figure out why. First, I followed the instruction http://stackoverflow.com/questions/29916995/intellij-idea-remotely-debug-java-console-program, changed suspend=y because the test code is short, it will exit before debug start if not suspend first. Unfortunately, with remote debug, I was not able to reproduce the issue. Same class file, directly run, will produce the issue, remote debug won't. Then, I tried *jdb *from my OpenJDK9 client vm, run directly on my target system to debug the code. If I set breakpoint, program run without issue, however, if don't set breakpoint, just "run" in jdb, problem reproduced. Next, I tried to use *javap *to decompile the output class file, run OpenJDK9 client vm javap and host side javap on same class file, the output is same for getTime function. One interesting point I noticed today is the output like below when problem happen, 2nd call of getTime() has value of 1st call result / 1000. Looks like 2nd call getTime(), double variable q was not updated startTime: 284092.063000 endTime: 284.092063 runTime: -283807.970937 What should I do so that I can come a conclusion that it's due to compiler issue? On Thu, Mar 2, 2017 at 12:32 PM, David Holmes wrote: > I built your sample program with latest JDK 9 for arm-vfp-sflt and ran on > an emulator I have available, and I had no issues. > > So I still have concerns this may be a build/compiler issue. > > David > > > On 2/03/2017 12:51 PM, David Holmes wrote: > >> I've moved this discussion over to hotspot-dev as this seems no longer a >> build issue but a C1 soft-float issue. >> >> David >> >> On 2/03/2017 12:35 PM, Zhu Yong wrote: >> >>> If run with -Xint, it works. >>> >>> I have created simplified FP test by remove all non-related code from >>> Whetstone test code. >>> The test code is here: >>> https://gist.github.com/yongzhy/d65c26d39fe5d549d1b406c7c84eacd4 >>> >>> I am not sure if the test code can help or not. The behaviour is weird : >>> - If I print both itime and q, program will run OK >>> - inside condition block if(q<1.0f), if I use exit code 2,3,4,5, problem >>> appears, however, if I use exit code >=6, program run OK. >>> >>> I always get exit code 5, the line q = (double)itime is wrong? >>> >>> public static double getTime() >>> { >>> double q; >>> long itime; >>> itime = System.currentTimeMillis(); >>> if(itime<1000000) { >>> System.exit(1); >>> } >>> //System.out.printf("time long value %d\n", itime); >>> q = (double) itime; >>> if(q < 1.0f) { >>> System.exit(5); // I always get exit code 5 >>> } >>> //System.out.printf("time float value %f\n", q); >>> return q / 1000.0; >>> } >>> >>> >>> >>> On Wed, Mar 1, 2017 at 5:31 PM, David Holmes >> > wrote: >>> >>> On 1/03/2017 6:45 PM, Zhu Yong wrote: >>> >>> OK, for the Whetstone Benchmark, I added debug printing and >>> found it's >>> due to function getTime(), it only get good value on 1st call, >>> all >>> following calls get 0s. >>> Debug printing of itime value is correct. So reason should be >>> the return >>> division line. Could it because toolchain's floating point >>> operation??? >>> But why server VM Ok? >>> >>> >>> Server and client implement FP logic differently, and particularly >>> as this is soft-fp, they may well not be in sync. Does -Xint work? >>> >>> Can you isolate to a simple FP test? >>> >>> David >>> >>> public static double getTime() >>> { >>> double q; >>> long itime; >>> itime = System.currentTimeMillis(); >>> q = (double) itime; >>> return q / 1000.0; >>> } >>> >>> On Wed, Mar 1, 2017 at 3:14 PM, David Holmes >>> >>> >> >> wrote: >>> >>> On 1/03/2017 4:26 PM, Zhu Yong wrote: >>> >>> We use armv7-marvell-linux-gnueabi-softfp toolchain >>> >>> gcc version 4.6.4 (Marvell GCC release >>> 20150204-c4af733b 64K >>> MAXPAGESIZE >>> ALIGN CVE-2015-0235) >>> >>> Is this toolchain too old? I am asking because I saw >>> other strange >>> issues as well: >>> >>> >>> We last used gcc 4.7.2. I can't really say if 4.6.4 is "too >>> old". >>> >>> I have successfully build server, client and minimal VM, >>> when I run >>> Dhrystone benchmark file (the same jar file from >>> http://www.okayan.jp/DhrystoneApplet/ >>> >>> >> >), the performance from >>> server VM >>> is much better than client and minimal VM. >>> (minimal: 1629852, client: 1615508, server: 2338871 ) >>> >>> >>> That's expected. The server VM is high performance. >>> >>> And when run Whetstone Benchmark >>> from >>> http://www.roylongbottom.org.uk/online/whetjava2.html >>> >>> >> >, >>> server VM >>> finished with good result, client and minimal VM not >>> able to finish >>> (program stuck there forever after print 1st line of >>> output). >>> >>> >>> That needs investigating. You need to try and generate a >>> stackdump >>> to see where things are stuck. >>> >>> David >>> >>> >>> On Wed, Mar 1, 2017 at 1:56 PM, David Holmes >>> >> >> > >>> >> >>> >> >>> wrote: >>> >>> On 1/03/2017 3:39 PM, Zhu Yong wrote: >>> >>> Thank you for the information, I managed to make >>> minimal >>> build >>> pass now. >>> >>> Initially I though JVM_EXCLUDE_FILES need to add >>> additional 3 >>> generated >>> files (they appeared >>> in _BUILD_LIBJVM_objectfilenames.txt) : >>> bytecodeInterpreterWithChecks.cpp jvmtiEnter.cpp >>> jvmtiEnterTrace.cpp >>> But build still fail with same error message. >>> >>> Finally I figured out it's due to those 8 doit() >>> functions >>> implementation in jvmtiEnvBase.hpp file. After >>> move all >>> those 8 >>> doit() >>> implementations to jvmtiEnvBase.cpp, build of >>> minimal VM >>> passed >>> without >>> any issue. >>> >>> >>> That seems to be a compiler issue. >>> jvmtiEnvBase.hpp is >>> unconditionally included in a couple of places >>> because we >>> have to >>> have enough of the JVMTI code to say JVMTI is not >>> available. >>> Those >>> doit() implementations, though arguably could be >>> conditional on >>> INCLUDE_JVMTI, are not referenced by any called code >>> and so >>> should >>> not linked in. But in your case it seems they are. >>> >>> What toolchain are you using? >>> >>> David >>> ----- >>> >>> Changes appended >>> ============= >>> >>> --- >>> .../src/share/vm/prims/jvmtiEnvBase.cpp >>> | 74 >>> ++++++++++++++++++++++ >>> .../src/share/vm/prims/jvmtiEnvBase.hpp >>> | 68 >>> +++----------------- >>> 2 files changed, 82 insertions(+), 60 >>> deletions(-) >>> mode change 100644 => 100755 >>> >>> hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> mode change 100644 => 100755 >>> >>> hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> >>> diff --git >>> >>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> >>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> old mode 100644 >>> new mode 100755 >>> index dd241a0..e5832ba >>> --- >>> >>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> +++ >>> >>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> @@ -1283,6 +1283,80 @@ >>> >>> VM_GetMultipleStackTraces::allocate_and_fill_stacks(jint >>> thread_count) { >>> "the last copied frame info must be >>> the last >>> record"); >>> } >>> >>> +void >>> +VM_UpdateForPopTopFrame::doit() { >>> + JavaThread* jt = _state->get_thread(); >>> + if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + _state->update_for_pop_top_frame(); >>> + } else { >>> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + } >>> +} >>> + >>> +void >>> +VM_SetFramePop::doit() { >>> + JavaThread* jt = _state->get_thread(); >>> + if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + int frame_number = _state->count_frames() - >>> _depth; >>> + >>> >>> >>> >>> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ >>> pop(frame_number); >>> >>> + } else { >>> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + } >>> +} >>> + >>> +void >>> +VM_GetOwnedMonitorInfo::doit() { >>> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() >>> + && >>> _java_thread->threadObj() != >>> NULL) { >>> + _result = ((JvmtiEnvBase >>> *)_env)->get_owned_monitors(_calling_thread, >>> _java_thread, >>> + >>> _owned_monitors_list); >>> + } >>> +} >>> + >>> +void >>> +VM_GetObjectMonitorUsage::doit() { >>> + _result = ((JvmtiEnvBase*) >>> _env)->get_object_monitor_usage(_calling_thread, >>> _object, >>> _info_ptr); >>> +} >>> + >>> +void >>> +VM_GetCurrentContendedMonitor::doit() { >>> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() && >>> + _java_thread->threadObj() != NULL) { >>> + _result = ((JvmtiEnvBase >>> >>> >>> >>> *)_env)->get_current_contended_monitor(_calling_thread,_ >>> java_thread,_owned_monitor_ptr); >>> >>> + } >>> +} >>> + >>> +void >>> +VM_GetStackTrace::doit() { >>> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() >>> + && >>> _java_thread->threadObj() != >>> NULL) { >>> + _result = ((JvmtiEnvBase >>> *)_env)->get_stack_trace(_java_thread, >>> + >>> _start_depth, >>> _max_count, >>> + >>> _frame_buffer, >>> _count_ptr); >>> + } >>> +} >>> + >>> +void >>> +VM_GetFrameCount::doit() { >>> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + JavaThread* jt = _state->get_thread(); >>> + if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + _result = >>> ((JvmtiEnvBase*)_env)->get_frame_count(_state, >>> _count_ptr); >>> + } >>> +} >>> + >>> +void >>> +VM_GetFrameLocation::doit() { >>> + _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() && >>> + _java_thread->threadObj() != NULL) { >>> + _result = >>> >>> ((JvmtiEnvBase*)_env)->get_frame_location(_java_thread, >>> _depth, >>> + >>> _method_ptr, >>> _location_ptr); >>> + } >>> +} >>> >>> void >>> VM_GetThreadListStackTraces::doit() { >>> diff --git >>> >>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> >>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> old mode 100644 >>> new mode 100755 >>> index 04e6869..00b9890 >>> --- >>> >>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> +++ >>> >>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> @@ -359,14 +359,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_UpdateForPopTopFrame; } >>> jvmtiError result() { return _result; } >>> - void doit() { >>> - JavaThread* jt = _state->get_thread(); >>> - if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() >>> != NULL) { >>> - _state->update_for_pop_top_frame(); >>> - } else { >>> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> // VM operation to set frame pop. >>> @@ -389,15 +382,7 @@ public: >>> bool allow_nested_vm_operations() const { >>> return true; } >>> VMOp_Type type() const { return >>> VMOp_SetFramePop; } >>> jvmtiError result() { return _result; } >>> - void doit() { >>> - JavaThread* jt = _state->get_thread(); >>> - if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() >>> != NULL) { >>> - int frame_number = _state->count_frames() >>> - _depth; >>> - >>> >>> >>> >>> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ >>> pop(frame_number); >>> >>> - } else { >>> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> >>> @@ -421,14 +406,7 @@ public: >>> _result = JVMTI_ERROR_NONE; >>> } >>> VMOp_Type type() const { return >>> VMOp_GetOwnedMonitorInfo; } >>> - void doit() { >>> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - if (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() >>> - && >>> _java_thread->threadObj() != >>> NULL) { >>> - _result = ((JvmtiEnvBase >>> *)_env)->get_owned_monitors(_calling_thread, >>> _java_thread, >>> - >>> _owned_monitors_list); >>> - } >>> - } >>> + void doit(); >>> jvmtiError result() { return _result; } >>> }; >>> >>> @@ -451,10 +429,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_GetObjectMonitorUsage; } >>> jvmtiError result() { return _result; } >>> - void doit() { >>> - _result = ((JvmtiEnvBase*) >>> _env)->get_object_monitor_usage(_calling_thread, >>> _object, >>> _info_ptr); >>> - } >>> - >>> + void doit(); >>> }; >>> >>> // VM operation to get current contended >>> monitor. >>> @@ -475,13 +450,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_GetCurrentContendedMonitor; } >>> jvmtiError result() { return _result; } >>> - void doit() { >>> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - if (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() && >>> - _java_thread->threadObj() != NULL) { >>> - _result = ((JvmtiEnvBase >>> >>> >>> >>> *)_env)->get_current_contended_monitor(_calling_thread,_ >>> java_thread,_owned_monitor_ptr); >>> >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> // VM operation to get stack trace at safepoint. >>> @@ -508,15 +477,7 @@ public: >>> } >>> jvmtiError result() { return _result; } >>> VMOp_Type type() const { return >>> VMOp_GetStackTrace; } >>> - void doit() { >>> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - if (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() >>> - && >>> _java_thread->threadObj() != >>> NULL) { >>> - _result = ((JvmtiEnvBase >>> *)_env)->get_stack_trace(_java_thread, >>> - >>> _start_depth, >>> _max_count, >>> - >>> _frame_buffer, >>> _count_ptr); >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> // forward declaration >>> @@ -606,13 +567,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_GetFrameCount; } >>> jvmtiError result() { return _result; } >>> - void doit() { >>> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - JavaThread* jt = _state->get_thread(); >>> - if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() >>> != NULL) { >>> - _result = >>> ((JvmtiEnvBase*)_env)->get_frame_count(_state, >>> _count_ptr); >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> // VM operation to frame location at safepoint. >>> @@ -636,14 +591,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_GetFrameLocation; } >>> jvmtiError result() { return _result; } >>> - void doit() { >>> - _result = JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - if (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() && >>> - _java_thread->threadObj() != NULL) { >>> - _result = >>> >>> ((JvmtiEnvBase*)_env)->get_frame_location(_java_thread, >>> _depth, >>> - >>> _method_ptr, >>> _location_ptr); >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> >>> -- >>> 2.1.4 >>> >>> >>> On Tue, Feb 28, 2017 at 6:11 PM, David Holmes >>> >> >>> >> > >>> >>> >> >> >>> >> >>> >> > >>> >>> >> >>> >> >>>> wrote: >>> >>> On 28/02/2017 7:35 PM, David Holmes wrote: >>> >>> On 28/02/2017 6:51 PM, Zhu Yong wrote: >>> >>> Dear All, >>> >>> I am testing cross compile >>> OpenJDK-9+158 for our >>> embedded >>> system using >>> buildroot, I can build server and >>> client >>> variants >>> successfully, but the >>> output compact1 profile file size >>> is too >>> big, then I >>> tried >>> to build >>> minimal >>> variant, failed when linking >>> libjvm.so. >>> >>> I checked >>> >>> >>> >>> build/linux-arm-normal-minimal-release/hotspot/variant-minimal/ >>> >>> libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt, >>> jvmtiEnvBase.o >>> and jvmtiEnvThreadState.o are not >>> listed in >>> the file >>> (which >>> is required >>> from the error message below). >>> >>> >>> Right - JVM TI is not part of the >>> Minimal VM. At the >>> moment it looks >>> like some service has either been >>> enabled when >>> it should not >>> have been, >>> or has not been correctly if'def in the >>> source. >>> >>> >>> As far as I can see your error messages are >>> complaining about >>> missing functions that are called from the >>> same >>> sources as the >>> functions that are missing! ie. >>> >>> undefined reference to >>> >>> `JvmtiEnvBase::get_current_con >>> tended_monitor(JavaThread*, >>> JavaThread*, >>> _jobject**)' >>> /tmp/cc27HS5M.ltrans0.ltrans.o: In function >>> `VM_GetOwnedMonitorInfo::doit() >>> >>> but VM_GetOwnedMonitorInfo is defined in >>> jvmtiEnv.cpp which >>> should >>> be included or excluded the same as >>> jvmtiEnBase.cpp. >>> Here's the >>> logic in the makefile that controls this: >>> >>> ifneq ($(call check-jvm-feature, jvmti), >>> true) >>> JVM_CFLAGS_FEATURES += -DINCLUDE_JVMTI=0 >>> JVM_EXCLUDE_FILES += >>> jvmtiGetLoadedClasses.cpp >>> jvmtiThreadState.cpp jvmtiExtensions.cpp \ >>> jvmtiImpl.cpp >>> jvmtiManageCapabilities.cpp >>> jvmtiRawMonitor.cpp >>> jvmtiUtil.cpp jvmtiTrace.cpp \ >>> jvmtiCodeBlobEvents.cpp jvmtiEnv.cpp >>> jvmtiRedefineClasses.cpp >>> jvmtiEnvBase.cpp jvmtiEnvThreadState.cpp \ >>> jvmtiTagMap.cpp >>> jvmtiEventController.cpp >>> evmCompat.cpp >>> jvmtiEnter.xsl jvmtiExport.cpp \ >>> jvmtiClassFileReconstituter.cpp >>> endif >>> >>> Can you run with LOG=trace so that each >>> compiled file is >>> listed in >>> the build log, then see if there are any >>> jvmti* >>> files listed >>> there. >>> >>> Thanks, >>> David >>> >>> >>> >>> >>> Can you provide the lines from your >>> spec.gmk >>> that define the >>> JVM_FEATURES_* variables please. >>> >>> Thanks, >>> David >>> ------ >>> >>> >>> === Output from failing command(s) >>> repeated >>> here === >>> * For target >>> >>> hotspot_variant-minimal_libjvm_objs_BUILD_LIBJVM_link: >>> /tmp/cc27HS5M.ltrans0.ltrans.o: In >>> function >>> `VM_GetStackTrace::doit() >>> [clone .local.640]': >>> cc27HS5M.ltrans0.o:(.text+0xce5e): >>> undefined >>> reference to >>> >>> `JvmtiEnvBase::get_stack_trace(JavaThread*, >>> int, int, >>> _jvmtiFrameInfo*, >>> int*)' >>> /tmp/cc27HS5M.ltrans0.ltrans.o: In >>> function >>> >>> `VM_GetCurrentContendedMonitor::doit() >>> [clone .local.639]': >>> cc27HS5M.ltrans0.o:(.text+0xcec2): >>> undefined >>> reference to >>> >>> >>> `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, >>> JavaThread*, >>> _jobject**)' >>> /tmp/cc27HS5M.ltrans0.ltrans.o: In >>> function >>> `VM_GetOwnedMonitorInfo::doit() >>> [clone .local.638]': >>> cc27HS5M.ltrans0.o:(.text+0xcf26): >>> undefined >>> reference to >>> >>> `JvmtiEnvBase::get_owned_monitors(JavaThread*, >>> JavaThread*, >>> GrowableArray<_ >>> jvmtiMonitorStackDepthInfo*>*)' >>> /tmp/cc27HS5M.ltrans0.ltrans.o: In >>> function >>> `VM_GetFrameCount::doit() >>> [clone .local.637]': >>> cc27HS5M.ltrans0.o:(.text+0xcf8a): >>> undefined >>> reference to >>> >>> `JvmtiEnvBase::get_frame_count(JvmtiThreadState*, int*)' >>> /tmp/cc27HS5M.ltrans0.ltrans.o: In >>> function >>> `VM_SetFramePop::doit() >>> [clone >>> .local.636]': >>> cc27HS5M.ltrans0.o:(.text+0xcfea): >>> undefined >>> reference to >>> `JvmtiThreadState::count_frames()' >>> cc27HS5M.ltrans0.o:(.text+0xd030): >>> undefined >>> reference to >>> >>> `JvmtiEnvThreadState::set_frame_pop(int)' >>> /tmp/cc27HS5M.ltrans0.ltrans.o: In >>> function >>> `VM_GetFrameLocation::doit() >>> [clone .local.641]': >>> ... (rest of output omitted) >>> >>> >>> My configuration: >>> >>> --with-jdk-variant=normal \ >>> --with-jvm-variants=minimal \ >>> --with-debug-level=release \ >>> --disable-warnings-as-errors \ >>> --disable-hotspot-gtest \ >>> --with-abi-profile=arm-vfp-sflt \ >>> --openjdk-target=$(GNU_TARGET_NAME) >>> \ >>> --with-sys-root=$(STAGING_DIR) \ >>> --with-tools-dir=$(HOST_DIR) \ >>> >>> --with-freetype-include=$(STAGING_DIR)/usr/include \ >>> >>> --with-freetype-lib=$(STAGING_DIR)/usr/lib \ >>> --with-freetype=$(STAGING_DIR)/usr/ >>> \ >>> >>> --with-alsa-include=$(STAGING_DIR)/usr/include \ >>> >>> --with-alsa-lib=$(STAGING_DIR)/usr/lib \ >>> --with-alsa=$(STAGING_DIR)/usr/ \ >>> --with-cups=$(STAGING_DIR)/usr/ \ >>> --with-x=$(STAGING_DIR)/usr/include >>> \ >>> >>> --with-extra-ldflags=--sysroot=$(STAGING_DIR) \ >>> --enable-headless-only \ >>> --disable-freetype-bundling \ >>> --enable-unlimited-crypto \ >>> >>> --with-boot-jdk=/opt/java/jdk1.8.0_102 >>> >>> >>> >>> >>> >>> From ioi.lam at oracle.com Mon Mar 6 19:16:41 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 06 Mar 2017 11:16:41 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> Message-ID: <58BDB599.7050505@oracle.com> Hi Thomas, On 3/6/17 8:51 AM, Thomas St?fe wrote: > Hi Coleen and Ioi, > > I had to port C++ code to platforms with terrible compilers for a time > in my life, that is why I like code to be as portable as possible. > That said, you are right in your argumentation, the SafeFetch solution > is not terribly elegant and Ioi's original way of determining the > vtable size is cleaner. > > I did some checks on some of our architectures with a test similar to > Ioi's and on a first glance it seems to work for simple cases (single > and public inheritance) on ppc (AIX) and Linux ia64. Although the > vtables seemed to me to contain function descriptors, not real > pointers to code, so this is something to keep in mind. But if the > live vtable are copied, the function descriptors they contain should > point to valid code too, so it should not matter. Just to remember to > not expect every slot in the array to be a valid code pointer. > I also notice on our current Solaris build the first slot in the vtable is the RTTI pointer, not a function address. This slot is different in the TesterA and TesterB classes, so I had to skip over it. I think this is safe because Metadata has about 10 virtual methods, so skipping over the first one should not cause any harm. > So, in short, I remove my objection to Ioi's original solution, as far > as that matters. > > I still think we rely on a lot here: Contiguous vtable containing > absolute memory addresses, vtable pointer at start of object and > vtable entries to be ordered from base->derived class. So I wonder how > much effort it would be (now or in the future as a separate change) to > have a fallback where - at loading time - instead of copying vtables > the vtable pointers in the objects were fixed up to point to the new > live vtables? I know this would be more expensive and potentially > defeat the point of shared classes. But maybe not, it depends on how > many objects are there, no? I think for portability, perhaps we can add a porting layer so the vtables can be copied in a different way? I think this is better than patching each object at run time. Also, I would suggest not adding such a layer now, but do it when such a need actually arises. Thanks - Ioi > > Kind Regards, Thomas > > On Sun, Mar 5, 2017 at 4:17 PM, > wrote: > > > Ioi, Some comments inline (where no comments, insert "ok") :) > > > On 3/2/17 10:37 PM, Ioi Lam wrote: > > Hi Coleen, > > Thanks for the comments. I have updated the webrev. See > in-line for responses. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ > > > > On 3/2/17 8:48 AM, coleen.phillimore at oracle.com > wrote: > > > Ioi > I like the concept of this a lot but have some stylistic > comments to help people reading this code later. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html > > > s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't > save many characters, especially in CppVtableInfo/Testers > > Done. > > + // Start at slot 1, because slot 0 may be RTTI (on > Solaris/Sparc) > + int i; > + for (i=1; ; i++) { > Since you're using 'i' later, can you rename it to > something descriptive. Or have another variable > "vtable_length" to use later. This looks like an old > style for loop. > > Done > > Can the functions for CppVtableInfo be declared outside of > the class declaration? They don't need to be inline and > then the debug code for testing the vtable size can be not > in the middle of the class declaration. Then you can > move the Tester classes to inside the same #ifndef PRODUCT > block. > > Can you put #endif // PRODUCT when the ifdef covers > several lines of code? > > Done > > vtab_of could be more descriptive, like cpp_vtable_for(). > > I changed to vtable_of(). Because the class name is already > CppVtableCloner, repeating the word "cpp" seems repetitive to me. > > Was PrintSharedSpaces was never converted to UL? > > Right. I've filed > https://bugs.openjdk.java.net/browse/JDK-8176132 > > (-XX:+PrintSharedSpaces should be converted to use Unified > Logging.) > > + int n = MAX_VTABLE_SIZE; > > Can you propagate MAX_VTABLE_SIZE to the places where it's > used. n isn't descriptive. This starts out with > max_vtable_size and then changes the size. Reusing 'n' > makes this really hard to follow. Not having a comment > that we only allocate enough slots for the vtable makes it > hard too. > > + // allocate CppVtableInfo in the MD section + _info = > (CppVtabInfo*)md_top; > + _info->set_vtab_size(n); // initially set to max_vtable_size > + + // allocate temporary local instance of the metadata > type T + T tmp; > + intptr_t* srcvtab = vtab_of(tmp); > + intptr_t* dstvtab = _info->vtab(); > + > > Fixed. > > Something like that for comments. dstvtab is the > destination_vtable in the MD section. > > > I've dropped the md_ prefix from the functions that deal with > the vtables, since they shouldn't care whether it's the "MD" > section or not. Now it looks like this: > > // Allocate and initialize the C++ vtables, starting from top, > but do not go past end. > intptr_t* > MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, > intptr_t* end) { > assert(DumpSharedSpaces, "dump-time only"); > // Layout (each slot is a intptr_t): > // [number of slots in the first vtable = n1] > // [ slots for the first vtable] > // [number of slots in the first second = n2] > // [ slots for the second vtable] > // ... > // The order of the vtables is the same as the > CPP_VTAB_PATCH_TYPES_DO macro. > CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); > return top; > } > > + for (int i=0; i + const intptr_t bad = intptr_t(0xdeadbeef); > + intptr_t num = SafeFetchN(&srcvtab[i], bad); > + if (num == bad > + // || i > 120 /* uncomment this line to test */ > + ) { > + _info->set_vtab_size(i-1); > + break; > + } > + dstvtab[i] = num; > + } > I dont understand this code. You get deadbeef for a bad > value if SafeFetchN gets a fault but why would it get a > fault at the end of the metadata's vtable? Couldn't it > just run onto the next vtable? I think your original way > of counting vtable entries might be better (sorry I didn't > have time to study that thread). > > I've modified the comments to this. Does it make sense to you? > > // It is not always safe to call memcpy(), because > srcvtable might be shorter than > // MAX_VTABLE_SIZE, and the C++ linker might have placed > the vtable at the very > // end of the last page of libjvm.so. Crossing over to the > next page might > // cause a page fault. > > My fear is the JVM would suddenly start crashing because the > order of .o files have changed on the linker's command line, > or if you enable some special linker optimization flags. It's > better safe than sorry. > > > This wasn't exactly what I was not understanding. I didn't see > that you are copying 120 entries from the old vtable and junk > memory beyond the old vtable, unless you get a segv, in which case > you copy less. I don't think you should copy random memory into > the vtable in the archive. This doesn't seem secure, even with > the segv protection. > > Since we already have assumptions about C++ vtable layout in the > code and it's mostly specified by various ABIs, and you have the > assert code, I think I would prefer that you copy only the vtable > entries into the archive. I guess Thomas Stuefe had a different > opinion. I've read the original thread. Two points: > > If new C++ compiler implementations add a discontigous vtable, > both the SafeFetchN and subclass additional virtual function at > end implementation will fail. I don't think C++ implementations > would do this and a contiguous vtable as first in the instance has > been standard for years. If our metadata adds multiple > inheritance, the same issue would be a problem for both > implementations, as well as for the implementation we have before > Ioi's fix. > > Ioi's subclass adding virtual function method would work for any > esoteric C++ implementations in my memory, except the vptr for the > old DECC++ compiler was after the nonstatic data members (which > would fail with all of our implementations). > > Since the code is there anyway for debug purposes, we're not > saving code by implementing SafeFetchN. The SafeFetchN > implementation isn't obvious at all what it's doing, and requires > better comments, especially if you don't know already what > SafeFetchN does. It looks really cryptic. The poisoned values > also bothered me in that they overload other poisoned values in > other parts of the jvm. > > Ioi, could you make all methods of CppVtableCloner out of line? > > The other changes look good, although I might have more requests > for comments. > > Thanks, > Coleen > > > Would be nice to have comments here too!! > > + intptr_t* start = md_top; > > This doesn't do anything (?) > > > Fixed. This was left over code. > > > + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); > > Why not zero the destination vtable in allocate? Or does > patching the vtable pointers call virtual functions? You > could prevent that so you don't need this code. > > I added this comment: > > // During patching, some virtual methods may be called, so > at this point > // the vtables must contain valid methods (as filled in by > CppVtableCloner::allocate). > MetaspaceShared::patch_cpp_vtable_pointers(); > > // The vtable clones contain addresses of the current process. > // We don't want to write these addresses into the archive. > MetaspaceShared::zero_cpp_vtable_clones_for_writing(); > > + // Restore the vtable in case we invoke any virtual methods. > + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); > Can this be restore_cpp_vtables since that's what it's > doing. The first is after the dump and the second call is > at UseSharedSpaces. A couple of comments in this > clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: > > I prefer to use the word clone. Otherwise when you just say > "vtable" it's not clear whether you're talking about the > original one (made by the c++ linker), or the cloned one in > the CDS archive. > > + static intptr_t* clone_vtable(const char* name, > intptr_t* p) { > + T tmp; // Allocate temporary dummy metadata object to > get vtable initialized > + CppVtabInfo* info = (CppVtabInfo*)p; > + int n = info->vtab_size(); > + intptr_t* srcvtab = vtab_of(tmp); > + intptr_t* dstvtab = info->vtab(); > + > + // We already checked (and, if necessary, adjusted n) > when the vtables were allocated, so we are > + // safe to do memcpy. > + if (PrintSharedSpaces) { > + tty->print_cr("%s copying %d vtable entries", name, n); > + } > + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); > + return dstvtab + n; > + } > > Done. I changed the wording > > T tmp; // Allocate temporary dummy metadata object to get > to the original vtable. > > As we are not really "initializing a vtable" here. > > Same with 'patch'. It'd be so much faster and easier to > read this code with more comments please. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html > > > Why are these testers here? > > > I updated the comment: > > // Used by CDS. These classes need to access the private > ConstantPool() constructor. > template friend class CppVtableTesterA; > template friend class CppVtableTesterB; > template friend class CppVtableCloner; > > > Thanks > - Ioi > > > On 3/1/17 3:25 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8005165 > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ > > > Hi, > > This is the official review (follow up of the > "Determining the size of C++ vtables" thread > onhotspot-dev at openjdk.java.net > ). > > The new code has the same assumption as the > existing code in JDK 10: for a C++ object that > contains virtual methods (e.g., ConstantPool), > we assume the first intptr_t slot of the > object is a _vptr, which points to a vtable, > which consists of no more than 150 intptr_t's. > > ConstantPool*p -->[ _vptr ] -------> [ > vtable slot 0 ] > [ field #0 ] [ > vtable slot 1 ] > [ field #1 ] [ > vtable slot 2 ] > [ field #2 ] [ > vtable slot 3 ] > [ .... ] [ > vtable slot 4] > [ > vtable slot 5 ] > [ > ... ] > > + In the existing code, we were pointing the > vtable slots to > code that's generated by HotSpot. > > + In the new code, we copy the vtable slots > from an existing > vtable (generated by the C++ linker). > > Per Thomas St?fe's advice, I don't try to > determine the size of the vtable (as that > would add one more compiler requirement where > new virtual methods added by a subclass must > be placed at a higher offset in the vtable). > > Instead, I have added code in non-product > builds to ensure that the vtables are no > longer than 150 entries. You can run with > "-XX:+PrintSharedSpaces -Xshare:dump" to print > out the actual size of the vtables for your > particular platform: > > ConstantPool has 12 virtual methods > InstanceKlass has 113 virtual methods > InstanceClassLoaderKlass has 113 virtual methods > InstanceMirrorKlass has 113 virtual methods > InstanceRefKlass has 113 virtual methods > Method has 12 virtual methods > ObjArrayKlass has 114 virtual methods > TypeArrayKlass has 114 virtual methods > > As mentioned in the code comments, if you have > an esoteric C++ compiler, the > verify_sufficient_size() function will > probably fail, but hopefully that would give > you some hints for porting this code. > > To avoid accidentally touching an unmapped > page, the code uses SafeFetchN for copying the > vtable contents, and would shrink the vtable > to less than 150 entries if necessary. I can't > test this for real, but I've added some code > to simulate an error: > > for (int i=0; i const intptr_t bad = intptr_t(0xdeadbeef); > intptr_t num = SafeFetchN(&srcvtab[i], bad); > if (num == bad > // || i > 120 /* uncomment this line > to test */ > ) { > _info->set_vtab_size(i-1); > break; > } > dstvtab[i] = num; > } > > Results: > > + Removed 850 lines of CPU-dependent code > > + CDS image is about 50K smaller > > + Previously Metadata objects must live in the > read-write section in the CDS > archive, because their _vptr was updated at > run time. Now _vptr is no longer > updated, so ConstantPool can be moved to the > read-only section (see JDK-8171392). > > Thanks > - Ioi > > > > > > > From christian.tornqvist at oracle.com Mon Mar 6 21:24:05 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Mon, 6 Mar 2017 16:24:05 -0500 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* In-Reply-To: References: <169f01d29434$a84a7d70$f8df7850$@oracle.com> <86ccedc2-37e1-9360-f0d5-5ed2338b0c8d@oracle.com> <310c01d2968f$61e0d520$25a27f60$@oracle.com> Message-ID: <336c01d296c0$004b5570$00e20050$@oracle.com> Hi Vladimir, That would be misleading though, it's used a lot more than just in tier2. Thanks, Christian -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Monday, March 6, 2017 11:59 AM To: Christian Tornqvist ; 'David Holmes' ; hotspot-dev at openjdk.java.net Subject: Re: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* Hi Christian, As I remember hotspot_not_fast_compiler is used in later tiers. Can we just name it accordingly, for example: hotspot_tier2_compiler? Thanks, Vladimir On 3/6/17 7:36 AM, Christian Tornqvist wrote: > Hi David, > > Not sure what that name would be though, the compiler is using this > group with flag rotation in multiple tiers. I'll leave the renaming of > this group to the compiler team to sort out. > > Thanks, > Christioan > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Friday, March 3, 2017 9:21 PM > To: Christian Tornqvist ; > hotspot-dev at openjdk.java.net > Subject: Re: RFR(S): 8176102 - Rename hotspot_fast* test groups to > hotspot_tier1* > > Hi Christian, > > Rename looks good. > > Should we also rename the now oddly named hotspot_not_fast_compiler? > > Thanks, > David > > On 4/03/2017 1:41 AM, Christian Tornqvist wrote: >> Hi everyone, >> >> >> >> Please review this small change that renames the hotspot_fast* test >> groups to hotspot_tier1, also renamed the hotspot_runtime_tier* to >> follow the same naming standard. >> >> Tested by running the change through JPRT. >> >> >> >> Webrevs: >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00 >> / >> >> >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8176102 >> >> >> >> Thanks, >> >> Christian >> > From chris.plummer at oracle.com Mon Mar 6 21:54:59 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 6 Mar 2017 13:54:59 -0800 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> <1d2586eb-0257-a330-2ff6-5028de510b4b@oracle.com> Message-ID: I haven't been following this too closely, but one thing to consider is that you are not getting proper floating point calling conventions. You mentioned your configure target ABI is arm-vfp-sflt. This means you have VFP support, but use the old softfloat calling conventions (FP values passed in GPRs). Most platforms these days with VFP support use the new hardfloat calling conventions (FP values passed in FPRs). So this could simply be the case of building the wrong binary for your platform. - Use arm-vfp-hflt is you have VFP support and want to use hard float calling conventions. - Use arm-vfp-sflt if you have VFP support and want to use the older soft float calling conventions. - Use arm-sflt is you have a true soft float platform. There's also a very old arm sflt ABI that I don't think we support. The main difference from arm-sflt is the word endianess of of 64-bit values. I believe the presence of -mfpu=vfp is what is used to get the more modern endianess, which to say the least is very non-obvious option for doing this, but is the reason you will see -mfpu=vfp for all our supported arm platforms (arm-vfp-hflt, arm-vfp-sflt, and arm-sflt). Here's the logic from the configure script: if test "x$ARM_FLOAT_TYPE" = xvfp-sflt; then ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=softfp -mfpu=vfp -DFLOAT_ARCH=-vfp-sflt' elif test "x$ARM_FLOAT_TYPE" = xvfp-hflt; then ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=hard -mfpu=vfp -DFLOAT_ARCH=-vfp-hflt' elif test "x$ARM_FLOAT_TYPE" = xsflt; then ARM_FLOAT_TYPE_FLAGS='-msoft-float -mfpu=vfp' fi BTW, for the arm-sflt case, FLOAT_ARCH gets set internally to "-sflt". I'm not sure why for arm-sflt it's handled in the source and the other two platforms set it in the configure script. cheers, Chris On 3/5/17 8:30 PM, Zhu Yong wrote: > Sometime later this week, I might get newer version of Marvell toolchain, I > will do a build and test again then. > I will update this thread when I get result. > > Thank you very much for helping. > > On Mon, Mar 6, 2017 at 9:09 AM, David Holmes > wrote: > >> On 3/03/2017 6:59 PM, Zhu Yong wrote: >> >>> Thank you very much for helping. >>> >>> Today I tried to debug the test code, but without luck, still can't >>> figure out why. >>> >>> First, I followed the instruction >>> http://stackoverflow.com/questions/29916995/intellij-idea- >>> remotely-debug-java-console-program, >>> changed suspend=y because the test code is short, it will exit before >>> debug start if not suspend first. Unfortunately, with remote debug, I >>> was not able to reproduce the issue. Same class file, directly run, will >>> produce the issue, remote debug won't. >>> >>> Then, I tried *jdb *from my OpenJDK9 client vm, run directly on my >>> target system to debug the code. If I set breakpoint, program run >>> without issue, however, if don't set breakpoint, just "run" in jdb, >>> problem reproduced. >>> >> This suggests to me that a floating-point register is not being preserved >> when needed. When you run with the breakpoint you will get full >> save/restore of all the register state. Just a theory. >> >> Next, I tried to use *javap *to decompile the output class file, run >>> OpenJDK9 client vm javap and host side javap on same class file, the >>> output is same for getTime function. >>> >> It is a native code issue not a bytecode issue, so javap wont help here. >> >> One interesting point I noticed today is the output like below when >>> problem happen, 2nd call of getTime() has value of 1st call result / >>> 1000. Looks like 2nd call getTime(), double variable q was not updated >>> startTime: 284092.063000 endTime: 284.092063 runTime: -283807.970937 >>> >>> What should I do so that I can come a conclusion that it's due to >>> compiler issue? >>> >> Can you get access to a different gcc to build with? >> >> David >> ----- >> >> >>> On Thu, Mar 2, 2017 at 12:32 PM, David Holmes >> > wrote: >>> >>> I built your sample program with latest JDK 9 for arm-vfp-sflt and >>> ran on an emulator I have available, and I had no issues. >>> >>> So I still have concerns this may be a build/compiler issue. >>> >>> David >>> >>> >>> On 2/03/2017 12:51 PM, David Holmes wrote: >>> >>> I've moved this discussion over to hotspot-dev as this seems no >>> longer a >>> build issue but a C1 soft-float issue. >>> >>> David >>> >>> On 2/03/2017 12:35 PM, Zhu Yong wrote: >>> >>> If run with -Xint, it works. >>> >>> I have created simplified FP test by remove all non-related >>> code from >>> Whetstone test code. >>> The test code is here: >>> https://gist.github.com/yongzhy/d65c26d39fe5d549d1b406c7c84e >>> acd4 >>> >> eacd4> >>> >>> I am not sure if the test code can help or not. The >>> behaviour is weird : >>> - If I print both itime and q, program will run OK >>> - inside condition block if(q<1.0f), if I use exit code >>> 2,3,4,5, problem >>> appears, however, if I use exit code >=6, program run OK. >>> >>> I always get exit code 5, the line q = (double)itime is wrong? >>> >>> public static double getTime() >>> { >>> double q; >>> long itime; >>> itime = System.currentTimeMillis(); >>> if(itime<1000000) { >>> System.exit(1); >>> } >>> //System.out.printf("time long value %d\n", >>> itime); >>> q = (double) itime; >>> if(q < 1.0f) { >>> System.exit(5); // I always get exit code 5 >>> } >>> //System.out.printf("time float value %f\n", q); >>> return q / 1000.0; >>> } >>> >>> >>> >>> On Wed, Mar 1, 2017 at 5:31 PM, David Holmes >>> >>> >> >> wrote: >>> >>> On 1/03/2017 6:45 PM, Zhu Yong wrote: >>> >>> OK, for the Whetstone Benchmark, I added debug >>> printing and >>> found it's >>> due to function getTime(), it only get good value on >>> 1st call, >>> all >>> following calls get 0s. >>> Debug printing of itime value is correct. So reason >>> should be >>> the return >>> division line. Could it because toolchain's floating >>> point >>> operation??? >>> But why server VM Ok? >>> >>> >>> Server and client implement FP logic differently, and >>> particularly >>> as this is soft-fp, they may well not be in sync. Does >>> -Xint work? >>> >>> Can you isolate to a simple FP test? >>> >>> David >>> >>> public static double getTime() >>> { >>> double q; >>> long itime; >>> itime = System.currentTimeMillis(); >>> q = (double) itime; >>> return q / 1000.0; >>> } >>> >>> On Wed, Mar 1, 2017 at 3:14 PM, David Holmes >>> >> >>> >> > >>> >> >>> >> >>> wrote: >>> >>> On 1/03/2017 4:26 PM, Zhu Yong wrote: >>> >>> We use armv7-marvell-linux-gnueabi-softfp >>> toolchain >>> >>> gcc version 4.6.4 (Marvell GCC release >>> 20150204-c4af733b 64K >>> MAXPAGESIZE >>> ALIGN CVE-2015-0235) >>> >>> Is this toolchain too old? I am asking >>> because I saw >>> other strange >>> issues as well: >>> >>> >>> We last used gcc 4.7.2. I can't really say if >>> 4.6.4 is "too >>> old". >>> >>> I have successfully build server, client and >>> minimal VM, >>> when I run >>> Dhrystone benchmark file (the same jar file >>> from >>> http://www.okayan.jp/DhrystoneApplet/ >>> >>> >> > >>> >> >>> >> >>), the performance >>> from >>> server VM >>> is much better than client and minimal VM. >>> (minimal: 1629852, client: 1615508, server: >>> 2338871 ) >>> >>> >>> That's expected. The server VM is high >>> performance. >>> >>> And when run Whetstone Benchmark >>> from >>> >>> http://www.roylongbottom.org.uk/online/whetjava2.html >>> >>> >>> >> > >>> >>> >> >>> >>> >> >>, >>> server VM >>> finished with good result, client and >>> minimal VM not >>> able to finish >>> (program stuck there forever after print 1st >>> line of >>> output). >>> >>> >>> That needs investigating. You need to try and >>> generate a >>> stackdump >>> to see where things are stuck. >>> >>> David >>> >>> >>> On Wed, Mar 1, 2017 at 1:56 PM, David Holmes >>> >> >>> >> > >>> >> david.holmes at oracle.com> >>> >> >> >>> >> >>> >> > >>> >> >>> >> >>>> wrote: >>> >>> On 1/03/2017 3:39 PM, Zhu Yong wrote: >>> >>> Thank you for the information, I >>> managed to make >>> minimal >>> build >>> pass now. >>> >>> Initially I though JVM_EXCLUDE_FILES >>> need to add >>> additional 3 >>> generated >>> files (they appeared >>> in _BUILD_LIBJVM_objectfilenames.txt) >>> : >>> bytecodeInterpreterWithChecks.cpp >>> jvmtiEnter.cpp >>> jvmtiEnterTrace.cpp >>> But build still fail with same error >>> message. >>> >>> Finally I figured out it's due to >>> those 8 doit() >>> functions >>> implementation in jvmtiEnvBase.hpp >>> file. After >>> move all >>> those 8 >>> doit() >>> implementations to jvmtiEnvBase.cpp, >>> build of >>> minimal VM >>> passed >>> without >>> any issue. >>> >>> >>> That seems to be a compiler issue. >>> jvmtiEnvBase.hpp is >>> unconditionally included in a couple of >>> places >>> because we >>> have to >>> have enough of the JVMTI code to say >>> JVMTI is not >>> available. >>> Those >>> doit() implementations, though arguably >>> could be >>> conditional on >>> INCLUDE_JVMTI, are not referenced by any >>> called code >>> and so >>> should >>> not linked in. But in your case it seems >>> they are. >>> >>> What toolchain are you using? >>> >>> David >>> ----- >>> >>> Changes appended >>> ============= >>> >>> --- >>> .../src/share/vm/prims/jvmtiE >>> nvBase.cpp >>> | 74 >>> ++++++++++++++++++++++ >>> .../src/share/vm/prims/jvmtiE >>> nvBase.hpp >>> | 68 >>> +++----------------- >>> 2 files changed, 82 insertions(+), 60 >>> deletions(-) >>> mode change 100644 => 100755 >>> >>> hotspot-9211c2e89c1c/src/share >>> /vm/prims/jvmtiEnvBase.cpp >>> mode change 100644 => 100755 >>> >>> hotspot-9211c2e89c1c/src/share >>> /vm/prims/jvmtiEnvBase.hpp >>> >>> diff --git >>> >>> >>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> >>> >>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> old mode 100644 >>> new mode 100755 >>> index dd241a0..e5832ba >>> --- >>> >>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> +++ >>> >>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>> @@ -1283,6 +1283,80 @@ >>> >>> VM_GetMultipleStackTraces::all >>> ocate_and_fill_stacks(jint >>> thread_count) { >>> "the last copied frame >>> info must be >>> the last >>> record"); >>> } >>> >>> +void >>> +VM_UpdateForPopTopFrame::doit() { >>> + JavaThread* jt = >>> _state->get_thread(); >>> + if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + _state->update_for_pop_top_fra >>> me(); >>> + } else { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + } >>> +} >>> + >>> +void >>> +VM_SetFramePop::doit() { >>> + JavaThread* jt = >>> _state->get_thread(); >>> + if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + int frame_number = >>> _state->count_frames() - >>> _depth; >>> + >>> >>> >>> >>> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ >>> pop(frame_number); >>> >>> + } else { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + } >>> +} >>> + >>> +void >>> +VM_GetOwnedMonitorInfo::doit() { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if >>> (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() >>> + >>> && >>> _java_thread->threadObj() != >>> NULL) { >>> + _result = ((JvmtiEnvBase >>> >>> *)_env)->get_owned_monitors(_calling_thread, >>> _java_thread, >>> + >>> _owned_monitors_list); >>> + } >>> +} >>> + >>> +void >>> +VM_GetObjectMonitorUsage::doit() { >>> + _result = ((JvmtiEnvBase*) >>> >>> _env)->get_object_monitor_usage(_calling_thread, >>> _object, >>> _info_ptr); >>> +} >>> + >>> +void >>> +VM_GetCurrentContendedMonitor::doit() >>> { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if >>> (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() && >>> + _java_thread->threadObj() != >>> NULL) { >>> + _result = ((JvmtiEnvBase >>> >>> >>> >>> *)_env)->get_current_contended_monitor(_calling_thread,_ >>> java_thread,_owned_monitor_ptr); >>> >>> + } >>> +} >>> + >>> +void >>> +VM_GetStackTrace::doit() { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if >>> (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() >>> + >>> && >>> _java_thread->threadObj() != >>> NULL) { >>> + _result = ((JvmtiEnvBase >>> *)_env)->get_stack_trace(_java_thread, >>> + >>> _start_depth, >>> _max_count, >>> + >>> _frame_buffer, >>> _count_ptr); >>> + } >>> +} >>> + >>> +void >>> +VM_GetFrameCount::doit() { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + JavaThread* jt = >>> _state->get_thread(); >>> + if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + _result = >>> ((JvmtiEnvBase*)_env)->get_fra >>> me_count(_state, >>> _count_ptr); >>> + } >>> +} >>> + >>> +void >>> +VM_GetFrameLocation::doit() { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if >>> (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() && >>> + _java_thread->threadObj() != >>> NULL) { >>> + _result = >>> >>> ((JvmtiEnvBase*)_env)->get_fra >>> me_location(_java_thread, >>> _depth, >>> + >>> _method_ptr, >>> _location_ptr); >>> + } >>> +} >>> >>> void >>> VM_GetThreadListStackTraces::doit() >>> { >>> diff --git >>> >>> >>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> >>> >>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> old mode 100644 >>> new mode 100755 >>> index 04e6869..00b9890 >>> --- >>> >>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> +++ >>> >>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>> @@ -359,14 +359,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_UpdateForPopTopFrame; } >>> jvmtiError result() { return >>> _result; } >>> - void doit() { >>> - JavaThread* jt = >>> _state->get_thread(); >>> - if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() >>> != NULL) { >>> - >>> _state->update_for_pop_top_frame(); >>> - } else { >>> - _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> // VM operation to set frame pop. >>> @@ -389,15 +382,7 @@ public: >>> bool allow_nested_vm_operations() >>> const { >>> return true; } >>> VMOp_Type type() const { return >>> VMOp_SetFramePop; } >>> jvmtiError result() { return >>> _result; } >>> - void doit() { >>> - JavaThread* jt = >>> _state->get_thread(); >>> - if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() >>> != NULL) { >>> - int frame_number = >>> _state->count_frames() >>> - _depth; >>> - >>> >>> >>> >>> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ >>> pop(frame_number); >>> >>> - } else { >>> - _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> >>> @@ -421,14 +406,7 @@ public: >>> _result = JVMTI_ERROR_NONE; >>> } >>> VMOp_Type type() const { return >>> VMOp_GetOwnedMonitorInfo; } >>> - void doit() { >>> - _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - if >>> (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() >>> - >>> && >>> _java_thread->threadObj() != >>> NULL) { >>> - _result = ((JvmtiEnvBase >>> >>> *)_env)->get_owned_monitors(_calling_thread, >>> _java_thread, >>> - >>> _owned_monitors_list); >>> - } >>> - } >>> + void doit(); >>> jvmtiError result() { return >>> _result; } >>> }; >>> >>> @@ -451,10 +429,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_GetObjectMonitorUsage; } >>> jvmtiError result() { return >>> _result; } >>> - void doit() { >>> - _result = ((JvmtiEnvBase*) >>> >>> _env)->get_object_monitor_usage(_calling_thread, >>> _object, >>> _info_ptr); >>> - } >>> - >>> + void doit(); >>> }; >>> >>> // VM operation to get current >>> contended >>> monitor. >>> @@ -475,13 +450,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_GetCurrentContendedMonitor; } >>> jvmtiError result() { return >>> _result; } >>> - void doit() { >>> - _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - if >>> (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() && >>> - _java_thread->threadObj() >>> != NULL) { >>> - _result = ((JvmtiEnvBase >>> >>> >>> >>> *)_env)->get_current_contended_monitor(_calling_thread,_ >>> java_thread,_owned_monitor_ptr); >>> >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> // VM operation to get stack trace >>> at safepoint. >>> @@ -508,15 +477,7 @@ public: >>> } >>> jvmtiError result() { return >>> _result; } >>> VMOp_Type type() const { return >>> VMOp_GetStackTrace; } >>> - void doit() { >>> - _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - if >>> (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() >>> - >>> && >>> _java_thread->threadObj() != >>> NULL) { >>> - _result = ((JvmtiEnvBase >>> *)_env)->get_stack_trace(_java >>> _thread, >>> - >>> _start_depth, >>> _max_count, >>> - >>> _frame_buffer, >>> _count_ptr); >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> // forward declaration >>> @@ -606,13 +567,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_GetFrameCount; } >>> jvmtiError result() { return >>> _result; } >>> - void doit() { >>> - _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - JavaThread* jt = >>> _state->get_thread(); >>> - if (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() >>> != NULL) { >>> - _result = >>> ((JvmtiEnvBase*)_env)->get_fra >>> me_count(_state, >>> _count_ptr); >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> // VM operation to frame location >>> at safepoint. >>> @@ -636,14 +591,7 @@ public: >>> } >>> VMOp_Type type() const { return >>> VMOp_GetFrameLocation; } >>> jvmtiError result() { return >>> _result; } >>> - void doit() { >>> - _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> - if >>> (Threads::includes(_java_thread) && >>> !_java_thread->is_exiting() && >>> - _java_thread->threadObj() >>> != NULL) { >>> - _result = >>> >>> ((JvmtiEnvBase*)_env)->get_fra >>> me_location(_java_thread, >>> _depth, >>> - >>> _method_ptr, >>> _location_ptr); >>> - } >>> - } >>> + void doit(); >>> }; >>> >>> >>> -- >>> 2.1.4 >>> >>> >>> On Tue, Feb 28, 2017 at 6:11 PM, >>> David Holmes >>> >> >>> >> > >>> >> >>> >> >> >>> >> >>> >> > >>> >> >>> >> >>> >>> >> >>> >> > >>> >> >>> >> >> >>> >>> >> >>> >> > >>> >> >>> >> >>>>> wrote: >>> >>> On 28/02/2017 7:35 PM, David >>> Holmes wrote: >>> >>> On 28/02/2017 6:51 PM, Zhu >>> Yong wrote: >>> >>> Dear All, >>> >>> I am testing cross compile >>> OpenJDK-9+158 for our >>> embedded >>> system using >>> buildroot, I can build >>> server and >>> client >>> variants >>> successfully, but the >>> output compact1 profile >>> file size >>> is too >>> big, then I >>> tried >>> to build >>> minimal >>> variant, failed when >>> linking >>> libjvm.so. >>> >>> I checked >>> >>> >>> >>> >>> build/linux-arm-normal-minimal-release/hotspot/variant- >>> minimal/ >>> >>> libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt, >>> jvmtiEnvBase.o >>> and >>> jvmtiEnvThreadState.o are not >>> listed in >>> the file >>> (which >>> is required >>> from the error message >>> below). >>> >>> >>> Right - JVM TI is not part >>> of the >>> Minimal VM. At the >>> moment it looks >>> like some service has either >>> been >>> enabled when >>> it should not >>> have been, >>> or has not been correctly >>> if'def in the >>> source. >>> >>> >>> As far as I can see your error >>> messages are >>> complaining about >>> missing functions that are >>> called from the >>> same >>> sources as the >>> functions that are missing! ie. >>> >>> undefined reference to >>> >>> >>> `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, >>> JavaThread*, >>> _jobject**)' >>> /tmp/cc27HS5M.ltrans0.ltrans.o: >>> In function >>> `VM_GetOwnedMonitorInfo::doit() >>> >>> but VM_GetOwnedMonitorInfo is >>> defined in >>> jvmtiEnv.cpp which >>> should >>> be included or excluded the same >>> as >>> jvmtiEnBase.cpp. >>> Here's the >>> logic in the makefile that >>> controls this: >>> >>> ifneq ($(call check-jvm-feature, >>> jvmti), >>> true) >>> JVM_CFLAGS_FEATURES += >>> -DINCLUDE_JVMTI=0 >>> JVM_EXCLUDE_FILES += >>> jvmtiGetLoadedClasses.cpp >>> jvmtiThreadState.cpp >>> jvmtiExtensions.cpp \ >>> jvmtiImpl.cpp >>> jvmtiManageCapabilities.cpp >>> jvmtiRawMonitor.cpp >>> jvmtiUtil.cpp jvmtiTrace.cpp \ >>> jvmtiCodeBlobEvents.cpp >>> jvmtiEnv.cpp >>> jvmtiRedefineClasses.cpp >>> jvmtiEnvBase.cpp >>> jvmtiEnvThreadState.cpp \ >>> jvmtiTagMap.cpp >>> jvmtiEventController.cpp >>> evmCompat.cpp >>> jvmtiEnter.xsl jvmtiExport.cpp \ >>> >>> jvmtiClassFileReconstituter.cpp >>> endif >>> >>> Can you run with LOG=trace so >>> that each >>> compiled file is >>> listed in >>> the build log, then see if there >>> are any >>> jvmti* >>> files listed >>> there. >>> >>> Thanks, >>> David >>> >>> >>> >>> >>> Can you provide the lines >>> from your >>> spec.gmk >>> that define the >>> JVM_FEATURES_* variables >>> please. >>> >>> Thanks, >>> David >>> ------ >>> >>> >>> === Output from failing >>> command(s) >>> repeated >>> here === >>> * For target >>> >>> >>> hotspot_variant-minimal_libjvm_objs_BUILD_LIBJVM_link: >>> >>> /tmp/cc27HS5M.ltrans0.ltrans.o: In >>> function >>> `VM_GetStackTrace::doit() >>> [clone .local.640]': >> From ioi.lam at oracle.com Tue Mar 7 01:25:23 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 06 Mar 2017 17:25:23 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> Message-ID: <58BE0C03.1040105@oracle.com> Hi Thomas & Coleen, Thanks for your comments. I have updated the rev: http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ [1] Switched back to computing the exact vtable size [2] Move non-trivial functions outside of their class declaration Thanks - Ioi On 3/6/17 8:51 AM, Thomas St?fe wrote: > Hi Coleen and Ioi, > > I had to port C++ code to platforms with terrible compilers for a time > in my life, that is why I like code to be as portable as possible. > That said, you are right in your argumentation, the SafeFetch solution > is not terribly elegant and Ioi's original way of determining the > vtable size is cleaner. > > I did some checks on some of our architectures with a test similar to > Ioi's and on a first glance it seems to work for simple cases (single > and public inheritance) on ppc (AIX) and Linux ia64. Although the > vtables seemed to me to contain function descriptors, not real > pointers to code, so this is something to keep in mind. But if the > live vtable are copied, the function descriptors they contain should > point to valid code too, so it should not matter. Just to remember to > not expect every slot in the array to be a valid code pointer. > > So, in short, I remove my objection to Ioi's original solution, as far > as that matters. > > I still think we rely on a lot here: Contiguous vtable containing > absolute memory addresses, vtable pointer at start of object and > vtable entries to be ordered from base->derived class. So I wonder how > much effort it would be (now or in the future as a separate change) to > have a fallback where - at loading time - instead of copying vtables > the vtable pointers in the objects were fixed up to point to the new > live vtables? I know this would be more expensive and potentially > defeat the point of shared classes. But maybe not, it depends on how > many objects are there, no? > > Kind Regards, Thomas > > On Sun, Mar 5, 2017 at 4:17 PM, > wrote: > > > Ioi, Some comments inline (where no comments, insert "ok") :) > > > On 3/2/17 10:37 PM, Ioi Lam wrote: > > Hi Coleen, > > Thanks for the comments. I have updated the webrev. See > in-line for responses. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ > > > > On 3/2/17 8:48 AM, coleen.phillimore at oracle.com > wrote: > > > Ioi > I like the concept of this a lot but have some stylistic > comments to help people reading this code later. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html > > > s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't > save many characters, especially in CppVtableInfo/Testers > > Done. > > + // Start at slot 1, because slot 0 may be RTTI (on > Solaris/Sparc) > + int i; > + for (i=1; ; i++) { > Since you're using 'i' later, can you rename it to > something descriptive. Or have another variable > "vtable_length" to use later. This looks like an old > style for loop. > > Done > > Can the functions for CppVtableInfo be declared outside of > the class declaration? They don't need to be inline and > then the debug code for testing the vtable size can be not > in the middle of the class declaration. Then you can > move the Tester classes to inside the same #ifndef PRODUCT > block. > > Can you put #endif // PRODUCT when the ifdef covers > several lines of code? > > Done > > vtab_of could be more descriptive, like cpp_vtable_for(). > > I changed to vtable_of(). Because the class name is already > CppVtableCloner, repeating the word "cpp" seems repetitive to me. > > Was PrintSharedSpaces was never converted to UL? > > Right. I've filed > https://bugs.openjdk.java.net/browse/JDK-8176132 > > (-XX:+PrintSharedSpaces should be converted to use Unified > Logging.) > > + int n = MAX_VTABLE_SIZE; > > Can you propagate MAX_VTABLE_SIZE to the places where it's > used. n isn't descriptive. This starts out with > max_vtable_size and then changes the size. Reusing 'n' > makes this really hard to follow. Not having a comment > that we only allocate enough slots for the vtable makes it > hard too. > > + // allocate CppVtableInfo in the MD section + _info = > (CppVtabInfo*)md_top; > + _info->set_vtab_size(n); // initially set to max_vtable_size > + + // allocate temporary local instance of the metadata > type T + T tmp; > + intptr_t* srcvtab = vtab_of(tmp); > + intptr_t* dstvtab = _info->vtab(); > + > > Fixed. > > Something like that for comments. dstvtab is the > destination_vtable in the MD section. > > > I've dropped the md_ prefix from the functions that deal with > the vtables, since they shouldn't care whether it's the "MD" > section or not. Now it looks like this: > > // Allocate and initialize the C++ vtables, starting from top, > but do not go past end. > intptr_t* > MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, > intptr_t* end) { > assert(DumpSharedSpaces, "dump-time only"); > // Layout (each slot is a intptr_t): > // [number of slots in the first vtable = n1] > // [ slots for the first vtable] > // [number of slots in the first second = n2] > // [ slots for the second vtable] > // ... > // The order of the vtables is the same as the > CPP_VTAB_PATCH_TYPES_DO macro. > CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); > return top; > } > > + for (int i=0; i + const intptr_t bad = intptr_t(0xdeadbeef); > + intptr_t num = SafeFetchN(&srcvtab[i], bad); > + if (num == bad > + // || i > 120 /* uncomment this line to test */ > + ) { > + _info->set_vtab_size(i-1); > + break; > + } > + dstvtab[i] = num; > + } > I dont understand this code. You get deadbeef for a bad > value if SafeFetchN gets a fault but why would it get a > fault at the end of the metadata's vtable? Couldn't it > just run onto the next vtable? I think your original way > of counting vtable entries might be better (sorry I didn't > have time to study that thread). > > I've modified the comments to this. Does it make sense to you? > > // It is not always safe to call memcpy(), because > srcvtable might be shorter than > // MAX_VTABLE_SIZE, and the C++ linker might have placed > the vtable at the very > // end of the last page of libjvm.so. Crossing over to the > next page might > // cause a page fault. > > My fear is the JVM would suddenly start crashing because the > order of .o files have changed on the linker's command line, > or if you enable some special linker optimization flags. It's > better safe than sorry. > > > This wasn't exactly what I was not understanding. I didn't see > that you are copying 120 entries from the old vtable and junk > memory beyond the old vtable, unless you get a segv, in which case > you copy less. I don't think you should copy random memory into > the vtable in the archive. This doesn't seem secure, even with > the segv protection. > > Since we already have assumptions about C++ vtable layout in the > code and it's mostly specified by various ABIs, and you have the > assert code, I think I would prefer that you copy only the vtable > entries into the archive. I guess Thomas Stuefe had a different > opinion. I've read the original thread. Two points: > > If new C++ compiler implementations add a discontigous vtable, > both the SafeFetchN and subclass additional virtual function at > end implementation will fail. I don't think C++ implementations > would do this and a contiguous vtable as first in the instance has > been standard for years. If our metadata adds multiple > inheritance, the same issue would be a problem for both > implementations, as well as for the implementation we have before > Ioi's fix. > > Ioi's subclass adding virtual function method would work for any > esoteric C++ implementations in my memory, except the vptr for the > old DECC++ compiler was after the nonstatic data members (which > would fail with all of our implementations). > > Since the code is there anyway for debug purposes, we're not > saving code by implementing SafeFetchN. The SafeFetchN > implementation isn't obvious at all what it's doing, and requires > better comments, especially if you don't know already what > SafeFetchN does. It looks really cryptic. The poisoned values > also bothered me in that they overload other poisoned values in > other parts of the jvm. > > Ioi, could you make all methods of CppVtableCloner out of line? > > The other changes look good, although I might have more requests > for comments. > > Thanks, > Coleen > > > Would be nice to have comments here too!! > > + intptr_t* start = md_top; > > This doesn't do anything (?) > > > Fixed. This was left over code. > > > + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); > > Why not zero the destination vtable in allocate? Or does > patching the vtable pointers call virtual functions? You > could prevent that so you don't need this code. > > I added this comment: > > // During patching, some virtual methods may be called, so > at this point > // the vtables must contain valid methods (as filled in by > CppVtableCloner::allocate). > MetaspaceShared::patch_cpp_vtable_pointers(); > > // The vtable clones contain addresses of the current process. > // We don't want to write these addresses into the archive. > MetaspaceShared::zero_cpp_vtable_clones_for_writing(); > > + // Restore the vtable in case we invoke any virtual methods. > + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); > Can this be restore_cpp_vtables since that's what it's > doing. The first is after the dump and the second call is > at UseSharedSpaces. A couple of comments in this > clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: > > I prefer to use the word clone. Otherwise when you just say > "vtable" it's not clear whether you're talking about the > original one (made by the c++ linker), or the cloned one in > the CDS archive. > > + static intptr_t* clone_vtable(const char* name, > intptr_t* p) { > + T tmp; // Allocate temporary dummy metadata object to > get vtable initialized > + CppVtabInfo* info = (CppVtabInfo*)p; > + int n = info->vtab_size(); > + intptr_t* srcvtab = vtab_of(tmp); > + intptr_t* dstvtab = info->vtab(); > + > + // We already checked (and, if necessary, adjusted n) > when the vtables were allocated, so we are > + // safe to do memcpy. > + if (PrintSharedSpaces) { > + tty->print_cr("%s copying %d vtable entries", name, n); > + } > + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); > + return dstvtab + n; > + } > > Done. I changed the wording > > T tmp; // Allocate temporary dummy metadata object to get > to the original vtable. > > As we are not really "initializing a vtable" here. > > Same with 'patch'. It'd be so much faster and easier to > read this code with more comments please. > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html > > > Why are these testers here? > > > I updated the comment: > > // Used by CDS. These classes need to access the private > ConstantPool() constructor. > template friend class CppVtableTesterA; > template friend class CppVtableTesterB; > template friend class CppVtableCloner; > > > Thanks > - Ioi > > > On 3/1/17 3:25 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8005165 > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ > > > Hi, > > This is the official review (follow up of the > "Determining the size of C++ vtables" thread > onhotspot-dev at openjdk.java.net > ). > > The new code has the same assumption as the > existing code in JDK 10: for a C++ object that > contains virtual methods (e.g., ConstantPool), > we assume the first intptr_t slot of the > object is a _vptr, which points to a vtable, > which consists of no more than 150 intptr_t's. > > ConstantPool*p -->[ _vptr ] -------> [ > vtable slot 0 ] > [ field #0 ] [ > vtable slot 1 ] > [ field #1 ] [ > vtable slot 2 ] > [ field #2 ] [ > vtable slot 3 ] > [ .... ] [ > vtable slot 4] > [ > vtable slot 5 ] > [ > ... ] > > + In the existing code, we were pointing the > vtable slots to > code that's generated by HotSpot. > > + In the new code, we copy the vtable slots > from an existing > vtable (generated by the C++ linker). > > Per Thomas St?fe's advice, I don't try to > determine the size of the vtable (as that > would add one more compiler requirement where > new virtual methods added by a subclass must > be placed at a higher offset in the vtable). > > Instead, I have added code in non-product > builds to ensure that the vtables are no > longer than 150 entries. You can run with > "-XX:+PrintSharedSpaces -Xshare:dump" to print > out the actual size of the vtables for your > particular platform: > > ConstantPool has 12 virtual methods > InstanceKlass has 113 virtual methods > InstanceClassLoaderKlass has 113 virtual methods > InstanceMirrorKlass has 113 virtual methods > InstanceRefKlass has 113 virtual methods > Method has 12 virtual methods > ObjArrayKlass has 114 virtual methods > TypeArrayKlass has 114 virtual methods > > As mentioned in the code comments, if you have > an esoteric C++ compiler, the > verify_sufficient_size() function will > probably fail, but hopefully that would give > you some hints for porting this code. > > To avoid accidentally touching an unmapped > page, the code uses SafeFetchN for copying the > vtable contents, and would shrink the vtable > to less than 150 entries if necessary. I can't > test this for real, but I've added some code > to simulate an error: > > for (int i=0; i const intptr_t bad = intptr_t(0xdeadbeef); > intptr_t num = SafeFetchN(&srcvtab[i], bad); > if (num == bad > // || i > 120 /* uncomment this line > to test */ > ) { > _info->set_vtab_size(i-1); > break; > } > dstvtab[i] = num; > } > > Results: > > + Removed 850 lines of CPU-dependent code > > + CDS image is about 50K smaller > > + Previously Metadata objects must live in the > read-write section in the CDS > archive, because their _vptr was updated at > run time. Now _vptr is no longer > updated, so ConstantPool can be moved to the > read-only section (see JDK-8171392). > > Thanks > - Ioi > > > > > > > From thomas.stuefe at gmail.com Tue Mar 7 12:22:36 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 7 Mar 2017 13:22:36 +0100 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BDB599.7050505@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BDB599.7050505@oracle.com> Message-ID: Hi Ioi, On Mon, Mar 6, 2017 at 8:16 PM, Ioi Lam wrote: > Hi Thomas, > > On 3/6/17 8:51 AM, Thomas St?fe wrote: > > Hi Coleen and Ioi, > > I had to port C++ code to platforms with terrible compilers for a time in > my life, that is why I like code to be as portable as possible. That said, > you are right in your argumentation, the SafeFetch solution is not terribly > elegant and Ioi's original way of determining the vtable size is cleaner. > > I did some checks on some of our architectures with a test similar to > Ioi's and on a first glance it seems to work for simple cases (single and > public inheritance) on ppc (AIX) and Linux ia64. Although the vtables > seemed to me to contain function descriptors, not real pointers to code, so > this is something to keep in mind. But if the live vtable are copied, the > function descriptors they contain should point to valid code too, so it > should not matter. Just to remember to not expect every slot in the array > to be a valid code pointer. > > I also notice on our current Solaris build the first slot in the vtable is > the RTTI pointer, not a function address. This slot is different in the > TesterA and TesterB classes, so I had to skip over it. I think this is safe > because Metadata has about 10 virtual methods, so skipping over the first > one should not cause any harm. > > So, in short, I remove my objection to Ioi's original solution, as far as > that matters. > > I still think we rely on a lot here: Contiguous vtable containing absolute > memory addresses, vtable pointer at start of object and vtable entries to > be ordered from base->derived class. So I wonder how much effort it would > be (now or in the future as a separate change) to have a fallback where - > at loading time - instead of copying vtables the vtable pointers in the > objects were fixed up to point to the new live vtables? I know this would > be more expensive and potentially defeat the point of shared classes. But > maybe not, it depends on how many objects are there, no? > > > I think for portability, perhaps we can add a porting layer so the vtables > can be copied in a different way? I think this is better than patching each > object at run time. > > That would be a sensible approach. > Also, I would suggest not adding such a layer now, but do it when such a > need actually arises. > > Sure. Kind Regards, Thomas > Thanks > - Ioi > > > Kind Regards, Thomas > > > > On Sun, Mar 5, 2017 at 4:17 PM, wrote: > >> >> Ioi, Some comments inline (where no comments, insert "ok") :) >> >> >> On 3/2/17 10:37 PM, Ioi Lam wrote: >> >>> Hi Coleen, >>> >>> Thanks for the comments. I have updated the webrev. See in-line for >>> responses. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind >>> ependent-cds-vtable.v03/ >>> >>> >>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com wrote: >>> >>>> >>>> Ioi >>>> I like the concept of this a lot but have some stylistic comments to >>>> help people reading this code later. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind >>>> ependent-cds-vtable.v02/src/share/vm/memory/metaspaceShared. >>>> cpp.udiff.html >>>> >>>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't save many >>>> characters, especially in CppVtableInfo/Testers >>>> >>>> Done. >>> >>>> + // Start at slot 1, because slot 0 may be RTTI (on Solaris/Sparc) >>>> + int i; >>>> + for (i=1; ; i++) { >>>> Since you're using 'i' later, can you rename it to something >>>> descriptive. Or have another variable "vtable_length" to use later. This >>>> looks like an old style for loop. >>>> >>>> Done >>> >>>> Can the functions for CppVtableInfo be declared outside of the class >>>> declaration? They don't need to be inline and then the debug code for >>>> testing the vtable size can be not in the middle of the class declaration. >>>> Then you can move the Tester classes to inside the same #ifndef PRODUCT >>>> block. >>>> >>>> Can you put #endif // PRODUCT when the ifdef covers several lines of >>>> code? >>>> >>>> Done >>> >>>> vtab_of could be more descriptive, like cpp_vtable_for(). >>>> >>>> I changed to vtable_of(). Because the class name is already >>> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >>> >>> Was PrintSharedSpaces was never converted to UL? >>>> >>>> Right. I've filed https://bugs.openjdk.java.net/browse/JDK-8176132 >>> (-XX:+PrintSharedSpaces should be converted to use Unified Logging.) >>> >>>> + int n = MAX_VTABLE_SIZE; >>>> >>>> Can you propagate MAX_VTABLE_SIZE to the places where it's used. n >>>> isn't descriptive. This starts out with max_vtable_size and then changes >>>> the size. Reusing 'n' makes this really hard to follow. Not having a >>>> comment that we only allocate enough slots for the vtable makes it hard too. >>>> >>>> + // allocate CppVtableInfo in the MD section + _info = >>>> (CppVtabInfo*)md_top; >>>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>>> + + // allocate temporary local instance of the metadata type T + T tmp; >>>> + intptr_t* srcvtab = vtab_of(tmp); >>>> + intptr_t* dstvtab = _info->vtab(); >>>> + >>>> >>> Fixed. >>> >>>> Something like that for comments. dstvtab is the destination_vtable in >>>> the MD section. >>>> >>> >>> I've dropped the md_ prefix from the functions that deal with the >>> vtables, since they shouldn't care whether it's the "MD" section or not. >>> Now it looks like this: >>> >>> // Allocate and initialize the C++ vtables, starting from top, but do >>> not go past end. >>> intptr_t* MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>> intptr_t* end) { >>> assert(DumpSharedSpaces, "dump-time only"); >>> // Layout (each slot is a intptr_t): >>> // [number of slots in the first vtable = n1] >>> // [ slots for the first vtable] >>> // [number of slots in the first second = n2] >>> // [ slots for the second vtable] >>> // ... >>> // The order of the vtables is the same as the CPP_VTAB_PATCH_TYPES_DO >>> macro. >>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>> return top; >>> } >>> >>> + for (int i=0; i>>> + const intptr_t bad = intptr_t(0xdeadbeef); >>>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>> + if (num == bad >>>> + // || i > 120 /* uncomment this line to test */ >>>> + ) { >>>> + _info->set_vtab_size(i-1); >>>> + break; >>>> + } >>>> + dstvtab[i] = num; >>>> + } >>>> I dont understand this code. You get deadbeef for a bad value if >>>> SafeFetchN gets a fault but why would it get a fault at the end of the >>>> metadata's vtable? Couldn't it just run onto the next vtable? I think >>>> your original way of counting vtable entries might be better (sorry I >>>> didn't have time to study that thread). >>>> >>>> I've modified the comments to this. Does it make sense to you? >>> >>> // It is not always safe to call memcpy(), because srcvtable might >>> be shorter than >>> // MAX_VTABLE_SIZE, and the C++ linker might have placed the vtable >>> at the very >>> // end of the last page of libjvm.so. Crossing over to the next page >>> might >>> // cause a page fault. >>> >>> My fear is the JVM would suddenly start crashing because the order of .o >>> files have changed on the linker's command line, or if you enable some >>> special linker optimization flags. It's better safe than sorry. >>> >> >> This wasn't exactly what I was not understanding. I didn't see that you >> are copying 120 entries from the old vtable and junk memory beyond the old >> vtable, unless you get a segv, in which case you copy less. I don't think >> you should copy random memory into the vtable in the archive. This doesn't >> seem secure, even with the segv protection. >> >> Since we already have assumptions about C++ vtable layout in the code and >> it's mostly specified by various ABIs, and you have the assert code, I >> think I would prefer that you copy only the vtable entries into the >> archive. I guess Thomas Stuefe had a different opinion. I've read the >> original thread. Two points: >> >> If new C++ compiler implementations add a discontigous vtable, both the >> SafeFetchN and subclass additional virtual function at end implementation >> will fail. I don't think C++ implementations would do this and a >> contiguous vtable as first in the instance has been standard for years. >> If our metadata adds multiple inheritance, the same issue would be a >> problem for both implementations, as well as for the implementation we have >> before Ioi's fix. >> >> Ioi's subclass adding virtual function method would work for any esoteric >> C++ implementations in my memory, except the vptr for the old DECC++ >> compiler was after the nonstatic data members (which would fail with all of >> our implementations). >> >> Since the code is there anyway for debug purposes, we're not saving code >> by implementing SafeFetchN. The SafeFetchN implementation isn't obvious at >> all what it's doing, and requires better comments, especially if you don't >> know already what SafeFetchN does. It looks really cryptic. The poisoned >> values also bothered me in that they overload other poisoned values in >> other parts of the jvm. >> >> Ioi, could you make all methods of CppVtableCloner out of line? >> >> The other changes look good, although I might have more requests for >> comments. >> >> Thanks, >> Coleen >> >> >> Would be nice to have comments here too!! >>>> >>>> + intptr_t* start = md_top; >>>> >>>> This doesn't do anything (?) >>>> >>> >>> Fixed. This was left over code. >>> >>>> >>>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>> >>>> Why not zero the destination vtable in allocate? Or does patching the >>>> vtable pointers call virtual functions? You could prevent that so you >>>> don't need this code. >>>> >>>> I added this comment: >>> >>> // During patching, some virtual methods may be called, so at this >>> point >>> // the vtables must contain valid methods (as filled in by >>> CppVtableCloner::allocate). >>> MetaspaceShared::patch_cpp_vtable_pointers(); >>> >>> // The vtable clones contain addresses of the current process. >>> // We don't want to write these addresses into the archive. >>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>> >>> + // Restore the vtable in case we invoke any virtual methods. >>>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>>> Can this be restore_cpp_vtables since that's what it's doing. The first >>>> is after the dump and the second call is at UseSharedSpaces. A couple of >>>> comments in this clone_cpp_vtables --> restore_cpp_vtables would be nice. >>>> eg: >>>> >>>> I prefer to use the word clone. Otherwise when you just say "vtable" >>> it's not clear whether you're talking about the original one (made by the >>> c++ linker), or the cloned one in the CDS archive. >>> >>>> + static intptr_t* clone_vtable(const char* name, intptr_t* p) { >>>> + T tmp; // Allocate temporary dummy metadata object to get vtable >>>> initialized >>>> + CppVtabInfo* info = (CppVtabInfo*)p; >>>> + int n = info->vtab_size(); >>>> + intptr_t* srcvtab = vtab_of(tmp); >>>> + intptr_t* dstvtab = info->vtab(); >>>> + >>>> + // We already checked (and, if necessary, adjusted n) when the >>>> vtables were allocated, so we are >>>> + // safe to do memcpy. >>>> + if (PrintSharedSpaces) { >>>> + tty->print_cr("%s copying %d vtable entries", name, n); >>>> + } >>>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>>> + return dstvtab + n; >>>> + } >>>> >>>> Done. I changed the wording >>> >>> T tmp; // Allocate temporary dummy metadata object to get to the >>> original vtable. >>> >>> As we are not really "initializing a vtable" here. >>> >>> Same with 'patch'. It'd be so much faster and easier to read this code >>>> with more comments please. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind >>>> ependent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>>> >>>> Why are these testers here? >>>> >>>> >>> I updated the comment: >>> >>> // Used by CDS. These classes need to access the private >>> ConstantPool() constructor. >>> template friend class CppVtableTesterA; >>> template friend class CppVtableTesterB; >>> template friend class CppVtableCloner; >>> >>> >>> Thanks >>> - Ioi >>> >>> >>>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-ind >>>>>>> ependent-cds-vtable.v02/ >>>>>>> Hi, >>>>>>> >>>>>>> This is the official review (follow up of the "Determining the size >>>>>>> of C++ vtables" thread onhotspot-dev at openjdk.java.net). >>>>>>> >>>>>>> The new code has the same assumption as the existing code in JDK 10: >>>>>>> for a C++ object that contains virtual methods (e.g., ConstantPool), we >>>>>>> assume the first intptr_t slot of the object is a _vptr, which points to a >>>>>>> vtable, which consists of no more than 150 intptr_t's. >>>>>>> >>>>>>> ConstantPool*p -->[ _vptr ] -------> [ vtable slot 0 ] >>>>>>> [ field #0 ] [ vtable slot 1 ] >>>>>>> [ field #1 ] [ vtable slot 2 ] >>>>>>> [ field #2 ] [ vtable slot 3 ] >>>>>>> [ .... ] [ vtable slot 4] >>>>>>> [ vtable slot 5 ] >>>>>>> [ ... ] >>>>>>> >>>>>>> + In the existing code, we were pointing the vtable slots to >>>>>>> code that's generated by HotSpot. >>>>>>> >>>>>>> + In the new code, we copy the vtable slots from an existing >>>>>>> vtable (generated by the C++ linker). >>>>>>> >>>>>>> Per Thomas St?fe's advice, I don't try to determine the size of the >>>>>>> vtable (as that would add one more compiler requirement where new virtual >>>>>>> methods added by a subclass must be placed at a higher offset in the >>>>>>> vtable). >>>>>>> >>>>>>> Instead, I have added code in non-product builds to ensure that the >>>>>>> vtables are no longer than 150 entries. You can run with >>>>>>> "-XX:+PrintSharedSpaces -Xshare:dump" to print out the actual size of the >>>>>>> vtables for your particular platform: >>>>>>> >>>>>>> ConstantPool has 12 virtual methods >>>>>>> InstanceKlass has 113 virtual methods >>>>>>> InstanceClassLoaderKlass has 113 virtual methods >>>>>>> InstanceMirrorKlass has 113 virtual methods >>>>>>> InstanceRefKlass has 113 virtual methods >>>>>>> Method has 12 virtual methods >>>>>>> ObjArrayKlass has 114 virtual methods >>>>>>> TypeArrayKlass has 114 virtual methods >>>>>>> >>>>>>> As mentioned in the code comments, if you have an esoteric C++ >>>>>>> compiler, the verify_sufficient_size() function will probably fail, but >>>>>>> hopefully that would give you some hints for porting this code. >>>>>>> >>>>>>> To avoid accidentally touching an unmapped page, the code uses >>>>>>> SafeFetchN for copying the vtable contents, and would shrink the vtable to >>>>>>> less than 150 entries if necessary. I can't test this for real, but I've >>>>>>> added some code to simulate an error: >>>>>>> >>>>>>> for (int i=0; i>>>>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>>>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>>> if (num == bad >>>>>>> // || i > 120 /* uncomment this line to test */ >>>>>>> ) { >>>>>>> _info->set_vtab_size(i-1); >>>>>>> break; >>>>>>> } >>>>>>> dstvtab[i] = num; >>>>>>> } >>>>>>> >>>>>>> Results: >>>>>>> >>>>>>> + Removed 850 lines of CPU-dependent code >>>>>>> >>>>>>> + CDS image is about 50K smaller >>>>>>> >>>>>>> + Previously Metadata objects must live in the read-write section in >>>>>>> the CDS >>>>>>> archive, because their _vptr was updated at run time. Now _vptr is >>>>>>> no longer >>>>>>> updated, so ConstantPool can be moved to the read-only section >>>>>>> (see JDK-8171392). >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>> >>> >> > > From coleen.phillimore at oracle.com Tue Mar 7 13:48:35 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 7 Mar 2017 08:48:35 -0500 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BE0C03.1040105@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> Message-ID: <1de9cffc-535d-d687-f01b-0b04c01044cb@oracle.com> This looks really good!! Are you going to do a follow up to remove the MC section? thanks! Coleen On 3/6/17 8:25 PM, Ioi Lam wrote: > Hi Thomas & Coleen, > > Thanks for your comments. I have updated the rev: > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ > > [1] Switched back to computing the exact vtable size > [2] Move non-trivial functions outside of their class declaration > > Thanks > - Ioi > > On 3/6/17 8:51 AM, Thomas St?fe wrote: >> Hi Coleen and Ioi, >> >> I had to port C++ code to platforms with terrible compilers for a >> time in my life, that is why I like code to be as portable as >> possible. That said, you are right in your argumentation, the >> SafeFetch solution is not terribly elegant and Ioi's original way of >> determining the vtable size is cleaner. >> >> I did some checks on some of our architectures with a test similar to >> Ioi's and on a first glance it seems to work for simple cases (single >> and public inheritance) on ppc (AIX) and Linux ia64. Although the >> vtables seemed to me to contain function descriptors, not real >> pointers to code, so this is something to keep in mind. But if the >> live vtable are copied, the function descriptors they contain should >> point to valid code too, so it should not matter. Just to remember to >> not expect every slot in the array to be a valid code pointer. >> >> So, in short, I remove my objection to Ioi's original solution, as >> far as that matters. >> >> I still think we rely on a lot here: Contiguous vtable containing >> absolute memory addresses, vtable pointer at start of object and >> vtable entries to be ordered from base->derived class. So I wonder >> how much effort it would be (now or in the future as a separate >> change) to have a fallback where - at loading time - instead of >> copying vtables the vtable pointers in the objects were fixed up to >> point to the new live vtables? I know this would be more expensive >> and potentially defeat the point of shared classes. But maybe not, it >> depends on how many objects are there, no? >> >> Kind Regards, Thomas >> >> On Sun, Mar 5, 2017 at 4:17 PM, > > wrote: >> >> >> Ioi, Some comments inline (where no comments, insert "ok") :) >> >> >> On 3/2/17 10:37 PM, Ioi Lam wrote: >> >> Hi Coleen, >> >> Thanks for the comments. I have updated the webrev. See >> in-line for responses. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >> >> >> >> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >> wrote: >> >> >> Ioi >> I like the concept of this a lot but have some stylistic >> comments to help people reading this code later. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >> >> >> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >> save many characters, especially in CppVtableInfo/Testers >> >> Done. >> >> + // Start at slot 1, because slot 0 may be RTTI (on >> Solaris/Sparc) >> + int i; >> + for (i=1; ; i++) { >> Since you're using 'i' later, can you rename it to >> something descriptive. Or have another variable >> "vtable_length" to use later. This looks like an old >> style for loop. >> >> Done >> >> Can the functions for CppVtableInfo be declared outside >> of the class declaration? They don't need to be inline >> and then the debug code for testing the vtable size can >> be not in the middle of the class declaration. Then you >> can move the Tester classes to inside the same #ifndef >> PRODUCT block. >> >> Can you put #endif // PRODUCT when the ifdef covers >> several lines of code? >> >> Done >> >> vtab_of could be more descriptive, like cpp_vtable_for(). >> >> I changed to vtable_of(). Because the class name is already >> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >> >> Was PrintSharedSpaces was never converted to UL? >> >> Right. I've filed >> https://bugs.openjdk.java.net/browse/JDK-8176132 >> >> (-XX:+PrintSharedSpaces should be converted to use Unified >> Logging.) >> >> + int n = MAX_VTABLE_SIZE; >> >> Can you propagate MAX_VTABLE_SIZE to the places where >> it's used. n isn't descriptive. This starts out with >> max_vtable_size and then changes the size. Reusing 'n' >> makes this really hard to follow. Not having a comment >> that we only allocate enough slots for the vtable makes >> it hard too. >> >> + // allocate CppVtableInfo in the MD section + _info = >> (CppVtabInfo*)md_top; >> + _info->set_vtab_size(n); // initially set to >> max_vtable_size >> + + // allocate temporary local instance of the metadata >> type T + T tmp; >> + intptr_t* srcvtab = vtab_of(tmp); >> + intptr_t* dstvtab = _info->vtab(); >> + >> >> Fixed. >> >> Something like that for comments. dstvtab is the >> destination_vtable in the MD section. >> >> >> I've dropped the md_ prefix from the functions that deal with >> the vtables, since they shouldn't care whether it's the "MD" >> section or not. Now it looks like this: >> >> // Allocate and initialize the C++ vtables, starting from >> top, but do not go past end. >> intptr_t* >> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >> intptr_t* end) { >> assert(DumpSharedSpaces, "dump-time only"); >> // Layout (each slot is a intptr_t): >> // [number of slots in the first vtable = n1] >> // [ slots for the first vtable] >> // [number of slots in the first second = n2] >> // [ slots for the second vtable] >> // ... >> // The order of the vtables is the same as the >> CPP_VTAB_PATCH_TYPES_DO macro. >> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >> return top; >> } >> >> + for (int i=0; i> + const intptr_t bad = intptr_t(0xdeadbeef); >> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >> + if (num == bad >> + // || i > 120 /* uncomment this line to test */ >> + ) { >> + _info->set_vtab_size(i-1); >> + break; >> + } >> + dstvtab[i] = num; >> + } >> I dont understand this code. You get deadbeef for a bad >> value if SafeFetchN gets a fault but why would it get a >> fault at the end of the metadata's vtable? Couldn't it >> just run onto the next vtable? I think your original way >> of counting vtable entries might be better (sorry I >> didn't have time to study that thread). >> >> I've modified the comments to this. Does it make sense to you? >> >> // It is not always safe to call memcpy(), because >> srcvtable might be shorter than >> // MAX_VTABLE_SIZE, and the C++ linker might have placed >> the vtable at the very >> // end of the last page of libjvm.so. Crossing over to >> the next page might >> // cause a page fault. >> >> My fear is the JVM would suddenly start crashing because the >> order of .o files have changed on the linker's command line, >> or if you enable some special linker optimization flags. It's >> better safe than sorry. >> >> >> This wasn't exactly what I was not understanding. I didn't see >> that you are copying 120 entries from the old vtable and junk >> memory beyond the old vtable, unless you get a segv, in which >> case you copy less. I don't think you should copy random memory >> into the vtable in the archive. This doesn't seem secure, even >> with the segv protection. >> >> Since we already have assumptions about C++ vtable layout in the >> code and it's mostly specified by various ABIs, and you have the >> assert code, I think I would prefer that you copy only the vtable >> entries into the archive. I guess Thomas Stuefe had a different >> opinion. I've read the original thread. Two points: >> >> If new C++ compiler implementations add a discontigous vtable, >> both the SafeFetchN and subclass additional virtual function at >> end implementation will fail. I don't think C++ implementations >> would do this and a contiguous vtable as first in the instance >> has been standard for years. If our metadata adds multiple >> inheritance, the same issue would be a problem for both >> implementations, as well as for the implementation we have before >> Ioi's fix. >> >> Ioi's subclass adding virtual function method would work for any >> esoteric C++ implementations in my memory, except the vptr for >> the old DECC++ compiler was after the nonstatic data members >> (which would fail with all of our implementations). >> >> Since the code is there anyway for debug purposes, we're not >> saving code by implementing SafeFetchN. The SafeFetchN >> implementation isn't obvious at all what it's doing, and requires >> better comments, especially if you don't know already what >> SafeFetchN does. It looks really cryptic. The poisoned values >> also bothered me in that they overload other poisoned values in >> other parts of the jvm. >> >> Ioi, could you make all methods of CppVtableCloner out of line? >> >> The other changes look good, although I might have more requests >> for comments. >> >> Thanks, >> Coleen >> >> >> Would be nice to have comments here too!! >> >> + intptr_t* start = md_top; >> >> This doesn't do anything (?) >> >> >> Fixed. This was left over code. >> >> >> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >> >> Why not zero the destination vtable in allocate? Or does >> patching the vtable pointers call virtual functions? You >> could prevent that so you don't need this code. >> >> I added this comment: >> >> // During patching, some virtual methods may be called, so >> at this point >> // the vtables must contain valid methods (as filled in by >> CppVtableCloner::allocate). >> MetaspaceShared::patch_cpp_vtable_pointers(); >> >> // The vtable clones contain addresses of the current process. >> // We don't want to write these addresses into the archive. >> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >> >> + // Restore the vtable in case we invoke any virtual >> methods. >> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >> Can this be restore_cpp_vtables since that's what it's >> doing. The first is after the dump and the second call is >> at UseSharedSpaces. A couple of comments in this >> clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: >> >> I prefer to use the word clone. Otherwise when you just say >> "vtable" it's not clear whether you're talking about the >> original one (made by the c++ linker), or the cloned one in >> the CDS archive. >> >> + static intptr_t* clone_vtable(const char* name, >> intptr_t* p) { >> + T tmp; // Allocate temporary dummy metadata object to >> get vtable initialized >> + CppVtabInfo* info = (CppVtabInfo*)p; >> + int n = info->vtab_size(); >> + intptr_t* srcvtab = vtab_of(tmp); >> + intptr_t* dstvtab = info->vtab(); >> + >> + // We already checked (and, if necessary, adjusted n) >> when the vtables were allocated, so we are >> + // safe to do memcpy. >> + if (PrintSharedSpaces) { >> + tty->print_cr("%s copying %d vtable entries", name, n); >> + } >> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >> + return dstvtab + n; >> + } >> >> Done. I changed the wording >> >> T tmp; // Allocate temporary dummy metadata object to get >> to the original vtable. >> >> As we are not really "initializing a vtable" here. >> >> Same with 'patch'. It'd be so much faster and easier to >> read this code with more comments please. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >> >> >> Why are these testers here? >> >> >> I updated the comment: >> >> // Used by CDS. These classes need to access the private >> ConstantPool() constructor. >> template friend class CppVtableTesterA; >> template friend class CppVtableTesterB; >> template friend class CppVtableCloner; >> >> >> Thanks >> - Ioi >> >> >> On 3/1/17 3:25 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8005165 >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >> >> >> Hi, >> >> This is the official review (follow up of the >> "Determining the size of C++ vtables" thread >> onhotspot-dev at openjdk.java.net >> ). >> >> The new code has the same assumption as the >> existing code in JDK 10: for a C++ object >> that contains virtual methods (e.g., >> ConstantPool), we assume the first intptr_t >> slot of the object is a _vptr, which points >> to a vtable, which consists of no more than >> 150 intptr_t's. >> >> ConstantPool*p -->[ _vptr ] -------> [ >> vtable slot 0 ] >> [ field #0 ] [ >> vtable slot 1 ] >> [ field #1 ] [ >> vtable slot 2 ] >> [ field #2 ] [ >> vtable slot 3 ] >> [ .... ] [ >> vtable slot 4] >> [ >> vtable slot 5 ] >> [ >> ... ] >> >> + In the existing code, we were pointing the >> vtable slots to >> code that's generated by HotSpot. >> >> + In the new code, we copy the vtable slots >> from an existing >> vtable (generated by the C++ linker). >> >> Per Thomas St?fe's advice, I don't try to >> determine the size of the vtable (as that >> would add one more compiler requirement where >> new virtual methods added by a subclass must >> be placed at a higher offset in the vtable). >> >> Instead, I have added code in non-product >> builds to ensure that the vtables are no >> longer than 150 entries. You can run with >> "-XX:+PrintSharedSpaces -Xshare:dump" to >> print out the actual size of the vtables for >> your particular platform: >> >> ConstantPool has 12 virtual methods >> InstanceKlass has 113 virtual methods >> InstanceClassLoaderKlass has 113 virtual >> methods >> InstanceMirrorKlass has 113 virtual methods >> InstanceRefKlass has 113 virtual methods >> Method has 12 virtual methods >> ObjArrayKlass has 114 virtual methods >> TypeArrayKlass has 114 virtual methods >> >> As mentioned in the code comments, if you >> have an esoteric C++ compiler, the >> verify_sufficient_size() function will >> probably fail, but hopefully that would give >> you some hints for porting this code. >> >> To avoid accidentally touching an unmapped >> page, the code uses SafeFetchN for copying >> the vtable contents, and would shrink the >> vtable to less than 150 entries if necessary. >> I can't test this for real, but I've added >> some code to simulate an error: >> >> for (int i=0; i> const intptr_t bad = intptr_t(0xdeadbeef); >> intptr_t num = SafeFetchN(&srcvtab[i], >> bad); >> if (num == bad >> // || i > 120 /* uncomment this >> line to test */ >> ) { >> _info->set_vtab_size(i-1); >> break; >> } >> dstvtab[i] = num; >> } >> >> Results: >> >> + Removed 850 lines of CPU-dependent code >> >> + CDS image is about 50K smaller >> >> + Previously Metadata objects must live in >> the read-write section in the CDS >> archive, because their _vptr was updated at >> run time. Now _vptr is no longer >> updated, so ConstantPool can be moved to >> the read-only section (see JDK-8171392). >> >> Thanks >> - Ioi >> >> >> >> >> >> >> > From thomas.stuefe at gmail.com Tue Mar 7 14:01:39 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 7 Mar 2017 15:01:39 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: References: Message-ID: (Resent to hotspot-dev) On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe wrote: > Hi all, > > I would like to get a review for this cleanup change for the > metaspace.cpp. This has been a long time brewing in my backlog, but I had > to wait until jdk10 is open. The cleanup is actually quite small, the > largest part of the change is the added test coding. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE > -1-8170933-unify-treatment-of-humongous-and-non-humongous- > chunks-on-chunkmanager-return/jdk10-webrev.00/webrev/ > > The change implements a suggestion made by Mikael Gerdin when > reviewing JDK-8170520 last year, who suggested that the ChunkManager class > should handle returns of both non-humongous and humongous chunks, see > original discussion here: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 > 016-November/021949.html > > "It would be great if you could have a look at unifying the > ChunkManager::return_chunks API for standard and humongous chunks so > that we wouldn't need the special casing for this in ~SpaceManager > That way we could also make humongous_dictionary() private to > ChunkManager which would improve encapsulation." > > The cleanup does this and a bit more, all changes having to do with the > fact that the ChunkManager class unnecessarily exposes internals to the > other classes in metaspace.cpp and that with a little bit of reorganization > this can be avoided. The benefit is simpler coding and way better > encapsulation, which will help a lot with future changes (in preparation > for https://bugs.openjdk.java.net/browse/JDK-8166690). > > -------- > > The changes in detail: > > The main issue was that ChunkManager::return_chunks() did not handle > humongous chunks. To return humongous chunks, one had to access the > humongous chunk dictionary inside the ChunkManager and add the chunk > yourself, including maintaining all counters. This happened in > ~SpaceManager and made this destructor very complicated. > > This is solved by moving the handling of humongous chunk returns to the > ChunkManager class itself. For convenience, I split the old > "ChunkManager::return_chunks" method into two new ones, > "ChunkManager::return_single_chunk()" which returns a single chunk to the > ChunkManager without walking the chunk chain, and > "ChunkManger::return_chunk_list()" which returns a chain of chunks to the > ChunkManager. Both functions are now able to handle both non-humongous and > humongous chunks. ~SpaceManager() was reimplemented (actually, quite > simplified) and just calls "ChunkManager::return_chunk_list()" for all > its chunk lists. > > So now we can remove the public access to the humongous chunk dictionary > in ChunkManger (ChunkManager::humongous_dictionary()) and make this > function private. > > ---- > > Then there was the fact that the non-humongous chunk lists were also > exposed to public via ChunkManager::free_chunks(). The only reason was that > when a VirtualSpaceNode is retired, it accessed the ChunkManager free lists > to find out the size of the items of the free lists. > > This was solved by adding a new function, ChunkManager::size_by_index(), > which returns the size of the items of a chunk list given by its chunk > index. > > So ChunkManager::free_chunks() could be made private too. > > --- > > The rest are minor changes: > > - made ChunkManager::find_free_chunks_list() private because it was not > used from outside the class > - Fixed an assert in dec_free_chunks_total() > - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried > to keep the existing logging intact or add useful logging where possible. > > ---- > > A large chunk of the changes is the added gtests. There is a new class > now, ChunkManagerReturnTest, which stresses the ChunkManager take/return > code. > > Note that I dislike the fact that this test is implemented inside > metaspace.cpp itself. For now I kept my new tests like the existing tests > inside this file, but I think these tests should be moved to > test/native/memory/test_chunkManager.cpp. I suggest doing this in a > separate fix though, because it needs a bit of remodeling (the tests need > to access internals in metaspace.cpp). > > ---- > > I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and > AIX. No issues popped up which were associated with my change. > > Thanks for reviewing and Kind Regards, Thomas > > > > > > > > > > > > From christian.tornqvist at oracle.com Tue Mar 7 16:24:05 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Tue, 7 Mar 2017 11:24:05 -0500 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* In-Reply-To: <336c01d296c0$004b5570$00e20050$@oracle.com> References: <169f01d29434$a84a7d70$f8df7850$@oracle.com> <86ccedc2-37e1-9360-f0d5-5ed2338b0c8d@oracle.com> <310c01d2968f$61e0d520$25a27f60$@oracle.com> <336c01d296c0$004b5570$00e20050$@oracle.com> Message-ID: <36b301d2975f$3dcb59f0$b9620dd0$@oracle.com> I'm going to go ahead and push this change, the compiler team can follow up on the renaming of hotspot_not_fast_compiler at a later time. Thanks, Christian -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Christian Tornqvist Sent: Monday, March 6, 2017 4:24 PM To: 'Vladimir Kozlov' ; 'David Holmes' ; hotspot-dev at openjdk.java.net Subject: RE: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* Hi Vladimir, That would be misleading though, it's used a lot more than just in tier2. Thanks, Christian -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Monday, March 6, 2017 11:59 AM To: Christian Tornqvist ; 'David Holmes' ; hotspot-dev at openjdk.java.net Subject: Re: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* Hi Christian, As I remember hotspot_not_fast_compiler is used in later tiers. Can we just name it accordingly, for example: hotspot_tier2_compiler? Thanks, Vladimir On 3/6/17 7:36 AM, Christian Tornqvist wrote: > Hi David, > > Not sure what that name would be though, the compiler is using this > group with flag rotation in multiple tiers. I'll leave the renaming of > this group to the compiler team to sort out. > > Thanks, > Christioan > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Friday, March 3, 2017 9:21 PM > To: Christian Tornqvist ; > hotspot-dev at openjdk.java.net > Subject: Re: RFR(S): 8176102 - Rename hotspot_fast* test groups to > hotspot_tier1* > > Hi Christian, > > Rename looks good. > > Should we also rename the now oddly named hotspot_not_fast_compiler? > > Thanks, > David > > On 4/03/2017 1:41 AM, Christian Tornqvist wrote: >> Hi everyone, >> >> >> >> Please review this small change that renames the hotspot_fast* test >> groups to hotspot_tier1, also renamed the hotspot_runtime_tier* to >> follow the same naming standard. >> >> Tested by running the change through JPRT. >> >> >> >> Webrevs: >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00 >> / >> >> >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8176102 >> >> >> >> Thanks, >> >> Christian >> > From vladimir.kozlov at oracle.com Tue Mar 7 16:31:49 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 7 Mar 2017 08:31:49 -0800 Subject: RFR(S): 8176102 - Rename hotspot_fast* test groups to hotspot_tier1* In-Reply-To: <36b301d2975f$3dcb59f0$b9620dd0$@oracle.com> References: <169f01d29434$a84a7d70$f8df7850$@oracle.com> <86ccedc2-37e1-9360-f0d5-5ed2338b0c8d@oracle.com> <310c01d2968f$61e0d520$25a27f60$@oracle.com> <336c01d296c0$004b5570$00e20050$@oracle.com> <36b301d2975f$3dcb59f0$b9620dd0$@oracle.com> Message-ID: <4dbedb33-765a-1f94-54d6-61e94d9ec439@oracle.com> Okay Thanks, Vladimir On 3/7/17 8:24 AM, Christian Tornqvist wrote: > I'm going to go ahead and push this change, the compiler team can follow up > on the renaming of hotspot_not_fast_compiler at a later time. > > Thanks, > Christian > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of > Christian Tornqvist > Sent: Monday, March 6, 2017 4:24 PM > To: 'Vladimir Kozlov' ; 'David Holmes' > ; hotspot-dev at openjdk.java.net > Subject: RE: RFR(S): 8176102 - Rename hotspot_fast* test groups to > hotspot_tier1* > > Hi Vladimir, > > That would be misleading though, it's used a lot more than just in tier2. > > Thanks, > Christian > > -----Original Message----- > From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] > Sent: Monday, March 6, 2017 11:59 AM > To: Christian Tornqvist ; 'David Holmes' > ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(S): 8176102 - Rename hotspot_fast* test groups to > hotspot_tier1* > > Hi Christian, > > As I remember hotspot_not_fast_compiler is used in later tiers. Can we just > name it accordingly, for example: hotspot_tier2_compiler? > > Thanks, > Vladimir > > On 3/6/17 7:36 AM, Christian Tornqvist wrote: >> Hi David, >> >> Not sure what that name would be though, the compiler is using this >> group with flag rotation in multiple tiers. I'll leave the renaming of >> this group to the compiler team to sort out. >> >> Thanks, >> Christioan >> >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Friday, March 3, 2017 9:21 PM >> To: Christian Tornqvist ; >> hotspot-dev at openjdk.java.net >> Subject: Re: RFR(S): 8176102 - Rename hotspot_fast* test groups to >> hotspot_tier1* >> >> Hi Christian, >> >> Rename looks good. >> >> Should we also rename the now oddly named hotspot_not_fast_compiler? >> >> Thanks, >> David >> >> On 4/03/2017 1:41 AM, Christian Tornqvist wrote: >>> Hi everyone, >>> >>> >>> >>> Please review this small change that renames the hotspot_fast* test >>> groups to hotspot_tier1, also renamed the hotspot_runtime_tier* to >>> follow the same naming standard. >>> >>> Tested by running the change through JPRT. >>> >>> >>> >>> Webrevs: >>> >>> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/root/webrev.00/ >>> >>> http://cr.openjdk.java.net/~ctornqvi/webrev/8176102/hotspot/webrev.00 >>> / >>> >>> >>> >>> Bug: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8176102 >>> >>> >>> >>> Thanks, >>> >>> Christian >>> >> > > From ioi.lam at oracle.com Tue Mar 7 17:05:00 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 07 Mar 2017 09:05:00 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <1de9cffc-535d-d687-f01b-0b04c01044cb@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> <1de9cffc-535d-d687-f01b-0b04c01044cb@oracle.com> Message-ID: <58BEE83C.9090309@oracle.com> Hi Coleen & Thomas, Thanks again for the review. On 3/7/17 5:48 AM, coleen.phillimore at oracle.com wrote: > This looks really good!! Are you going to do a follow up to remove > the MC section? Yes, MC will be removed in JDK-8072061 - Automatically determine optimal sizes for the CDS region sizes Thanks - Ioi > thanks! > Coleen > > On 3/6/17 8:25 PM, Ioi Lam wrote: >> Hi Thomas & Coleen, >> >> Thanks for your comments. I have updated the rev: >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ >> >> [1] Switched back to computing the exact vtable size >> [2] Move non-trivial functions outside of their class declaration >> >> Thanks >> - Ioi >> >> On 3/6/17 8:51 AM, Thomas St?fe wrote: >>> Hi Coleen and Ioi, >>> >>> I had to port C++ code to platforms with terrible compilers for a >>> time in my life, that is why I like code to be as portable as >>> possible. That said, you are right in your argumentation, the >>> SafeFetch solution is not terribly elegant and Ioi's original way of >>> determining the vtable size is cleaner. >>> >>> I did some checks on some of our architectures with a test similar >>> to Ioi's and on a first glance it seems to work for simple cases >>> (single and public inheritance) on ppc (AIX) and Linux ia64. >>> Although the vtables seemed to me to contain function descriptors, >>> not real pointers to code, so this is something to keep in mind. But >>> if the live vtable are copied, the function descriptors they contain >>> should point to valid code too, so it should not matter. Just to >>> remember to not expect every slot in the array to be a valid code >>> pointer. >>> >>> So, in short, I remove my objection to Ioi's original solution, as >>> far as that matters. >>> >>> I still think we rely on a lot here: Contiguous vtable containing >>> absolute memory addresses, vtable pointer at start of object and >>> vtable entries to be ordered from base->derived class. So I wonder >>> how much effort it would be (now or in the future as a separate >>> change) to have a fallback where - at loading time - instead of >>> copying vtables the vtable pointers in the objects were fixed up to >>> point to the new live vtables? I know this would be more expensive >>> and potentially defeat the point of shared classes. But maybe not, >>> it depends on how many objects are there, no? >>> >>> Kind Regards, Thomas >>> >>> On Sun, Mar 5, 2017 at 4:17 PM, >> > wrote: >>> >>> >>> Ioi, Some comments inline (where no comments, insert "ok") :) >>> >>> >>> On 3/2/17 10:37 PM, Ioi Lam wrote: >>> >>> Hi Coleen, >>> >>> Thanks for the comments. I have updated the webrev. See >>> in-line for responses. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>> >>> >>> >>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >>> wrote: >>> >>> >>> Ioi >>> I like the concept of this a lot but have some stylistic >>> comments to help people reading this code later. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>> >>> >>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >>> save many characters, especially in CppVtableInfo/Testers >>> >>> Done. >>> >>> + // Start at slot 1, because slot 0 may be RTTI (on >>> Solaris/Sparc) >>> + int i; >>> + for (i=1; ; i++) { >>> Since you're using 'i' later, can you rename it to >>> something descriptive. Or have another variable >>> "vtable_length" to use later. This looks like an old >>> style for loop. >>> >>> Done >>> >>> Can the functions for CppVtableInfo be declared outside >>> of the class declaration? They don't need to be inline >>> and then the debug code for testing the vtable size can >>> be not in the middle of the class declaration. Then >>> you can move the Tester classes to inside the same >>> #ifndef PRODUCT block. >>> >>> Can you put #endif // PRODUCT when the ifdef covers >>> several lines of code? >>> >>> Done >>> >>> vtab_of could be more descriptive, like cpp_vtable_for(). >>> >>> I changed to vtable_of(). Because the class name is already >>> CppVtableCloner, repeating the word "cpp" seems repetitive >>> to me. >>> >>> Was PrintSharedSpaces was never converted to UL? >>> >>> Right. I've filed >>> https://bugs.openjdk.java.net/browse/JDK-8176132 >>> >>> (-XX:+PrintSharedSpaces should be converted to use Unified >>> Logging.) >>> >>> + int n = MAX_VTABLE_SIZE; >>> >>> Can you propagate MAX_VTABLE_SIZE to the places where >>> it's used. n isn't descriptive. This starts out with >>> max_vtable_size and then changes the size. Reusing 'n' >>> makes this really hard to follow. Not having a comment >>> that we only allocate enough slots for the vtable makes >>> it hard too. >>> >>> + // allocate CppVtableInfo in the MD section + _info = >>> (CppVtabInfo*)md_top; >>> + _info->set_vtab_size(n); // initially set to >>> max_vtable_size >>> + + // allocate temporary local instance of the metadata >>> type T + T tmp; >>> + intptr_t* srcvtab = vtab_of(tmp); >>> + intptr_t* dstvtab = _info->vtab(); >>> + >>> >>> Fixed. >>> >>> Something like that for comments. dstvtab is the >>> destination_vtable in the MD section. >>> >>> >>> I've dropped the md_ prefix from the functions that deal >>> with the vtables, since they shouldn't care whether it's the >>> "MD" section or not. Now it looks like this: >>> >>> // Allocate and initialize the C++ vtables, starting from >>> top, but do not go past end. >>> intptr_t* >>> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>> intptr_t* end) { >>> assert(DumpSharedSpaces, "dump-time only"); >>> // Layout (each slot is a intptr_t): >>> // [number of slots in the first vtable = n1] >>> // [ slots for the first vtable] >>> // [number of slots in the first second = n2] >>> // [ slots for the second vtable] >>> // ... >>> // The order of the vtables is the same as the >>> CPP_VTAB_PATCH_TYPES_DO macro. >>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>> return top; >>> } >>> >>> + for (int i=0; i>> + const intptr_t bad = intptr_t(0xdeadbeef); >>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>> + if (num == bad >>> + // || i > 120 /* uncomment this line to test */ >>> + ) { >>> + _info->set_vtab_size(i-1); >>> + break; >>> + } >>> + dstvtab[i] = num; >>> + } >>> I dont understand this code. You get deadbeef for a >>> bad value if SafeFetchN gets a fault but why would it >>> get a fault at the end of the metadata's vtable? >>> Couldn't it just run onto the next vtable? I think >>> your original way of counting vtable entries might be >>> better (sorry I didn't have time to study that thread). >>> >>> I've modified the comments to this. Does it make sense to you? >>> >>> // It is not always safe to call memcpy(), because >>> srcvtable might be shorter than >>> // MAX_VTABLE_SIZE, and the C++ linker might have placed >>> the vtable at the very >>> // end of the last page of libjvm.so. Crossing over to >>> the next page might >>> // cause a page fault. >>> >>> My fear is the JVM would suddenly start crashing because the >>> order of .o files have changed on the linker's command line, >>> or if you enable some special linker optimization flags. >>> It's better safe than sorry. >>> >>> >>> This wasn't exactly what I was not understanding. I didn't see >>> that you are copying 120 entries from the old vtable and junk >>> memory beyond the old vtable, unless you get a segv, in which >>> case you copy less. I don't think you should copy random >>> memory into the vtable in the archive. This doesn't seem >>> secure, even with the segv protection. >>> >>> Since we already have assumptions about C++ vtable layout in the >>> code and it's mostly specified by various ABIs, and you have the >>> assert code, I think I would prefer that you copy only the >>> vtable entries into the archive. I guess Thomas Stuefe had a >>> different opinion. I've read the original thread. Two points: >>> >>> If new C++ compiler implementations add a discontigous vtable, >>> both the SafeFetchN and subclass additional virtual function at >>> end implementation will fail. I don't think C++ implementations >>> would do this and a contiguous vtable as first in the instance >>> has been standard for years. If our metadata adds multiple >>> inheritance, the same issue would be a problem for both >>> implementations, as well as for the implementation we have >>> before Ioi's fix. >>> >>> Ioi's subclass adding virtual function method would work for any >>> esoteric C++ implementations in my memory, except the vptr for >>> the old DECC++ compiler was after the nonstatic data members >>> (which would fail with all of our implementations). >>> >>> Since the code is there anyway for debug purposes, we're not >>> saving code by implementing SafeFetchN. The SafeFetchN >>> implementation isn't obvious at all what it's doing, and >>> requires better comments, especially if you don't know already >>> what SafeFetchN does. It looks really cryptic. The poisoned >>> values also bothered me in that they overload other poisoned >>> values in other parts of the jvm. >>> >>> Ioi, could you make all methods of CppVtableCloner out of line? >>> >>> The other changes look good, although I might have more requests >>> for comments. >>> >>> Thanks, >>> Coleen >>> >>> >>> Would be nice to have comments here too!! >>> >>> + intptr_t* start = md_top; >>> >>> This doesn't do anything (?) >>> >>> >>> Fixed. This was left over code. >>> >>> >>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>> >>> Why not zero the destination vtable in allocate? Or >>> does patching the vtable pointers call virtual >>> functions? You could prevent that so you don't need >>> this code. >>> >>> I added this comment: >>> >>> // During patching, some virtual methods may be called, so >>> at this point >>> // the vtables must contain valid methods (as filled in by >>> CppVtableCloner::allocate). >>> MetaspaceShared::patch_cpp_vtable_pointers(); >>> >>> // The vtable clones contain addresses of the current process. >>> // We don't want to write these addresses into the archive. >>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>> >>> + // Restore the vtable in case we invoke any virtual >>> methods. >>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>> Can this be restore_cpp_vtables since that's what it's >>> doing. The first is after the dump and the second call >>> is at UseSharedSpaces. A couple of comments in this >>> clone_cpp_vtables --> restore_cpp_vtables would be >>> nice. eg: >>> >>> I prefer to use the word clone. Otherwise when you just say >>> "vtable" it's not clear whether you're talking about the >>> original one (made by the c++ linker), or the cloned one in >>> the CDS archive. >>> >>> + static intptr_t* clone_vtable(const char* name, >>> intptr_t* p) { >>> + T tmp; // Allocate temporary dummy metadata object >>> to get vtable initialized >>> + CppVtabInfo* info = (CppVtabInfo*)p; >>> + int n = info->vtab_size(); >>> + intptr_t* srcvtab = vtab_of(tmp); >>> + intptr_t* dstvtab = info->vtab(); >>> + >>> + // We already checked (and, if necessary, adjusted n) >>> when the vtables were allocated, so we are >>> + // safe to do memcpy. >>> + if (PrintSharedSpaces) { >>> + tty->print_cr("%s copying %d vtable entries", name, n); >>> + } >>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>> + return dstvtab + n; >>> + } >>> >>> Done. I changed the wording >>> >>> T tmp; // Allocate temporary dummy metadata object to >>> get to the original vtable. >>> >>> As we are not really "initializing a vtable" here. >>> >>> Same with 'patch'. It'd be so much faster and easier to >>> read this code with more comments please. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>> >>> >>> Why are these testers here? >>> >>> >>> I updated the comment: >>> >>> // Used by CDS. These classes need to access the private >>> ConstantPool() constructor. >>> template friend class CppVtableTesterA; >>> template friend class CppVtableTesterB; >>> template friend class CppVtableCloner; >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>> >>> >>> Hi, >>> >>> This is the official review (follow up of >>> the "Determining the size of C++ vtables" >>> thread onhotspot-dev at openjdk.java.net >>> ). >>> >>> The new code has the same assumption as the >>> existing code in JDK 10: for a C++ object >>> that contains virtual methods (e.g., >>> ConstantPool), we assume the first intptr_t >>> slot of the object is a _vptr, which points >>> to a vtable, which consists of no more than >>> 150 intptr_t's. >>> >>> ConstantPool*p -->[ _vptr ] -------> [ >>> vtable slot 0 ] >>> [ field #0 ] [ >>> vtable slot 1 ] >>> [ field #1 ] [ >>> vtable slot 2 ] >>> [ field #2 ] [ >>> vtable slot 3 ] >>> [ .... ] [ >>> vtable slot 4] >>> [ >>> vtable slot 5 ] >>> [ >>> ... ] >>> >>> + In the existing code, we were pointing the >>> vtable slots to >>> code that's generated by HotSpot. >>> >>> + In the new code, we copy the vtable slots >>> from an existing >>> vtable (generated by the C++ linker). >>> >>> Per Thomas St?fe's advice, I don't try to >>> determine the size of the vtable (as that >>> would add one more compiler requirement >>> where new virtual methods added by a >>> subclass must be placed at a higher offset >>> in the vtable). >>> >>> Instead, I have added code in non-product >>> builds to ensure that the vtables are no >>> longer than 150 entries. You can run with >>> "-XX:+PrintSharedSpaces -Xshare:dump" to >>> print out the actual size of the vtables for >>> your particular platform: >>> >>> ConstantPool has 12 virtual methods >>> InstanceKlass has 113 virtual methods >>> InstanceClassLoaderKlass has 113 virtual >>> methods >>> InstanceMirrorKlass has 113 virtual methods >>> InstanceRefKlass has 113 virtual methods >>> Method has 12 virtual methods >>> ObjArrayKlass has 114 virtual methods >>> TypeArrayKlass has 114 virtual methods >>> >>> As mentioned in the code comments, if you >>> have an esoteric C++ compiler, the >>> verify_sufficient_size() function will >>> probably fail, but hopefully that would give >>> you some hints for porting this code. >>> >>> To avoid accidentally touching an unmapped >>> page, the code uses SafeFetchN for copying >>> the vtable contents, and would shrink the >>> vtable to less than 150 entries if >>> necessary. I can't test this for real, but >>> I've added some code to simulate an error: >>> >>> for (int i=0; i>> const intptr_t bad = intptr_t(0xdeadbeef); >>> intptr_t num = SafeFetchN(&srcvtab[i], >>> bad); >>> if (num == bad >>> // || i > 120 /* uncomment this >>> line to test */ >>> ) { >>> _info->set_vtab_size(i-1); >>> break; >>> } >>> dstvtab[i] = num; >>> } >>> >>> Results: >>> >>> + Removed 850 lines of CPU-dependent code >>> >>> + CDS image is about 50K smaller >>> >>> + Previously Metadata objects must live in >>> the read-write section in the CDS >>> archive, because their _vptr was updated >>> at run time. Now _vptr is no longer >>> updated, so ConstantPool can be moved to >>> the read-only section (see JDK-8171392). >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> >>> >>> >> > From jiangli.zhou at oracle.com Tue Mar 7 17:58:22 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 7 Mar 2017 09:58:22 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BE0C03.1040105@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> Message-ID: <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> Hi Ioi, I think there is no other usage of Universe::init_self_patching_vtbl_list() after you removed the ones from metaspaceShared.cpp. You can remove the function from universe.*. Thanks, Jiangli > On Mar 6, 2017, at 5:25 PM, Ioi Lam wrote: > > Hi Thomas & Coleen, > > Thanks for your comments. I have updated the rev: > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ > > [1] Switched back to computing the exact vtable size > [2] Move non-trivial functions outside of their class declaration > > Thanks > - Ioi > > On 3/6/17 8:51 AM, Thomas St?fe wrote: >> Hi Coleen and Ioi, >> >> I had to port C++ code to platforms with terrible compilers for a time in my life, that is why I like code to be as portable as possible. That said, you are right in your argumentation, the SafeFetch solution is not terribly elegant and Ioi's original way of determining the vtable size is cleaner. >> >> I did some checks on some of our architectures with a test similar to Ioi's and on a first glance it seems to work for simple cases (single and public inheritance) on ppc (AIX) and Linux ia64. Although the vtables seemed to me to contain function descriptors, not real pointers to code, so this is something to keep in mind. But if the live vtable are copied, the function descriptors they contain should point to valid code too, so it should not matter. Just to remember to not expect every slot in the array to be a valid code pointer. >> >> So, in short, I remove my objection to Ioi's original solution, as far as that matters. >> >> I still think we rely on a lot here: Contiguous vtable containing absolute memory addresses, vtable pointer at start of object and vtable entries to be ordered from base->derived class. So I wonder how much effort it would be (now or in the future as a separate change) to have a fallback where - at loading time - instead of copying vtables the vtable pointers in the objects were fixed up to point to the new live vtables? I know this would be more expensive and potentially defeat the point of shared classes. But maybe not, it depends on how many objects are there, no? >> >> Kind Regards, Thomas >> >> On Sun, Mar 5, 2017 at 4:17 PM, > wrote: >> >> >> Ioi, Some comments inline (where no comments, insert "ok") :) >> >> >> On 3/2/17 10:37 PM, Ioi Lam wrote: >> >> Hi Coleen, >> >> Thanks for the comments. I have updated the webrev. See >> in-line for responses. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >> >> >> >> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >> wrote: >> >> >> Ioi >> I like the concept of this a lot but have some stylistic >> comments to help people reading this code later. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >> >> >> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >> save many characters, especially in CppVtableInfo/Testers >> >> Done. >> >> + // Start at slot 1, because slot 0 may be RTTI (on >> Solaris/Sparc) >> + int i; >> + for (i=1; ; i++) { >> Since you're using 'i' later, can you rename it to >> something descriptive. Or have another variable >> "vtable_length" to use later. This looks like an old >> style for loop. >> >> Done >> >> Can the functions for CppVtableInfo be declared outside of >> the class declaration? They don't need to be inline and >> then the debug code for testing the vtable size can be not >> in the middle of the class declaration. Then you can >> move the Tester classes to inside the same #ifndef PRODUCT >> block. >> >> Can you put #endif // PRODUCT when the ifdef covers >> several lines of code? >> >> Done >> >> vtab_of could be more descriptive, like cpp_vtable_for(). >> >> I changed to vtable_of(). Because the class name is already >> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >> >> Was PrintSharedSpaces was never converted to UL? >> >> Right. I've filed >> https://bugs.openjdk.java.net/browse/JDK-8176132 >> >> (-XX:+PrintSharedSpaces should be converted to use Unified >> Logging.) >> >> + int n = MAX_VTABLE_SIZE; >> >> Can you propagate MAX_VTABLE_SIZE to the places where it's >> used. n isn't descriptive. This starts out with >> max_vtable_size and then changes the size. Reusing 'n' >> makes this really hard to follow. Not having a comment >> that we only allocate enough slots for the vtable makes it >> hard too. >> >> + // allocate CppVtableInfo in the MD section + _info = >> (CppVtabInfo*)md_top; >> + _info->set_vtab_size(n); // initially set to max_vtable_size >> + + // allocate temporary local instance of the metadata >> type T + T tmp; >> + intptr_t* srcvtab = vtab_of(tmp); >> + intptr_t* dstvtab = _info->vtab(); >> + >> >> Fixed. >> >> Something like that for comments. dstvtab is the >> destination_vtable in the MD section. >> >> >> I've dropped the md_ prefix from the functions that deal with >> the vtables, since they shouldn't care whether it's the "MD" >> section or not. Now it looks like this: >> >> // Allocate and initialize the C++ vtables, starting from top, >> but do not go past end. >> intptr_t* >> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >> intptr_t* end) { >> assert(DumpSharedSpaces, "dump-time only"); >> // Layout (each slot is a intptr_t): >> // [number of slots in the first vtable = n1] >> // [ slots for the first vtable] >> // [number of slots in the first second = n2] >> // [ slots for the second vtable] >> // ... >> // The order of the vtables is the same as the >> CPP_VTAB_PATCH_TYPES_DO macro. >> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >> return top; >> } >> >> + for (int i=0; i> + const intptr_t bad = intptr_t(0xdeadbeef); >> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >> + if (num == bad >> + // || i > 120 /* uncomment this line to test */ >> + ) { >> + _info->set_vtab_size(i-1); >> + break; >> + } >> + dstvtab[i] = num; >> + } >> I dont understand this code. You get deadbeef for a bad >> value if SafeFetchN gets a fault but why would it get a >> fault at the end of the metadata's vtable? Couldn't it >> just run onto the next vtable? I think your original way >> of counting vtable entries might be better (sorry I didn't >> have time to study that thread). >> >> I've modified the comments to this. Does it make sense to you? >> >> // It is not always safe to call memcpy(), because >> srcvtable might be shorter than >> // MAX_VTABLE_SIZE, and the C++ linker might have placed >> the vtable at the very >> // end of the last page of libjvm.so. Crossing over to the >> next page might >> // cause a page fault. >> >> My fear is the JVM would suddenly start crashing because the >> order of .o files have changed on the linker's command line, >> or if you enable some special linker optimization flags. It's >> better safe than sorry. >> >> >> This wasn't exactly what I was not understanding. I didn't see >> that you are copying 120 entries from the old vtable and junk >> memory beyond the old vtable, unless you get a segv, in which case >> you copy less. I don't think you should copy random memory into >> the vtable in the archive. This doesn't seem secure, even with >> the segv protection. >> >> Since we already have assumptions about C++ vtable layout in the >> code and it's mostly specified by various ABIs, and you have the >> assert code, I think I would prefer that you copy only the vtable >> entries into the archive. I guess Thomas Stuefe had a different >> opinion. I've read the original thread. Two points: >> >> If new C++ compiler implementations add a discontigous vtable, >> both the SafeFetchN and subclass additional virtual function at >> end implementation will fail. I don't think C++ implementations >> would do this and a contiguous vtable as first in the instance has >> been standard for years. If our metadata adds multiple >> inheritance, the same issue would be a problem for both >> implementations, as well as for the implementation we have before >> Ioi's fix. >> >> Ioi's subclass adding virtual function method would work for any >> esoteric C++ implementations in my memory, except the vptr for the >> old DECC++ compiler was after the nonstatic data members (which >> would fail with all of our implementations). >> >> Since the code is there anyway for debug purposes, we're not >> saving code by implementing SafeFetchN. The SafeFetchN >> implementation isn't obvious at all what it's doing, and requires >> better comments, especially if you don't know already what >> SafeFetchN does. It looks really cryptic. The poisoned values >> also bothered me in that they overload other poisoned values in >> other parts of the jvm. >> >> Ioi, could you make all methods of CppVtableCloner out of line? >> >> The other changes look good, although I might have more requests >> for comments. >> >> Thanks, >> Coleen >> >> >> Would be nice to have comments here too!! >> >> + intptr_t* start = md_top; >> >> This doesn't do anything (?) >> >> >> Fixed. This was left over code. >> >> >> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >> >> Why not zero the destination vtable in allocate? Or does >> patching the vtable pointers call virtual functions? You >> could prevent that so you don't need this code. >> >> I added this comment: >> >> // During patching, some virtual methods may be called, so >> at this point >> // the vtables must contain valid methods (as filled in by >> CppVtableCloner::allocate). >> MetaspaceShared::patch_cpp_vtable_pointers(); >> >> // The vtable clones contain addresses of the current process. >> // We don't want to write these addresses into the archive. >> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >> >> + // Restore the vtable in case we invoke any virtual methods. >> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >> Can this be restore_cpp_vtables since that's what it's >> doing. The first is after the dump and the second call is >> at UseSharedSpaces. A couple of comments in this >> clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: >> >> I prefer to use the word clone. Otherwise when you just say >> "vtable" it's not clear whether you're talking about the >> original one (made by the c++ linker), or the cloned one in >> the CDS archive. >> >> + static intptr_t* clone_vtable(const char* name, >> intptr_t* p) { >> + T tmp; // Allocate temporary dummy metadata object to >> get vtable initialized >> + CppVtabInfo* info = (CppVtabInfo*)p; >> + int n = info->vtab_size(); >> + intptr_t* srcvtab = vtab_of(tmp); >> + intptr_t* dstvtab = info->vtab(); >> + >> + // We already checked (and, if necessary, adjusted n) >> when the vtables were allocated, so we are >> + // safe to do memcpy. >> + if (PrintSharedSpaces) { >> + tty->print_cr("%s copying %d vtable entries", name, n); >> + } >> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >> + return dstvtab + n; >> + } >> >> Done. I changed the wording >> >> T tmp; // Allocate temporary dummy metadata object to get >> to the original vtable. >> >> As we are not really "initializing a vtable" here. >> >> Same with 'patch'. It'd be so much faster and easier to >> read this code with more comments please. >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >> >> >> Why are these testers here? >> >> >> I updated the comment: >> >> // Used by CDS. These classes need to access the private >> ConstantPool() constructor. >> template friend class CppVtableTesterA; >> template friend class CppVtableTesterB; >> template friend class CppVtableCloner; >> >> >> Thanks >> - Ioi >> >> >> On 3/1/17 3:25 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8005165 >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >> >> >> Hi, >> >> This is the official review (follow up of the >> "Determining the size of C++ vtables" thread >> onhotspot-dev at openjdk.java.net >> ). >> >> The new code has the same assumption as the >> existing code in JDK 10: for a C++ object that >> contains virtual methods (e.g., ConstantPool), >> we assume the first intptr_t slot of the >> object is a _vptr, which points to a vtable, >> which consists of no more than 150 intptr_t's. >> >> ConstantPool*p -->[ _vptr ] -------> [ >> vtable slot 0 ] >> [ field #0 ] [ >> vtable slot 1 ] >> [ field #1 ] [ >> vtable slot 2 ] >> [ field #2 ] [ >> vtable slot 3 ] >> [ .... ] [ >> vtable slot 4] >> [ >> vtable slot 5 ] >> [ >> ... ] >> >> + In the existing code, we were pointing the >> vtable slots to >> code that's generated by HotSpot. >> >> + In the new code, we copy the vtable slots >> from an existing >> vtable (generated by the C++ linker). >> >> Per Thomas St?fe's advice, I don't try to >> determine the size of the vtable (as that >> would add one more compiler requirement where >> new virtual methods added by a subclass must >> be placed at a higher offset in the vtable). >> >> Instead, I have added code in non-product >> builds to ensure that the vtables are no >> longer than 150 entries. You can run with >> "-XX:+PrintSharedSpaces -Xshare:dump" to print >> out the actual size of the vtables for your >> particular platform: >> >> ConstantPool has 12 virtual methods >> InstanceKlass has 113 virtual methods >> InstanceClassLoaderKlass has 113 virtual methods >> InstanceMirrorKlass has 113 virtual methods >> InstanceRefKlass has 113 virtual methods >> Method has 12 virtual methods >> ObjArrayKlass has 114 virtual methods >> TypeArrayKlass has 114 virtual methods >> >> As mentioned in the code comments, if you have >> an esoteric C++ compiler, the >> verify_sufficient_size() function will >> probably fail, but hopefully that would give >> you some hints for porting this code. >> >> To avoid accidentally touching an unmapped >> page, the code uses SafeFetchN for copying the >> vtable contents, and would shrink the vtable >> to less than 150 entries if necessary. I can't >> test this for real, but I've added some code >> to simulate an error: >> >> for (int i=0; i> const intptr_t bad = intptr_t(0xdeadbeef); >> intptr_t num = SafeFetchN(&srcvtab[i], bad); >> if (num == bad >> // || i > 120 /* uncomment this line >> to test */ >> ) { >> _info->set_vtab_size(i-1); >> break; >> } >> dstvtab[i] = num; >> } >> >> Results: >> >> + Removed 850 lines of CPU-dependent code >> >> + CDS image is about 50K smaller >> >> + Previously Metadata objects must live in the >> read-write section in the CDS >> archive, because their _vptr was updated at >> run time. Now _vptr is no longer >> updated, so ConstantPool can be moved to the >> read-only section (see JDK-8171392). >> >> Thanks >> - Ioi >> >> >> >> >> >> >> > From christian.tornqvist at oracle.com Tue Mar 7 20:02:55 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Tue, 7 Mar 2017 15:02:55 -0500 Subject: RFR(M): 8175300 - Enable artifact resolution for jtreg tests Message-ID: <37ec01d2977d$cfb1fc20$6f15f460$@oracle.com> Hi everyone, Please review this change that adds the ability to have external dependencies in jtreg tests. As a proof of concept, I've added Scimark to the Hotspot repository. The concept is this, annotate the test with one or more @Artifact annotation describing the external dependencies, these can then either be resolved manually (using -Djdk.test.lib.artifacts.) or automatically using a custom ArtifactManager. The make changes were contributed by Erik Joelsson. These changes have been verified by running the Scimark test locally (using make) and in JPRT. Bugs: https://bugs.openjdk.java.net/browse/JDK-8175300 Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8175300/webrev.00/ Thanks, Christian From ioi.lam at oracle.com Tue Mar 7 21:24:00 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 07 Mar 2017 13:24:00 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> Message-ID: <58BF24F0.6090704@oracle.com> Hi Jiangli, Good catch! When I try to remove the code, I actually found my last version (v04) would not work with Method::is_valid_method(). The following assert would fail. This means that shared methods be skipped when we walk the stack (such as when doing a crash dump): void Method::restore_unshareable_info(TRAPS) { - assert(is_method(), "ensure C++ vtable is restored"); + assert(is_method() && is_valid_method(), "ensure C++ vtable is restored"); So I made a more comprehensive fix to make sure Method::is_valid_method works. Here's a diff from the previous version: http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v05/ Thanks - Ioi On 3/7/17 9:58 AM, Jiangli Zhou wrote: > Hi Ioi, > > I think there is no other usage of Universe::init_self_patching_vtbl_list() after you removed the ones from metaspaceShared.cpp. You can remove the function from universe.*. > > Thanks, > Jiangli > >> On Mar 6, 2017, at 5:25 PM, Ioi Lam wrote: >> >> Hi Thomas & Coleen, >> >> Thanks for your comments. I have updated the rev: >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ >> >> [1] Switched back to computing the exact vtable size >> [2] Move non-trivial functions outside of their class declaration >> >> Thanks >> - Ioi >> >> On 3/6/17 8:51 AM, Thomas St?fe wrote: >>> Hi Coleen and Ioi, >>> >>> I had to port C++ code to platforms with terrible compilers for a time in my life, that is why I like code to be as portable as possible. That said, you are right in your argumentation, the SafeFetch solution is not terribly elegant and Ioi's original way of determining the vtable size is cleaner. >>> >>> I did some checks on some of our architectures with a test similar to Ioi's and on a first glance it seems to work for simple cases (single and public inheritance) on ppc (AIX) and Linux ia64. Although the vtables seemed to me to contain function descriptors, not real pointers to code, so this is something to keep in mind. But if the live vtable are copied, the function descriptors they contain should point to valid code too, so it should not matter. Just to remember to not expect every slot in the array to be a valid code pointer. >>> >>> So, in short, I remove my objection to Ioi's original solution, as far as that matters. >>> >>> I still think we rely on a lot here: Contiguous vtable containing absolute memory addresses, vtable pointer at start of object and vtable entries to be ordered from base->derived class. So I wonder how much effort it would be (now or in the future as a separate change) to have a fallback where - at loading time - instead of copying vtables the vtable pointers in the objects were fixed up to point to the new live vtables? I know this would be more expensive and potentially defeat the point of shared classes. But maybe not, it depends on how many objects are there, no? >>> >>> Kind Regards, Thomas >>> >>> On Sun, Mar 5, 2017 at 4:17 PM, > wrote: >>> >>> >>> Ioi, Some comments inline (where no comments, insert "ok") :) >>> >>> >>> On 3/2/17 10:37 PM, Ioi Lam wrote: >>> >>> Hi Coleen, >>> >>> Thanks for the comments. I have updated the webrev. See >>> in-line for responses. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>> >>> >>> >>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >>> wrote: >>> >>> >>> Ioi >>> I like the concept of this a lot but have some stylistic >>> comments to help people reading this code later. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>> >>> >>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >>> save many characters, especially in CppVtableInfo/Testers >>> >>> Done. >>> >>> + // Start at slot 1, because slot 0 may be RTTI (on >>> Solaris/Sparc) >>> + int i; >>> + for (i=1; ; i++) { >>> Since you're using 'i' later, can you rename it to >>> something descriptive. Or have another variable >>> "vtable_length" to use later. This looks like an old >>> style for loop. >>> >>> Done >>> >>> Can the functions for CppVtableInfo be declared outside of >>> the class declaration? They don't need to be inline and >>> then the debug code for testing the vtable size can be not >>> in the middle of the class declaration. Then you can >>> move the Tester classes to inside the same #ifndef PRODUCT >>> block. >>> >>> Can you put #endif // PRODUCT when the ifdef covers >>> several lines of code? >>> >>> Done >>> >>> vtab_of could be more descriptive, like cpp_vtable_for(). >>> >>> I changed to vtable_of(). Because the class name is already >>> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >>> >>> Was PrintSharedSpaces was never converted to UL? >>> >>> Right. I've filed >>> https://bugs.openjdk.java.net/browse/JDK-8176132 >>> >>> (-XX:+PrintSharedSpaces should be converted to use Unified >>> Logging.) >>> >>> + int n = MAX_VTABLE_SIZE; >>> >>> Can you propagate MAX_VTABLE_SIZE to the places where it's >>> used. n isn't descriptive. This starts out with >>> max_vtable_size and then changes the size. Reusing 'n' >>> makes this really hard to follow. Not having a comment >>> that we only allocate enough slots for the vtable makes it >>> hard too. >>> >>> + // allocate CppVtableInfo in the MD section + _info = >>> (CppVtabInfo*)md_top; >>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>> + + // allocate temporary local instance of the metadata >>> type T + T tmp; >>> + intptr_t* srcvtab = vtab_of(tmp); >>> + intptr_t* dstvtab = _info->vtab(); >>> + >>> >>> Fixed. >>> >>> Something like that for comments. dstvtab is the >>> destination_vtable in the MD section. >>> >>> >>> I've dropped the md_ prefix from the functions that deal with >>> the vtables, since they shouldn't care whether it's the "MD" >>> section or not. Now it looks like this: >>> >>> // Allocate and initialize the C++ vtables, starting from top, >>> but do not go past end. >>> intptr_t* >>> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>> intptr_t* end) { >>> assert(DumpSharedSpaces, "dump-time only"); >>> // Layout (each slot is a intptr_t): >>> // [number of slots in the first vtable = n1] >>> // [ slots for the first vtable] >>> // [number of slots in the first second = n2] >>> // [ slots for the second vtable] >>> // ... >>> // The order of the vtables is the same as the >>> CPP_VTAB_PATCH_TYPES_DO macro. >>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>> return top; >>> } >>> >>> + for (int i=0; i>> + const intptr_t bad = intptr_t(0xdeadbeef); >>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>> + if (num == bad >>> + // || i > 120 /* uncomment this line to test */ >>> + ) { >>> + _info->set_vtab_size(i-1); >>> + break; >>> + } >>> + dstvtab[i] = num; >>> + } >>> I dont understand this code. You get deadbeef for a bad >>> value if SafeFetchN gets a fault but why would it get a >>> fault at the end of the metadata's vtable? Couldn't it >>> just run onto the next vtable? I think your original way >>> of counting vtable entries might be better (sorry I didn't >>> have time to study that thread). >>> >>> I've modified the comments to this. Does it make sense to you? >>> >>> // It is not always safe to call memcpy(), because >>> srcvtable might be shorter than >>> // MAX_VTABLE_SIZE, and the C++ linker might have placed >>> the vtable at the very >>> // end of the last page of libjvm.so. Crossing over to the >>> next page might >>> // cause a page fault. >>> >>> My fear is the JVM would suddenly start crashing because the >>> order of .o files have changed on the linker's command line, >>> or if you enable some special linker optimization flags. It's >>> better safe than sorry. >>> >>> >>> This wasn't exactly what I was not understanding. I didn't see >>> that you are copying 120 entries from the old vtable and junk >>> memory beyond the old vtable, unless you get a segv, in which case >>> you copy less. I don't think you should copy random memory into >>> the vtable in the archive. This doesn't seem secure, even with >>> the segv protection. >>> >>> Since we already have assumptions about C++ vtable layout in the >>> code and it's mostly specified by various ABIs, and you have the >>> assert code, I think I would prefer that you copy only the vtable >>> entries into the archive. I guess Thomas Stuefe had a different >>> opinion. I've read the original thread. Two points: >>> >>> If new C++ compiler implementations add a discontigous vtable, >>> both the SafeFetchN and subclass additional virtual function at >>> end implementation will fail. I don't think C++ implementations >>> would do this and a contiguous vtable as first in the instance has >>> been standard for years. If our metadata adds multiple >>> inheritance, the same issue would be a problem for both >>> implementations, as well as for the implementation we have before >>> Ioi's fix. >>> >>> Ioi's subclass adding virtual function method would work for any >>> esoteric C++ implementations in my memory, except the vptr for the >>> old DECC++ compiler was after the nonstatic data members (which >>> would fail with all of our implementations). >>> >>> Since the code is there anyway for debug purposes, we're not >>> saving code by implementing SafeFetchN. The SafeFetchN >>> implementation isn't obvious at all what it's doing, and requires >>> better comments, especially if you don't know already what >>> SafeFetchN does. It looks really cryptic. The poisoned values >>> also bothered me in that they overload other poisoned values in >>> other parts of the jvm. >>> >>> Ioi, could you make all methods of CppVtableCloner out of line? >>> >>> The other changes look good, although I might have more requests >>> for comments. >>> >>> Thanks, >>> Coleen >>> >>> >>> Would be nice to have comments here too!! >>> >>> + intptr_t* start = md_top; >>> >>> This doesn't do anything (?) >>> >>> >>> Fixed. This was left over code. >>> >>> >>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>> >>> Why not zero the destination vtable in allocate? Or does >>> patching the vtable pointers call virtual functions? You >>> could prevent that so you don't need this code. >>> >>> I added this comment: >>> >>> // During patching, some virtual methods may be called, so >>> at this point >>> // the vtables must contain valid methods (as filled in by >>> CppVtableCloner::allocate). >>> MetaspaceShared::patch_cpp_vtable_pointers(); >>> >>> // The vtable clones contain addresses of the current process. >>> // We don't want to write these addresses into the archive. >>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>> >>> + // Restore the vtable in case we invoke any virtual methods. >>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>> Can this be restore_cpp_vtables since that's what it's >>> doing. The first is after the dump and the second call is >>> at UseSharedSpaces. A couple of comments in this >>> clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: >>> >>> I prefer to use the word clone. Otherwise when you just say >>> "vtable" it's not clear whether you're talking about the >>> original one (made by the c++ linker), or the cloned one in >>> the CDS archive. >>> >>> + static intptr_t* clone_vtable(const char* name, >>> intptr_t* p) { >>> + T tmp; // Allocate temporary dummy metadata object to >>> get vtable initialized >>> + CppVtabInfo* info = (CppVtabInfo*)p; >>> + int n = info->vtab_size(); >>> + intptr_t* srcvtab = vtab_of(tmp); >>> + intptr_t* dstvtab = info->vtab(); >>> + >>> + // We already checked (and, if necessary, adjusted n) >>> when the vtables were allocated, so we are >>> + // safe to do memcpy. >>> + if (PrintSharedSpaces) { >>> + tty->print_cr("%s copying %d vtable entries", name, n); >>> + } >>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>> + return dstvtab + n; >>> + } >>> >>> Done. I changed the wording >>> >>> T tmp; // Allocate temporary dummy metadata object to get >>> to the original vtable. >>> >>> As we are not really "initializing a vtable" here. >>> >>> Same with 'patch'. It'd be so much faster and easier to >>> read this code with more comments please. >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>> >>> >>> Why are these testers here? >>> >>> >>> I updated the comment: >>> >>> // Used by CDS. These classes need to access the private >>> ConstantPool() constructor. >>> template friend class CppVtableTesterA; >>> template friend class CppVtableTesterB; >>> template friend class CppVtableCloner; >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>> >>> >>> Hi, >>> >>> This is the official review (follow up of the >>> "Determining the size of C++ vtables" thread >>> onhotspot-dev at openjdk.java.net >>> ). >>> >>> The new code has the same assumption as the >>> existing code in JDK 10: for a C++ object that >>> contains virtual methods (e.g., ConstantPool), >>> we assume the first intptr_t slot of the >>> object is a _vptr, which points to a vtable, >>> which consists of no more than 150 intptr_t's. >>> >>> ConstantPool*p -->[ _vptr ] -------> [ >>> vtable slot 0 ] >>> [ field #0 ] [ >>> vtable slot 1 ] >>> [ field #1 ] [ >>> vtable slot 2 ] >>> [ field #2 ] [ >>> vtable slot 3 ] >>> [ .... ] [ >>> vtable slot 4] >>> [ >>> vtable slot 5 ] >>> [ >>> ... ] >>> >>> + In the existing code, we were pointing the >>> vtable slots to >>> code that's generated by HotSpot. >>> >>> + In the new code, we copy the vtable slots >>> from an existing >>> vtable (generated by the C++ linker). >>> >>> Per Thomas St?fe's advice, I don't try to >>> determine the size of the vtable (as that >>> would add one more compiler requirement where >>> new virtual methods added by a subclass must >>> be placed at a higher offset in the vtable). >>> >>> Instead, I have added code in non-product >>> builds to ensure that the vtables are no >>> longer than 150 entries. You can run with >>> "-XX:+PrintSharedSpaces -Xshare:dump" to print >>> out the actual size of the vtables for your >>> particular platform: >>> >>> ConstantPool has 12 virtual methods >>> InstanceKlass has 113 virtual methods >>> InstanceClassLoaderKlass has 113 virtual methods >>> InstanceMirrorKlass has 113 virtual methods >>> InstanceRefKlass has 113 virtual methods >>> Method has 12 virtual methods >>> ObjArrayKlass has 114 virtual methods >>> TypeArrayKlass has 114 virtual methods >>> >>> As mentioned in the code comments, if you have >>> an esoteric C++ compiler, the >>> verify_sufficient_size() function will >>> probably fail, but hopefully that would give >>> you some hints for porting this code. >>> >>> To avoid accidentally touching an unmapped >>> page, the code uses SafeFetchN for copying the >>> vtable contents, and would shrink the vtable >>> to less than 150 entries if necessary. I can't >>> test this for real, but I've added some code >>> to simulate an error: >>> >>> for (int i=0; i>> const intptr_t bad = intptr_t(0xdeadbeef); >>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>> if (num == bad >>> // || i > 120 /* uncomment this line >>> to test */ >>> ) { >>> _info->set_vtab_size(i-1); >>> break; >>> } >>> dstvtab[i] = num; >>> } >>> >>> Results: >>> >>> + Removed 850 lines of CPU-dependent code >>> >>> + CDS image is about 50K smaller >>> >>> + Previously Metadata objects must live in the >>> read-write section in the CDS >>> archive, because their _vptr was updated at >>> run time. Now _vptr is no longer >>> updated, so ConstantPool can be moved to the >>> read-only section (see JDK-8171392). >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> >>> >>> From coleen.phillimore at oracle.com Tue Mar 7 23:01:47 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 7 Mar 2017 18:01:47 -0500 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BF24F0.6090704@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> <58BF24F0.6090704@oracle.com> Message-ID: Yes, good find. This looks good too. Nice to remove init_self_patching_vtbl_list and comments referring to it. thanks, Coleen On 3/7/17 4:24 PM, Ioi Lam wrote: > Hi Jiangli, > > Good catch! > > When I try to remove the code, I actually found my last version (v04) > would not work with Method::is_valid_method(). The following assert > would fail. This means that shared methods be skipped when we walk the > stack (such as when doing a crash dump): > > void Method::restore_unshareable_info(TRAPS) { > - assert(is_method(), "ensure C++ vtable is restored"); > + assert(is_method() && is_valid_method(), "ensure C++ vtable is > restored"); > > So I made a more comprehensive fix to make sure > Method::is_valid_method works. Here's a diff from the previous version: > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v05/ > > > Thanks > - Ioi > > > On 3/7/17 9:58 AM, Jiangli Zhou wrote: >> Hi Ioi, >> >> I think there is no other usage of >> Universe::init_self_patching_vtbl_list() after you removed the ones >> from metaspaceShared.cpp. You can remove the function from universe.*. >> >> Thanks, >> Jiangli >> >>> On Mar 6, 2017, at 5:25 PM, Ioi Lam wrote: >>> >>> Hi Thomas & Coleen, >>> >>> Thanks for your comments. I have updated the rev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ >>> >>> >>> [1] Switched back to computing the exact vtable size >>> [2] Move non-trivial functions outside of their class declaration >>> >>> Thanks >>> - Ioi >>> >>> On 3/6/17 8:51 AM, Thomas St?fe wrote: >>>> Hi Coleen and Ioi, >>>> >>>> I had to port C++ code to platforms with terrible compilers for a >>>> time in my life, that is why I like code to be as portable as >>>> possible. That said, you are right in your argumentation, the >>>> SafeFetch solution is not terribly elegant and Ioi's original way >>>> of determining the vtable size is cleaner. >>>> >>>> I did some checks on some of our architectures with a test similar >>>> to Ioi's and on a first glance it seems to work for simple cases >>>> (single and public inheritance) on ppc (AIX) and Linux ia64. >>>> Although the vtables seemed to me to contain function descriptors, >>>> not real pointers to code, so this is something to keep in mind. >>>> But if the live vtable are copied, the function descriptors they >>>> contain should point to valid code too, so it should not matter. >>>> Just to remember to not expect every slot in the array to be a >>>> valid code pointer. >>>> >>>> So, in short, I remove my objection to Ioi's original solution, as >>>> far as that matters. >>>> >>>> I still think we rely on a lot here: Contiguous vtable containing >>>> absolute memory addresses, vtable pointer at start of object and >>>> vtable entries to be ordered from base->derived class. So I wonder >>>> how much effort it would be (now or in the future as a separate >>>> change) to have a fallback where - at loading time - instead of >>>> copying vtables the vtable pointers in the objects were fixed up to >>>> point to the new live vtables? I know this would be more expensive >>>> and potentially defeat the point of shared classes. But maybe not, >>>> it depends on how many objects are there, no? >>>> >>>> Kind Regards, Thomas >>>> >>>> On Sun, Mar 5, 2017 at 4:17 PM, >>> > wrote: >>>> >>>> >>>> Ioi, Some comments inline (where no comments, insert "ok") :) >>>> >>>> >>>> On 3/2/17 10:37 PM, Ioi Lam wrote: >>>> >>>> Hi Coleen, >>>> >>>> Thanks for the comments. I have updated the webrev. See >>>> in-line for responses. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>>> >>>> >>>> >>>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >>>> wrote: >>>> >>>> >>>> Ioi >>>> I like the concept of this a lot but have some stylistic >>>> comments to help people reading this code later. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>>> >>>> >>>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >>>> save many characters, especially in CppVtableInfo/Testers >>>> >>>> Done. >>>> >>>> + // Start at slot 1, because slot 0 may be RTTI (on >>>> Solaris/Sparc) >>>> + int i; >>>> + for (i=1; ; i++) { >>>> Since you're using 'i' later, can you rename it to >>>> something descriptive. Or have another variable >>>> "vtable_length" to use later. This looks like an old >>>> style for loop. >>>> >>>> Done >>>> >>>> Can the functions for CppVtableInfo be declared outside of >>>> the class declaration? They don't need to be inline and >>>> then the debug code for testing the vtable size can be not >>>> in the middle of the class declaration. Then you can >>>> move the Tester classes to inside the same #ifndef PRODUCT >>>> block. >>>> >>>> Can you put #endif // PRODUCT when the ifdef covers >>>> several lines of code? >>>> >>>> Done >>>> >>>> vtab_of could be more descriptive, like cpp_vtable_for(). >>>> >>>> I changed to vtable_of(). Because the class name is already >>>> CppVtableCloner, repeating the word "cpp" seems repetitive >>>> to me. >>>> >>>> Was PrintSharedSpaces was never converted to UL? >>>> >>>> Right. I've filed >>>> https://bugs.openjdk.java.net/browse/JDK-8176132 >>>> >>>> (-XX:+PrintSharedSpaces should be converted to use Unified >>>> Logging.) >>>> >>>> + int n = MAX_VTABLE_SIZE; >>>> >>>> Can you propagate MAX_VTABLE_SIZE to the places where it's >>>> used. n isn't descriptive. This starts out with >>>> max_vtable_size and then changes the size. Reusing 'n' >>>> makes this really hard to follow. Not having a comment >>>> that we only allocate enough slots for the vtable makes it >>>> hard too. >>>> >>>> + // allocate CppVtableInfo in the MD section + _info = >>>> (CppVtabInfo*)md_top; >>>> + _info->set_vtab_size(n); // initially set to >>>> max_vtable_size >>>> + + // allocate temporary local instance of the metadata >>>> type T + T tmp; >>>> + intptr_t* srcvtab = vtab_of(tmp); >>>> + intptr_t* dstvtab = _info->vtab(); >>>> + >>>> >>>> Fixed. >>>> >>>> Something like that for comments. dstvtab is the >>>> destination_vtable in the MD section. >>>> >>>> >>>> I've dropped the md_ prefix from the functions that deal with >>>> the vtables, since they shouldn't care whether it's the "MD" >>>> section or not. Now it looks like this: >>>> >>>> // Allocate and initialize the C++ vtables, starting from top, >>>> but do not go past end. >>>> intptr_t* >>>> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>>> intptr_t* end) { >>>> assert(DumpSharedSpaces, "dump-time only"); >>>> // Layout (each slot is a intptr_t): >>>> // [number of slots in the first vtable = n1] >>>> // [ slots for the first vtable] >>>> // [number of slots in the first second = n2] >>>> // [ slots for the second vtable] >>>> // ... >>>> // The order of the vtables is the same as the >>>> CPP_VTAB_PATCH_TYPES_DO macro. >>>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>>> return top; >>>> } >>>> >>>> + for (int i=0; i>>> + const intptr_t bad = intptr_t(0xdeadbeef); >>>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>> + if (num == bad >>>> + // || i > 120 /* uncomment this line to test */ >>>> + ) { >>>> + _info->set_vtab_size(i-1); >>>> + break; >>>> + } >>>> + dstvtab[i] = num; >>>> + } >>>> I dont understand this code. You get deadbeef for a bad >>>> value if SafeFetchN gets a fault but why would it get a >>>> fault at the end of the metadata's vtable? Couldn't it >>>> just run onto the next vtable? I think your original way >>>> of counting vtable entries might be better (sorry I didn't >>>> have time to study that thread). >>>> >>>> I've modified the comments to this. Does it make sense to you? >>>> >>>> // It is not always safe to call memcpy(), because >>>> srcvtable might be shorter than >>>> // MAX_VTABLE_SIZE, and the C++ linker might have placed >>>> the vtable at the very >>>> // end of the last page of libjvm.so. Crossing over to the >>>> next page might >>>> // cause a page fault. >>>> >>>> My fear is the JVM would suddenly start crashing because the >>>> order of .o files have changed on the linker's command line, >>>> or if you enable some special linker optimization flags. It's >>>> better safe than sorry. >>>> >>>> >>>> This wasn't exactly what I was not understanding. I didn't see >>>> that you are copying 120 entries from the old vtable and junk >>>> memory beyond the old vtable, unless you get a segv, in which case >>>> you copy less. I don't think you should copy random memory into >>>> the vtable in the archive. This doesn't seem secure, even with >>>> the segv protection. >>>> >>>> Since we already have assumptions about C++ vtable layout in the >>>> code and it's mostly specified by various ABIs, and you have the >>>> assert code, I think I would prefer that you copy only the vtable >>>> entries into the archive. I guess Thomas Stuefe had a different >>>> opinion. I've read the original thread. Two points: >>>> >>>> If new C++ compiler implementations add a discontigous vtable, >>>> both the SafeFetchN and subclass additional virtual function at >>>> end implementation will fail. I don't think C++ implementations >>>> would do this and a contiguous vtable as first in the instance has >>>> been standard for years. If our metadata adds multiple >>>> inheritance, the same issue would be a problem for both >>>> implementations, as well as for the implementation we have before >>>> Ioi's fix. >>>> >>>> Ioi's subclass adding virtual function method would work for any >>>> esoteric C++ implementations in my memory, except the vptr for the >>>> old DECC++ compiler was after the nonstatic data members (which >>>> would fail with all of our implementations). >>>> >>>> Since the code is there anyway for debug purposes, we're not >>>> saving code by implementing SafeFetchN. The SafeFetchN >>>> implementation isn't obvious at all what it's doing, and requires >>>> better comments, especially if you don't know already what >>>> SafeFetchN does. It looks really cryptic. The poisoned values >>>> also bothered me in that they overload other poisoned values in >>>> other parts of the jvm. >>>> >>>> Ioi, could you make all methods of CppVtableCloner out of line? >>>> >>>> The other changes look good, although I might have more requests >>>> for comments. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> Would be nice to have comments here too!! >>>> >>>> + intptr_t* start = md_top; >>>> >>>> This doesn't do anything (?) >>>> >>>> >>>> Fixed. This was left over code. >>>> >>>> >>>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>> >>>> Why not zero the destination vtable in allocate? Or does >>>> patching the vtable pointers call virtual functions? You >>>> could prevent that so you don't need this code. >>>> >>>> I added this comment: >>>> >>>> // During patching, some virtual methods may be called, so >>>> at this point >>>> // the vtables must contain valid methods (as filled in by >>>> CppVtableCloner::allocate). >>>> MetaspaceShared::patch_cpp_vtable_pointers(); >>>> >>>> // The vtable clones contain addresses of the current >>>> process. >>>> // We don't want to write these addresses into the archive. >>>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>> >>>> + // Restore the vtable in case we invoke any virtual >>>> methods. >>>> + >>>> MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>>> Can this be restore_cpp_vtables since that's what it's >>>> doing. The first is after the dump and the second call is >>>> at UseSharedSpaces. A couple of comments in this >>>> clone_cpp_vtables --> restore_cpp_vtables would be >>>> nice. eg: >>>> >>>> I prefer to use the word clone. Otherwise when you just say >>>> "vtable" it's not clear whether you're talking about the >>>> original one (made by the c++ linker), or the cloned one in >>>> the CDS archive. >>>> >>>> + static intptr_t* clone_vtable(const char* name, >>>> intptr_t* p) { >>>> + T tmp; // Allocate temporary dummy metadata object to >>>> get vtable initialized >>>> + CppVtabInfo* info = (CppVtabInfo*)p; >>>> + int n = info->vtab_size(); >>>> + intptr_t* srcvtab = vtab_of(tmp); >>>> + intptr_t* dstvtab = info->vtab(); >>>> + >>>> + // We already checked (and, if necessary, adjusted n) >>>> when the vtables were allocated, so we are >>>> + // safe to do memcpy. >>>> + if (PrintSharedSpaces) { >>>> + tty->print_cr("%s copying %d vtable entries", name, n); >>>> + } >>>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>>> + return dstvtab + n; >>>> + } >>>> >>>> Done. I changed the wording >>>> >>>> T tmp; // Allocate temporary dummy metadata object to get >>>> to the original vtable. >>>> >>>> As we are not really "initializing a vtable" here. >>>> >>>> Same with 'patch'. It'd be so much faster and easier to >>>> read this code with more comments please. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>>> >>>> >>>> Why are these testers here? >>>> >>>> >>>> I updated the comment: >>>> >>>> // Used by CDS. These classes need to access the private >>>> ConstantPool() constructor. >>>> template friend class CppVtableTesterA; >>>> template friend class CppVtableTesterB; >>>> template friend class CppVtableCloner; >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>> >>>> >>>> Hi, >>>> >>>> This is the official review (follow up of the >>>> "Determining the size of C++ vtables" thread >>>> onhotspot-dev at openjdk.java.net >>>> ). >>>> >>>> The new code has the same assumption as the >>>> existing code in JDK 10: for a C++ object that >>>> contains virtual methods (e.g., ConstantPool), >>>> we assume the first intptr_t slot of the >>>> object is a _vptr, which points to a vtable, >>>> which consists of no more than 150 intptr_t's. >>>> >>>> ConstantPool*p -->[ _vptr ] -------> [ >>>> vtable slot 0 ] >>>> [ field #0 ] [ >>>> vtable slot 1 ] >>>> [ field #1 ] [ >>>> vtable slot 2 ] >>>> [ field #2 ] [ >>>> vtable slot 3 ] >>>> [ .... ] [ >>>> vtable slot 4] >>>> [ >>>> vtable slot 5 ] >>>> [ >>>> ... ] >>>> >>>> + In the existing code, we were pointing the >>>> vtable slots to >>>> code that's generated by HotSpot. >>>> >>>> + In the new code, we copy the vtable slots >>>> from an existing >>>> vtable (generated by the C++ linker). >>>> >>>> Per Thomas St?fe's advice, I don't try to >>>> determine the size of the vtable (as that >>>> would add one more compiler requirement where >>>> new virtual methods added by a subclass must >>>> be placed at a higher offset in the vtable). >>>> >>>> Instead, I have added code in non-product >>>> builds to ensure that the vtables are no >>>> longer than 150 entries. You can run with >>>> "-XX:+PrintSharedSpaces -Xshare:dump" to print >>>> out the actual size of the vtables for your >>>> particular platform: >>>> >>>> ConstantPool has 12 virtual methods >>>> InstanceKlass has 113 virtual methods >>>> InstanceClassLoaderKlass has 113 virtual >>>> methods >>>> InstanceMirrorKlass has 113 virtual methods >>>> InstanceRefKlass has 113 virtual methods >>>> Method has 12 virtual methods >>>> ObjArrayKlass has 114 virtual methods >>>> TypeArrayKlass has 114 virtual methods >>>> >>>> As mentioned in the code comments, if you have >>>> an esoteric C++ compiler, the >>>> verify_sufficient_size() function will >>>> probably fail, but hopefully that would give >>>> you some hints for porting this code. >>>> >>>> To avoid accidentally touching an unmapped >>>> page, the code uses SafeFetchN for copying the >>>> vtable contents, and would shrink the vtable >>>> to less than 150 entries if necessary. I can't >>>> test this for real, but I've added some code >>>> to simulate an error: >>>> >>>> for (int i=0; i>>> const intptr_t bad = >>>> intptr_t(0xdeadbeef); >>>> intptr_t num = >>>> SafeFetchN(&srcvtab[i], bad); >>>> if (num == bad >>>> // || i > 120 /* uncomment this line >>>> to test */ >>>> ) { >>>> _info->set_vtab_size(i-1); >>>> break; >>>> } >>>> dstvtab[i] = num; >>>> } >>>> >>>> Results: >>>> >>>> + Removed 850 lines of CPU-dependent code >>>> >>>> + CDS image is about 50K smaller >>>> >>>> + Previously Metadata objects must live in the >>>> read-write section in the CDS >>>> archive, because their _vptr was updated at >>>> run time. Now _vptr is no longer >>>> updated, so ConstantPool can be moved to the >>>> read-only section (see JDK-8171392). >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> >>>> >>>> >>>> > From jiangli.zhou at Oracle.COM Wed Mar 8 00:04:36 2017 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Tue, 7 Mar 2017 16:04:36 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BF24F0.6090704@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> <58BF24F0.6090704@oracle.com> Message-ID: Hi Ioi, Some minor comments/suggestions for the latest changes: You added MetaspaceShared::is_valid_method(). There is an existing Method::is_valid_method() function. It might cause confusion for anyone who?s not familiar with the CDS code when trying to determine which is_valid_method() to use in the future. How about renaming MetaspaceShared::is_valid_method() to is_valid_archived_method() and move it to method.*? Also, MetaspaceShared::is_valid_shared_object() might be more meaningful if it is rename as MetaspaceShared::is_valid_shared_metadata or MetaspaceShared::is_valid_archive_metadata. Thanks, Jiangli > On Mar 7, 2017, at 1:24 PM, Ioi Lam wrote: > > Hi Jiangli, > > Good catch! > > When I try to remove the code, I actually found my last version (v04) would not work with Method::is_valid_method(). The following assert would fail. This means that shared methods be skipped when we walk the stack (such as when doing a crash dump): > > void Method::restore_unshareable_info(TRAPS) { > - assert(is_method(), "ensure C++ vtable is restored"); > + assert(is_method() && is_valid_method(), "ensure C++ vtable is restored"); > > So I made a more comprehensive fix to make sure Method::is_valid_method works. Here's a diff from the previous version: > > http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v05/ > > Thanks > - Ioi > > > On 3/7/17 9:58 AM, Jiangli Zhou wrote: >> Hi Ioi, >> >> I think there is no other usage of Universe::init_self_patching_vtbl_list() after you removed the ones from metaspaceShared.cpp. You can remove the function from universe.*. >> >> Thanks, >> Jiangli >> >>> On Mar 6, 2017, at 5:25 PM, Ioi Lam wrote: >>> >>> Hi Thomas & Coleen, >>> >>> Thanks for your comments. I have updated the rev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ >>> >>> [1] Switched back to computing the exact vtable size >>> [2] Move non-trivial functions outside of their class declaration >>> >>> Thanks >>> - Ioi >>> >>> On 3/6/17 8:51 AM, Thomas St?fe wrote: >>>> Hi Coleen and Ioi, >>>> >>>> I had to port C++ code to platforms with terrible compilers for a time in my life, that is why I like code to be as portable as possible. That said, you are right in your argumentation, the SafeFetch solution is not terribly elegant and Ioi's original way of determining the vtable size is cleaner. >>>> >>>> I did some checks on some of our architectures with a test similar to Ioi's and on a first glance it seems to work for simple cases (single and public inheritance) on ppc (AIX) and Linux ia64. Although the vtables seemed to me to contain function descriptors, not real pointers to code, so this is something to keep in mind. But if the live vtable are copied, the function descriptors they contain should point to valid code too, so it should not matter. Just to remember to not expect every slot in the array to be a valid code pointer. >>>> >>>> So, in short, I remove my objection to Ioi's original solution, as far as that matters. >>>> >>>> I still think we rely on a lot here: Contiguous vtable containing absolute memory addresses, vtable pointer at start of object and vtable entries to be ordered from base->derived class. So I wonder how much effort it would be (now or in the future as a separate change) to have a fallback where - at loading time - instead of copying vtables the vtable pointers in the objects were fixed up to point to the new live vtables? I know this would be more expensive and potentially defeat the point of shared classes. But maybe not, it depends on how many objects are there, no? >>>> >>>> Kind Regards, Thomas >>>> >>>> On Sun, Mar 5, 2017 at 4:17 PM, > wrote: >>>> >>>> >>>> Ioi, Some comments inline (where no comments, insert "ok") :) >>>> >>>> >>>> On 3/2/17 10:37 PM, Ioi Lam wrote: >>>> >>>> Hi Coleen, >>>> >>>> Thanks for the comments. I have updated the webrev. See >>>> in-line for responses. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>>> >>>> >>>> >>>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >>>> wrote: >>>> >>>> >>>> Ioi >>>> I like the concept of this a lot but have some stylistic >>>> comments to help people reading this code later. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>>> >>>> >>>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >>>> save many characters, especially in CppVtableInfo/Testers >>>> >>>> Done. >>>> >>>> + // Start at slot 1, because slot 0 may be RTTI (on >>>> Solaris/Sparc) >>>> + int i; >>>> + for (i=1; ; i++) { >>>> Since you're using 'i' later, can you rename it to >>>> something descriptive. Or have another variable >>>> "vtable_length" to use later. This looks like an old >>>> style for loop. >>>> >>>> Done >>>> >>>> Can the functions for CppVtableInfo be declared outside of >>>> the class declaration? They don't need to be inline and >>>> then the debug code for testing the vtable size can be not >>>> in the middle of the class declaration. Then you can >>>> move the Tester classes to inside the same #ifndef PRODUCT >>>> block. >>>> >>>> Can you put #endif // PRODUCT when the ifdef covers >>>> several lines of code? >>>> >>>> Done >>>> >>>> vtab_of could be more descriptive, like cpp_vtable_for(). >>>> >>>> I changed to vtable_of(). Because the class name is already >>>> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >>>> >>>> Was PrintSharedSpaces was never converted to UL? >>>> >>>> Right. I've filed >>>> https://bugs.openjdk.java.net/browse/JDK-8176132 >>>> >>>> (-XX:+PrintSharedSpaces should be converted to use Unified >>>> Logging.) >>>> >>>> + int n = MAX_VTABLE_SIZE; >>>> >>>> Can you propagate MAX_VTABLE_SIZE to the places where it's >>>> used. n isn't descriptive. This starts out with >>>> max_vtable_size and then changes the size. Reusing 'n' >>>> makes this really hard to follow. Not having a comment >>>> that we only allocate enough slots for the vtable makes it >>>> hard too. >>>> >>>> + // allocate CppVtableInfo in the MD section + _info = >>>> (CppVtabInfo*)md_top; >>>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>>> + + // allocate temporary local instance of the metadata >>>> type T + T tmp; >>>> + intptr_t* srcvtab = vtab_of(tmp); >>>> + intptr_t* dstvtab = _info->vtab(); >>>> + >>>> >>>> Fixed. >>>> >>>> Something like that for comments. dstvtab is the >>>> destination_vtable in the MD section. >>>> >>>> >>>> I've dropped the md_ prefix from the functions that deal with >>>> the vtables, since they shouldn't care whether it's the "MD" >>>> section or not. Now it looks like this: >>>> >>>> // Allocate and initialize the C++ vtables, starting from top, >>>> but do not go past end. >>>> intptr_t* >>>> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>>> intptr_t* end) { >>>> assert(DumpSharedSpaces, "dump-time only"); >>>> // Layout (each slot is a intptr_t): >>>> // [number of slots in the first vtable = n1] >>>> // [ slots for the first vtable] >>>> // [number of slots in the first second = n2] >>>> // [ slots for the second vtable] >>>> // ... >>>> // The order of the vtables is the same as the >>>> CPP_VTAB_PATCH_TYPES_DO macro. >>>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>>> return top; >>>> } >>>> >>>> + for (int i=0; i>>> + const intptr_t bad = intptr_t(0xdeadbeef); >>>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>> + if (num == bad >>>> + // || i > 120 /* uncomment this line to test */ >>>> + ) { >>>> + _info->set_vtab_size(i-1); >>>> + break; >>>> + } >>>> + dstvtab[i] = num; >>>> + } >>>> I dont understand this code. You get deadbeef for a bad >>>> value if SafeFetchN gets a fault but why would it get a >>>> fault at the end of the metadata's vtable? Couldn't it >>>> just run onto the next vtable? I think your original way >>>> of counting vtable entries might be better (sorry I didn't >>>> have time to study that thread). >>>> >>>> I've modified the comments to this. Does it make sense to you? >>>> >>>> // It is not always safe to call memcpy(), because >>>> srcvtable might be shorter than >>>> // MAX_VTABLE_SIZE, and the C++ linker might have placed >>>> the vtable at the very >>>> // end of the last page of libjvm.so. Crossing over to the >>>> next page might >>>> // cause a page fault. >>>> >>>> My fear is the JVM would suddenly start crashing because the >>>> order of .o files have changed on the linker's command line, >>>> or if you enable some special linker optimization flags. It's >>>> better safe than sorry. >>>> >>>> >>>> This wasn't exactly what I was not understanding. I didn't see >>>> that you are copying 120 entries from the old vtable and junk >>>> memory beyond the old vtable, unless you get a segv, in which case >>>> you copy less. I don't think you should copy random memory into >>>> the vtable in the archive. This doesn't seem secure, even with >>>> the segv protection. >>>> >>>> Since we already have assumptions about C++ vtable layout in the >>>> code and it's mostly specified by various ABIs, and you have the >>>> assert code, I think I would prefer that you copy only the vtable >>>> entries into the archive. I guess Thomas Stuefe had a different >>>> opinion. I've read the original thread. Two points: >>>> >>>> If new C++ compiler implementations add a discontigous vtable, >>>> both the SafeFetchN and subclass additional virtual function at >>>> end implementation will fail. I don't think C++ implementations >>>> would do this and a contiguous vtable as first in the instance has >>>> been standard for years. If our metadata adds multiple >>>> inheritance, the same issue would be a problem for both >>>> implementations, as well as for the implementation we have before >>>> Ioi's fix. >>>> >>>> Ioi's subclass adding virtual function method would work for any >>>> esoteric C++ implementations in my memory, except the vptr for the >>>> old DECC++ compiler was after the nonstatic data members (which >>>> would fail with all of our implementations). >>>> >>>> Since the code is there anyway for debug purposes, we're not >>>> saving code by implementing SafeFetchN. The SafeFetchN >>>> implementation isn't obvious at all what it's doing, and requires >>>> better comments, especially if you don't know already what >>>> SafeFetchN does. It looks really cryptic. The poisoned values >>>> also bothered me in that they overload other poisoned values in >>>> other parts of the jvm. >>>> >>>> Ioi, could you make all methods of CppVtableCloner out of line? >>>> >>>> The other changes look good, although I might have more requests >>>> for comments. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> Would be nice to have comments here too!! >>>> >>>> + intptr_t* start = md_top; >>>> >>>> This doesn't do anything (?) >>>> >>>> >>>> Fixed. This was left over code. >>>> >>>> >>>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>> >>>> Why not zero the destination vtable in allocate? Or does >>>> patching the vtable pointers call virtual functions? You >>>> could prevent that so you don't need this code. >>>> >>>> I added this comment: >>>> >>>> // During patching, some virtual methods may be called, so >>>> at this point >>>> // the vtables must contain valid methods (as filled in by >>>> CppVtableCloner::allocate). >>>> MetaspaceShared::patch_cpp_vtable_pointers(); >>>> >>>> // The vtable clones contain addresses of the current process. >>>> // We don't want to write these addresses into the archive. >>>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>> >>>> + // Restore the vtable in case we invoke any virtual methods. >>>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>>> Can this be restore_cpp_vtables since that's what it's >>>> doing. The first is after the dump and the second call is >>>> at UseSharedSpaces. A couple of comments in this >>>> clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: >>>> >>>> I prefer to use the word clone. Otherwise when you just say >>>> "vtable" it's not clear whether you're talking about the >>>> original one (made by the c++ linker), or the cloned one in >>>> the CDS archive. >>>> >>>> + static intptr_t* clone_vtable(const char* name, >>>> intptr_t* p) { >>>> + T tmp; // Allocate temporary dummy metadata object to >>>> get vtable initialized >>>> + CppVtabInfo* info = (CppVtabInfo*)p; >>>> + int n = info->vtab_size(); >>>> + intptr_t* srcvtab = vtab_of(tmp); >>>> + intptr_t* dstvtab = info->vtab(); >>>> + >>>> + // We already checked (and, if necessary, adjusted n) >>>> when the vtables were allocated, so we are >>>> + // safe to do memcpy. >>>> + if (PrintSharedSpaces) { >>>> + tty->print_cr("%s copying %d vtable entries", name, n); >>>> + } >>>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>>> + return dstvtab + n; >>>> + } >>>> >>>> Done. I changed the wording >>>> >>>> T tmp; // Allocate temporary dummy metadata object to get >>>> to the original vtable. >>>> >>>> As we are not really "initializing a vtable" here. >>>> >>>> Same with 'patch'. It'd be so much faster and easier to >>>> read this code with more comments please. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>>> >>>> >>>> Why are these testers here? >>>> >>>> >>>> I updated the comment: >>>> >>>> // Used by CDS. These classes need to access the private >>>> ConstantPool() constructor. >>>> template friend class CppVtableTesterA; >>>> template friend class CppVtableTesterB; >>>> template friend class CppVtableCloner; >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>> >>>> >>>> Hi, >>>> >>>> This is the official review (follow up of the >>>> "Determining the size of C++ vtables" thread >>>> onhotspot-dev at openjdk.java.net >>>> ). >>>> >>>> The new code has the same assumption as the >>>> existing code in JDK 10: for a C++ object that >>>> contains virtual methods (e.g., ConstantPool), >>>> we assume the first intptr_t slot of the >>>> object is a _vptr, which points to a vtable, >>>> which consists of no more than 150 intptr_t's. >>>> >>>> ConstantPool*p -->[ _vptr ] -------> [ >>>> vtable slot 0 ] >>>> [ field #0 ] [ >>>> vtable slot 1 ] >>>> [ field #1 ] [ >>>> vtable slot 2 ] >>>> [ field #2 ] [ >>>> vtable slot 3 ] >>>> [ .... ] [ >>>> vtable slot 4] >>>> [ >>>> vtable slot 5 ] >>>> [ >>>> ... ] >>>> >>>> + In the existing code, we were pointing the >>>> vtable slots to >>>> code that's generated by HotSpot. >>>> >>>> + In the new code, we copy the vtable slots >>>> from an existing >>>> vtable (generated by the C++ linker). >>>> >>>> Per Thomas St?fe's advice, I don't try to >>>> determine the size of the vtable (as that >>>> would add one more compiler requirement where >>>> new virtual methods added by a subclass must >>>> be placed at a higher offset in the vtable). >>>> >>>> Instead, I have added code in non-product >>>> builds to ensure that the vtables are no >>>> longer than 150 entries. You can run with >>>> "-XX:+PrintSharedSpaces -Xshare:dump" to print >>>> out the actual size of the vtables for your >>>> particular platform: >>>> >>>> ConstantPool has 12 virtual methods >>>> InstanceKlass has 113 virtual methods >>>> InstanceClassLoaderKlass has 113 virtual methods >>>> InstanceMirrorKlass has 113 virtual methods >>>> InstanceRefKlass has 113 virtual methods >>>> Method has 12 virtual methods >>>> ObjArrayKlass has 114 virtual methods >>>> TypeArrayKlass has 114 virtual methods >>>> >>>> As mentioned in the code comments, if you have >>>> an esoteric C++ compiler, the >>>> verify_sufficient_size() function will >>>> probably fail, but hopefully that would give >>>> you some hints for porting this code. >>>> >>>> To avoid accidentally touching an unmapped >>>> page, the code uses SafeFetchN for copying the >>>> vtable contents, and would shrink the vtable >>>> to less than 150 entries if necessary. I can't >>>> test this for real, but I've added some code >>>> to simulate an error: >>>> >>>> for (int i=0; i>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>> if (num == bad >>>> // || i > 120 /* uncomment this line >>>> to test */ >>>> ) { >>>> _info->set_vtab_size(i-1); >>>> break; >>>> } >>>> dstvtab[i] = num; >>>> } >>>> >>>> Results: >>>> >>>> + Removed 850 lines of CPU-dependent code >>>> >>>> + CDS image is about 50K smaller >>>> >>>> + Previously Metadata objects must live in the >>>> read-write section in the CDS >>>> archive, because their _vptr was updated at >>>> run time. Now _vptr is no longer >>>> updated, so ConstantPool can be moved to the >>>> read-only section (see JDK-8171392). >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> >>>> >>>> >>>> > From ioi.lam at oracle.com Wed Mar 8 01:50:35 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 07 Mar 2017 17:50:35 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> <58BF24F0.6090704@oracle.com> Message-ID: <58BF636B.7050609@oracle.com> On 3/7/17 4:04 PM, Jiangli Zhou wrote: > Hi Ioi, > > Some minor comments/suggestions for the latest changes: > > You added MetaspaceShared::is_valid_method(). There is an existing Method::is_valid_method() function. It might cause confusion for anyone who?s not familiar with the CDS code when trying to determine which is_valid_method() to use in the future. How about renaming MetaspaceShared::is_valid_method() to is_valid_archived_method() and move it to method.*? I renamed it to MetaspaceShared::is_valid_shared_method(), to be consistent with existing names such as MetaspaceShared::is_in_shared_space(). I don't want to move it to method.cpp because the logic of determining whether the Method's vptr points to the cloned vtable is inside metaspaceShared.cpp > Also, MetaspaceShared::is_valid_shared_object() might be more meaningful if it is rename as MetaspaceShared::is_valid_shared_metadata or MetaspaceShared::is_valid_archive_metadata. There's no MetaspaceShared::is_valid_shared_object(). I think you mean CppVtableCloner::is_valid_shared_object. The meaning of the name is pretty clear: is it a valid shared object of type Thanks - Ioi > > Thanks, > Jiangli > >> On Mar 7, 2017, at 1:24 PM, Ioi Lam wrote: >> >> Hi Jiangli, >> >> Good catch! >> >> When I try to remove the code, I actually found my last version (v04) would not work with Method::is_valid_method(). The following assert would fail. This means that shared methods be skipped when we walk the stack (such as when doing a crash dump): >> >> void Method::restore_unshareable_info(TRAPS) { >> - assert(is_method(), "ensure C++ vtable is restored"); >> + assert(is_method() && is_valid_method(), "ensure C++ vtable is restored"); >> >> So I made a more comprehensive fix to make sure Method::is_valid_method works. Here's a diff from the previous version: >> >> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v05/ >> >> Thanks >> - Ioi >> >> >> On 3/7/17 9:58 AM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> I think there is no other usage of Universe::init_self_patching_vtbl_list() after you removed the ones from metaspaceShared.cpp. You can remove the function from universe.*. >>> >>> Thanks, >>> Jiangli >>> >>>> On Mar 6, 2017, at 5:25 PM, Ioi Lam wrote: >>>> >>>> Hi Thomas & Coleen, >>>> >>>> Thanks for your comments. I have updated the rev: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ >>>> >>>> [1] Switched back to computing the exact vtable size >>>> [2] Move non-trivial functions outside of their class declaration >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 3/6/17 8:51 AM, Thomas St?fe wrote: >>>>> Hi Coleen and Ioi, >>>>> >>>>> I had to port C++ code to platforms with terrible compilers for a time in my life, that is why I like code to be as portable as possible. That said, you are right in your argumentation, the SafeFetch solution is not terribly elegant and Ioi's original way of determining the vtable size is cleaner. >>>>> >>>>> I did some checks on some of our architectures with a test similar to Ioi's and on a first glance it seems to work for simple cases (single and public inheritance) on ppc (AIX) and Linux ia64. Although the vtables seemed to me to contain function descriptors, not real pointers to code, so this is something to keep in mind. But if the live vtable are copied, the function descriptors they contain should point to valid code too, so it should not matter. Just to remember to not expect every slot in the array to be a valid code pointer. >>>>> >>>>> So, in short, I remove my objection to Ioi's original solution, as far as that matters. >>>>> >>>>> I still think we rely on a lot here: Contiguous vtable containing absolute memory addresses, vtable pointer at start of object and vtable entries to be ordered from base->derived class. So I wonder how much effort it would be (now or in the future as a separate change) to have a fallback where - at loading time - instead of copying vtables the vtable pointers in the objects were fixed up to point to the new live vtables? I know this would be more expensive and potentially defeat the point of shared classes. But maybe not, it depends on how many objects are there, no? >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> On Sun, Mar 5, 2017 at 4:17 PM, > wrote: >>>>> >>>>> >>>>> Ioi, Some comments inline (where no comments, insert "ok") :) >>>>> >>>>> >>>>> On 3/2/17 10:37 PM, Ioi Lam wrote: >>>>> >>>>> Hi Coleen, >>>>> >>>>> Thanks for the comments. I have updated the webrev. See >>>>> in-line for responses. >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>>>> >>>>> >>>>> >>>>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >>>>> wrote: >>>>> >>>>> >>>>> Ioi >>>>> I like the concept of this a lot but have some stylistic >>>>> comments to help people reading this code later. >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>>>> >>>>> >>>>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >>>>> save many characters, especially in CppVtableInfo/Testers >>>>> >>>>> Done. >>>>> >>>>> + // Start at slot 1, because slot 0 may be RTTI (on >>>>> Solaris/Sparc) >>>>> + int i; >>>>> + for (i=1; ; i++) { >>>>> Since you're using 'i' later, can you rename it to >>>>> something descriptive. Or have another variable >>>>> "vtable_length" to use later. This looks like an old >>>>> style for loop. >>>>> >>>>> Done >>>>> >>>>> Can the functions for CppVtableInfo be declared outside of >>>>> the class declaration? They don't need to be inline and >>>>> then the debug code for testing the vtable size can be not >>>>> in the middle of the class declaration. Then you can >>>>> move the Tester classes to inside the same #ifndef PRODUCT >>>>> block. >>>>> >>>>> Can you put #endif // PRODUCT when the ifdef covers >>>>> several lines of code? >>>>> >>>>> Done >>>>> >>>>> vtab_of could be more descriptive, like cpp_vtable_for(). >>>>> >>>>> I changed to vtable_of(). Because the class name is already >>>>> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >>>>> >>>>> Was PrintSharedSpaces was never converted to UL? >>>>> >>>>> Right. I've filed >>>>> https://bugs.openjdk.java.net/browse/JDK-8176132 >>>>> >>>>> (-XX:+PrintSharedSpaces should be converted to use Unified >>>>> Logging.) >>>>> >>>>> + int n = MAX_VTABLE_SIZE; >>>>> >>>>> Can you propagate MAX_VTABLE_SIZE to the places where it's >>>>> used. n isn't descriptive. This starts out with >>>>> max_vtable_size and then changes the size. Reusing 'n' >>>>> makes this really hard to follow. Not having a comment >>>>> that we only allocate enough slots for the vtable makes it >>>>> hard too. >>>>> >>>>> + // allocate CppVtableInfo in the MD section + _info = >>>>> (CppVtabInfo*)md_top; >>>>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>>>> + + // allocate temporary local instance of the metadata >>>>> type T + T tmp; >>>>> + intptr_t* srcvtab = vtab_of(tmp); >>>>> + intptr_t* dstvtab = _info->vtab(); >>>>> + >>>>> >>>>> Fixed. >>>>> >>>>> Something like that for comments. dstvtab is the >>>>> destination_vtable in the MD section. >>>>> >>>>> >>>>> I've dropped the md_ prefix from the functions that deal with >>>>> the vtables, since they shouldn't care whether it's the "MD" >>>>> section or not. Now it looks like this: >>>>> >>>>> // Allocate and initialize the C++ vtables, starting from top, >>>>> but do not go past end. >>>>> intptr_t* >>>>> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>>>> intptr_t* end) { >>>>> assert(DumpSharedSpaces, "dump-time only"); >>>>> // Layout (each slot is a intptr_t): >>>>> // [number of slots in the first vtable = n1] >>>>> // [ slots for the first vtable] >>>>> // [number of slots in the first second = n2] >>>>> // [ slots for the second vtable] >>>>> // ... >>>>> // The order of the vtables is the same as the >>>>> CPP_VTAB_PATCH_TYPES_DO macro. >>>>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>>>> return top; >>>>> } >>>>> >>>>> + for (int i=0; i>>>> + const intptr_t bad = intptr_t(0xdeadbeef); >>>>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>> + if (num == bad >>>>> + // || i > 120 /* uncomment this line to test */ >>>>> + ) { >>>>> + _info->set_vtab_size(i-1); >>>>> + break; >>>>> + } >>>>> + dstvtab[i] = num; >>>>> + } >>>>> I dont understand this code. You get deadbeef for a bad >>>>> value if SafeFetchN gets a fault but why would it get a >>>>> fault at the end of the metadata's vtable? Couldn't it >>>>> just run onto the next vtable? I think your original way >>>>> of counting vtable entries might be better (sorry I didn't >>>>> have time to study that thread). >>>>> >>>>> I've modified the comments to this. Does it make sense to you? >>>>> >>>>> // It is not always safe to call memcpy(), because >>>>> srcvtable might be shorter than >>>>> // MAX_VTABLE_SIZE, and the C++ linker might have placed >>>>> the vtable at the very >>>>> // end of the last page of libjvm.so. Crossing over to the >>>>> next page might >>>>> // cause a page fault. >>>>> >>>>> My fear is the JVM would suddenly start crashing because the >>>>> order of .o files have changed on the linker's command line, >>>>> or if you enable some special linker optimization flags. It's >>>>> better safe than sorry. >>>>> >>>>> >>>>> This wasn't exactly what I was not understanding. I didn't see >>>>> that you are copying 120 entries from the old vtable and junk >>>>> memory beyond the old vtable, unless you get a segv, in which case >>>>> you copy less. I don't think you should copy random memory into >>>>> the vtable in the archive. This doesn't seem secure, even with >>>>> the segv protection. >>>>> >>>>> Since we already have assumptions about C++ vtable layout in the >>>>> code and it's mostly specified by various ABIs, and you have the >>>>> assert code, I think I would prefer that you copy only the vtable >>>>> entries into the archive. I guess Thomas Stuefe had a different >>>>> opinion. I've read the original thread. Two points: >>>>> >>>>> If new C++ compiler implementations add a discontigous vtable, >>>>> both the SafeFetchN and subclass additional virtual function at >>>>> end implementation will fail. I don't think C++ implementations >>>>> would do this and a contiguous vtable as first in the instance has >>>>> been standard for years. If our metadata adds multiple >>>>> inheritance, the same issue would be a problem for both >>>>> implementations, as well as for the implementation we have before >>>>> Ioi's fix. >>>>> >>>>> Ioi's subclass adding virtual function method would work for any >>>>> esoteric C++ implementations in my memory, except the vptr for the >>>>> old DECC++ compiler was after the nonstatic data members (which >>>>> would fail with all of our implementations). >>>>> >>>>> Since the code is there anyway for debug purposes, we're not >>>>> saving code by implementing SafeFetchN. The SafeFetchN >>>>> implementation isn't obvious at all what it's doing, and requires >>>>> better comments, especially if you don't know already what >>>>> SafeFetchN does. It looks really cryptic. The poisoned values >>>>> also bothered me in that they overload other poisoned values in >>>>> other parts of the jvm. >>>>> >>>>> Ioi, could you make all methods of CppVtableCloner out of line? >>>>> >>>>> The other changes look good, although I might have more requests >>>>> for comments. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> Would be nice to have comments here too!! >>>>> >>>>> + intptr_t* start = md_top; >>>>> >>>>> This doesn't do anything (?) >>>>> >>>>> >>>>> Fixed. This was left over code. >>>>> >>>>> >>>>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>>> >>>>> Why not zero the destination vtable in allocate? Or does >>>>> patching the vtable pointers call virtual functions? You >>>>> could prevent that so you don't need this code. >>>>> >>>>> I added this comment: >>>>> >>>>> // During patching, some virtual methods may be called, so >>>>> at this point >>>>> // the vtables must contain valid methods (as filled in by >>>>> CppVtableCloner::allocate). >>>>> MetaspaceShared::patch_cpp_vtable_pointers(); >>>>> >>>>> // The vtable clones contain addresses of the current process. >>>>> // We don't want to write these addresses into the archive. >>>>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>>> >>>>> + // Restore the vtable in case we invoke any virtual methods. >>>>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>>>> Can this be restore_cpp_vtables since that's what it's >>>>> doing. The first is after the dump and the second call is >>>>> at UseSharedSpaces. A couple of comments in this >>>>> clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: >>>>> >>>>> I prefer to use the word clone. Otherwise when you just say >>>>> "vtable" it's not clear whether you're talking about the >>>>> original one (made by the c++ linker), or the cloned one in >>>>> the CDS archive. >>>>> >>>>> + static intptr_t* clone_vtable(const char* name, >>>>> intptr_t* p) { >>>>> + T tmp; // Allocate temporary dummy metadata object to >>>>> get vtable initialized >>>>> + CppVtabInfo* info = (CppVtabInfo*)p; >>>>> + int n = info->vtab_size(); >>>>> + intptr_t* srcvtab = vtab_of(tmp); >>>>> + intptr_t* dstvtab = info->vtab(); >>>>> + >>>>> + // We already checked (and, if necessary, adjusted n) >>>>> when the vtables were allocated, so we are >>>>> + // safe to do memcpy. >>>>> + if (PrintSharedSpaces) { >>>>> + tty->print_cr("%s copying %d vtable entries", name, n); >>>>> + } >>>>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>>>> + return dstvtab + n; >>>>> + } >>>>> >>>>> Done. I changed the wording >>>>> >>>>> T tmp; // Allocate temporary dummy metadata object to get >>>>> to the original vtable. >>>>> >>>>> As we are not really "initializing a vtable" here. >>>>> >>>>> Same with 'patch'. It'd be so much faster and easier to >>>>> read this code with more comments please. >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>>>> >>>>> >>>>> Why are these testers here? >>>>> >>>>> >>>>> I updated the comment: >>>>> >>>>> // Used by CDS. These classes need to access the private >>>>> ConstantPool() constructor. >>>>> template friend class CppVtableTesterA; >>>>> template friend class CppVtableTesterB; >>>>> template friend class CppVtableCloner; >>>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>>> >>>>> >>>>> Hi, >>>>> >>>>> This is the official review (follow up of the >>>>> "Determining the size of C++ vtables" thread >>>>> onhotspot-dev at openjdk.java.net >>>>> ). >>>>> >>>>> The new code has the same assumption as the >>>>> existing code in JDK 10: for a C++ object that >>>>> contains virtual methods (e.g., ConstantPool), >>>>> we assume the first intptr_t slot of the >>>>> object is a _vptr, which points to a vtable, >>>>> which consists of no more than 150 intptr_t's. >>>>> >>>>> ConstantPool*p -->[ _vptr ] -------> [ >>>>> vtable slot 0 ] >>>>> [ field #0 ] [ >>>>> vtable slot 1 ] >>>>> [ field #1 ] [ >>>>> vtable slot 2 ] >>>>> [ field #2 ] [ >>>>> vtable slot 3 ] >>>>> [ .... ] [ >>>>> vtable slot 4] >>>>> [ >>>>> vtable slot 5 ] >>>>> [ >>>>> ... ] >>>>> >>>>> + In the existing code, we were pointing the >>>>> vtable slots to >>>>> code that's generated by HotSpot. >>>>> >>>>> + In the new code, we copy the vtable slots >>>>> from an existing >>>>> vtable (generated by the C++ linker). >>>>> >>>>> Per Thomas St?fe's advice, I don't try to >>>>> determine the size of the vtable (as that >>>>> would add one more compiler requirement where >>>>> new virtual methods added by a subclass must >>>>> be placed at a higher offset in the vtable). >>>>> >>>>> Instead, I have added code in non-product >>>>> builds to ensure that the vtables are no >>>>> longer than 150 entries. You can run with >>>>> "-XX:+PrintSharedSpaces -Xshare:dump" to print >>>>> out the actual size of the vtables for your >>>>> particular platform: >>>>> >>>>> ConstantPool has 12 virtual methods >>>>> InstanceKlass has 113 virtual methods >>>>> InstanceClassLoaderKlass has 113 virtual methods >>>>> InstanceMirrorKlass has 113 virtual methods >>>>> InstanceRefKlass has 113 virtual methods >>>>> Method has 12 virtual methods >>>>> ObjArrayKlass has 114 virtual methods >>>>> TypeArrayKlass has 114 virtual methods >>>>> >>>>> As mentioned in the code comments, if you have >>>>> an esoteric C++ compiler, the >>>>> verify_sufficient_size() function will >>>>> probably fail, but hopefully that would give >>>>> you some hints for porting this code. >>>>> >>>>> To avoid accidentally touching an unmapped >>>>> page, the code uses SafeFetchN for copying the >>>>> vtable contents, and would shrink the vtable >>>>> to less than 150 entries if necessary. I can't >>>>> test this for real, but I've added some code >>>>> to simulate an error: >>>>> >>>>> for (int i=0; i>>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>> if (num == bad >>>>> // || i > 120 /* uncomment this line >>>>> to test */ >>>>> ) { >>>>> _info->set_vtab_size(i-1); >>>>> break; >>>>> } >>>>> dstvtab[i] = num; >>>>> } >>>>> >>>>> Results: >>>>> >>>>> + Removed 850 lines of CPU-dependent code >>>>> >>>>> + CDS image is about 50K smaller >>>>> >>>>> + Previously Metadata objects must live in the >>>>> read-write section in the CDS >>>>> archive, because their _vptr was updated at >>>>> run time. Now _vptr is no longer >>>>> updated, so ConstantPool can be moved to the >>>>> read-only section (see JDK-8171392). >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> From jiangli.zhou at oracle.com Wed Mar 8 04:12:45 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 7 Mar 2017 20:12:45 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BF636B.7050609@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> <58BF24F0.6090704@oracle.com> <58BF636B.7050609@oracle.com> Message-ID: <3B78A8C3-216D-43F2-A777-A6DF85828651@oracle.com> Hi Ioi, > On Mar 7, 2017, at 5:50 PM, Ioi Lam wrote: > > > > On 3/7/17 4:04 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> Some minor comments/suggestions for the latest changes: >> >> You added MetaspaceShared::is_valid_method(). There is an existing Method::is_valid_method() function. It might cause confusion for anyone who?s not familiar with the CDS code when trying to determine which is_valid_method() to use in the future. How about renaming MetaspaceShared::is_valid_method() to is_valid_archived_method() and move it to method.*? > > I renamed it to MetaspaceShared::is_valid_shared_method(), to be consistent with existing names such as MetaspaceShared::is_in_shared_space(). > > I don't want to move it to method.cpp because the logic of determining whether the Method's vptr points to the cloned vtable is inside metaspaceShared.cpp Ok. > >> Also, MetaspaceShared::is_valid_shared_object() might be more meaningful if it is rename as MetaspaceShared::is_valid_shared_metadata or MetaspaceShared::is_valid_archive_metadata. > There's no MetaspaceShared::is_valid_shared_object(). I think you mean CppVtableCloner::is_valid_shared_object. > The meaning of the name is pretty clear: is it a valid shared object of type Using ?shared_object? might be a bit too broad here. We have both shared metadata and shared String objects. This API is used to determine if it is a valid data of the Metadata types that have C++ vtable. Also, I just noticed the implementation doesn?t do any specific ?is_shared? check. How about changing it to CppVtableCloner::is_valid()? Thanks, Jiangli > > Thanks > - Ioi >> >> Thanks, >> Jiangli >> >>> On Mar 7, 2017, at 1:24 PM, Ioi Lam wrote: >>> >>> Hi Jiangli, >>> >>> Good catch! >>> >>> When I try to remove the code, I actually found my last version (v04) would not work with Method::is_valid_method(). The following assert would fail. This means that shared methods be skipped when we walk the stack (such as when doing a crash dump): >>> >>> void Method::restore_unshareable_info(TRAPS) { >>> - assert(is_method(), "ensure C++ vtable is restored"); >>> + assert(is_method() && is_valid_method(), "ensure C++ vtable is restored"); >>> >>> So I made a more comprehensive fix to make sure Method::is_valid_method works. Here's a diff from the previous version: >>> >>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v05/ >>> >>> Thanks >>> - Ioi >>> >>> >>> On 3/7/17 9:58 AM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> I think there is no other usage of Universe::init_self_patching_vtbl_list() after you removed the ones from metaspaceShared.cpp. You can remove the function from universe.*. >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On Mar 6, 2017, at 5:25 PM, Ioi Lam wrote: >>>>> >>>>> Hi Thomas & Coleen, >>>>> >>>>> Thanks for your comments. I have updated the rev: >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ >>>>> >>>>> [1] Switched back to computing the exact vtable size >>>>> [2] Move non-trivial functions outside of their class declaration >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 3/6/17 8:51 AM, Thomas St?fe wrote: >>>>>> Hi Coleen and Ioi, >>>>>> >>>>>> I had to port C++ code to platforms with terrible compilers for a time in my life, that is why I like code to be as portable as possible. That said, you are right in your argumentation, the SafeFetch solution is not terribly elegant and Ioi's original way of determining the vtable size is cleaner. >>>>>> >>>>>> I did some checks on some of our architectures with a test similar to Ioi's and on a first glance it seems to work for simple cases (single and public inheritance) on ppc (AIX) and Linux ia64. Although the vtables seemed to me to contain function descriptors, not real pointers to code, so this is something to keep in mind. But if the live vtable are copied, the function descriptors they contain should point to valid code too, so it should not matter. Just to remember to not expect every slot in the array to be a valid code pointer. >>>>>> >>>>>> So, in short, I remove my objection to Ioi's original solution, as far as that matters. >>>>>> >>>>>> I still think we rely on a lot here: Contiguous vtable containing absolute memory addresses, vtable pointer at start of object and vtable entries to be ordered from base->derived class. So I wonder how much effort it would be (now or in the future as a separate change) to have a fallback where - at loading time - instead of copying vtables the vtable pointers in the objects were fixed up to point to the new live vtables? I know this would be more expensive and potentially defeat the point of shared classes. But maybe not, it depends on how many objects are there, no? >>>>>> >>>>>> Kind Regards, Thomas >>>>>> >>>>>> On Sun, Mar 5, 2017 at 4:17 PM, > wrote: >>>>>> >>>>>> >>>>>> Ioi, Some comments inline (where no comments, insert "ok") :) >>>>>> >>>>>> >>>>>> On 3/2/17 10:37 PM, Ioi Lam wrote: >>>>>> >>>>>> Hi Coleen, >>>>>> >>>>>> Thanks for the comments. I have updated the webrev. See >>>>>> in-line for responses. >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>>>>> >>>>>> >>>>>> >>>>>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >>>>>> wrote: >>>>>> >>>>>> >>>>>> Ioi >>>>>> I like the concept of this a lot but have some stylistic >>>>>> comments to help people reading this code later. >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>>>>> >>>>>> >>>>>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >>>>>> save many characters, especially in CppVtableInfo/Testers >>>>>> >>>>>> Done. >>>>>> >>>>>> + // Start at slot 1, because slot 0 may be RTTI (on >>>>>> Solaris/Sparc) >>>>>> + int i; >>>>>> + for (i=1; ; i++) { >>>>>> Since you're using 'i' later, can you rename it to >>>>>> something descriptive. Or have another variable >>>>>> "vtable_length" to use later. This looks like an old >>>>>> style for loop. >>>>>> >>>>>> Done >>>>>> >>>>>> Can the functions for CppVtableInfo be declared outside of >>>>>> the class declaration? They don't need to be inline and >>>>>> then the debug code for testing the vtable size can be not >>>>>> in the middle of the class declaration. Then you can >>>>>> move the Tester classes to inside the same #ifndef PRODUCT >>>>>> block. >>>>>> >>>>>> Can you put #endif // PRODUCT when the ifdef covers >>>>>> several lines of code? >>>>>> >>>>>> Done >>>>>> >>>>>> vtab_of could be more descriptive, like cpp_vtable_for(). >>>>>> >>>>>> I changed to vtable_of(). Because the class name is already >>>>>> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >>>>>> >>>>>> Was PrintSharedSpaces was never converted to UL? >>>>>> >>>>>> Right. I've filed >>>>>> https://bugs.openjdk.java.net/browse/JDK-8176132 >>>>>> >>>>>> (-XX:+PrintSharedSpaces should be converted to use Unified >>>>>> Logging.) >>>>>> >>>>>> + int n = MAX_VTABLE_SIZE; >>>>>> >>>>>> Can you propagate MAX_VTABLE_SIZE to the places where it's >>>>>> used. n isn't descriptive. This starts out with >>>>>> max_vtable_size and then changes the size. Reusing 'n' >>>>>> makes this really hard to follow. Not having a comment >>>>>> that we only allocate enough slots for the vtable makes it >>>>>> hard too. >>>>>> >>>>>> + // allocate CppVtableInfo in the MD section + _info = >>>>>> (CppVtabInfo*)md_top; >>>>>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>>>>> + + // allocate temporary local instance of the metadata >>>>>> type T + T tmp; >>>>>> + intptr_t* srcvtab = vtab_of(tmp); >>>>>> + intptr_t* dstvtab = _info->vtab(); >>>>>> + >>>>>> >>>>>> Fixed. >>>>>> >>>>>> Something like that for comments. dstvtab is the >>>>>> destination_vtable in the MD section. >>>>>> >>>>>> >>>>>> I've dropped the md_ prefix from the functions that deal with >>>>>> the vtables, since they shouldn't care whether it's the "MD" >>>>>> section or not. Now it looks like this: >>>>>> >>>>>> // Allocate and initialize the C++ vtables, starting from top, >>>>>> but do not go past end. >>>>>> intptr_t* >>>>>> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>>>>> intptr_t* end) { >>>>>> assert(DumpSharedSpaces, "dump-time only"); >>>>>> // Layout (each slot is a intptr_t): >>>>>> // [number of slots in the first vtable = n1] >>>>>> // [ slots for the first vtable] >>>>>> // [number of slots in the first second = n2] >>>>>> // [ slots for the second vtable] >>>>>> // ... >>>>>> // The order of the vtables is the same as the >>>>>> CPP_VTAB_PATCH_TYPES_DO macro. >>>>>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>>>>> return top; >>>>>> } >>>>>> >>>>>> + for (int i=0; i>>>>> + const intptr_t bad = intptr_t(0xdeadbeef); >>>>>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>> + if (num == bad >>>>>> + // || i > 120 /* uncomment this line to test */ >>>>>> + ) { >>>>>> + _info->set_vtab_size(i-1); >>>>>> + break; >>>>>> + } >>>>>> + dstvtab[i] = num; >>>>>> + } >>>>>> I dont understand this code. You get deadbeef for a bad >>>>>> value if SafeFetchN gets a fault but why would it get a >>>>>> fault at the end of the metadata's vtable? Couldn't it >>>>>> just run onto the next vtable? I think your original way >>>>>> of counting vtable entries might be better (sorry I didn't >>>>>> have time to study that thread). >>>>>> >>>>>> I've modified the comments to this. Does it make sense to you? >>>>>> >>>>>> // It is not always safe to call memcpy(), because >>>>>> srcvtable might be shorter than >>>>>> // MAX_VTABLE_SIZE, and the C++ linker might have placed >>>>>> the vtable at the very >>>>>> // end of the last page of libjvm.so. Crossing over to the >>>>>> next page might >>>>>> // cause a page fault. >>>>>> >>>>>> My fear is the JVM would suddenly start crashing because the >>>>>> order of .o files have changed on the linker's command line, >>>>>> or if you enable some special linker optimization flags. It's >>>>>> better safe than sorry. >>>>>> >>>>>> >>>>>> This wasn't exactly what I was not understanding. I didn't see >>>>>> that you are copying 120 entries from the old vtable and junk >>>>>> memory beyond the old vtable, unless you get a segv, in which case >>>>>> you copy less. I don't think you should copy random memory into >>>>>> the vtable in the archive. This doesn't seem secure, even with >>>>>> the segv protection. >>>>>> >>>>>> Since we already have assumptions about C++ vtable layout in the >>>>>> code and it's mostly specified by various ABIs, and you have the >>>>>> assert code, I think I would prefer that you copy only the vtable >>>>>> entries into the archive. I guess Thomas Stuefe had a different >>>>>> opinion. I've read the original thread. Two points: >>>>>> >>>>>> If new C++ compiler implementations add a discontigous vtable, >>>>>> both the SafeFetchN and subclass additional virtual function at >>>>>> end implementation will fail. I don't think C++ implementations >>>>>> would do this and a contiguous vtable as first in the instance has >>>>>> been standard for years. If our metadata adds multiple >>>>>> inheritance, the same issue would be a problem for both >>>>>> implementations, as well as for the implementation we have before >>>>>> Ioi's fix. >>>>>> >>>>>> Ioi's subclass adding virtual function method would work for any >>>>>> esoteric C++ implementations in my memory, except the vptr for the >>>>>> old DECC++ compiler was after the nonstatic data members (which >>>>>> would fail with all of our implementations). >>>>>> >>>>>> Since the code is there anyway for debug purposes, we're not >>>>>> saving code by implementing SafeFetchN. The SafeFetchN >>>>>> implementation isn't obvious at all what it's doing, and requires >>>>>> better comments, especially if you don't know already what >>>>>> SafeFetchN does. It looks really cryptic. The poisoned values >>>>>> also bothered me in that they overload other poisoned values in >>>>>> other parts of the jvm. >>>>>> >>>>>> Ioi, could you make all methods of CppVtableCloner out of line? >>>>>> >>>>>> The other changes look good, although I might have more requests >>>>>> for comments. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>>> Would be nice to have comments here too!! >>>>>> >>>>>> + intptr_t* start = md_top; >>>>>> >>>>>> This doesn't do anything (?) >>>>>> >>>>>> >>>>>> Fixed. This was left over code. >>>>>> >>>>>> >>>>>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>>>> >>>>>> Why not zero the destination vtable in allocate? Or does >>>>>> patching the vtable pointers call virtual functions? You >>>>>> could prevent that so you don't need this code. >>>>>> >>>>>> I added this comment: >>>>>> >>>>>> // During patching, some virtual methods may be called, so >>>>>> at this point >>>>>> // the vtables must contain valid methods (as filled in by >>>>>> CppVtableCloner::allocate). >>>>>> MetaspaceShared::patch_cpp_vtable_pointers(); >>>>>> >>>>>> // The vtable clones contain addresses of the current process. >>>>>> // We don't want to write these addresses into the archive. >>>>>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>>>> >>>>>> + // Restore the vtable in case we invoke any virtual methods. >>>>>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>>>>> Can this be restore_cpp_vtables since that's what it's >>>>>> doing. The first is after the dump and the second call is >>>>>> at UseSharedSpaces. A couple of comments in this >>>>>> clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: >>>>>> >>>>>> I prefer to use the word clone. Otherwise when you just say >>>>>> "vtable" it's not clear whether you're talking about the >>>>>> original one (made by the c++ linker), or the cloned one in >>>>>> the CDS archive. >>>>>> >>>>>> + static intptr_t* clone_vtable(const char* name, >>>>>> intptr_t* p) { >>>>>> + T tmp; // Allocate temporary dummy metadata object to >>>>>> get vtable initialized >>>>>> + CppVtabInfo* info = (CppVtabInfo*)p; >>>>>> + int n = info->vtab_size(); >>>>>> + intptr_t* srcvtab = vtab_of(tmp); >>>>>> + intptr_t* dstvtab = info->vtab(); >>>>>> + >>>>>> + // We already checked (and, if necessary, adjusted n) >>>>>> when the vtables were allocated, so we are >>>>>> + // safe to do memcpy. >>>>>> + if (PrintSharedSpaces) { >>>>>> + tty->print_cr("%s copying %d vtable entries", name, n); >>>>>> + } >>>>>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>>>>> + return dstvtab + n; >>>>>> + } >>>>>> >>>>>> Done. I changed the wording >>>>>> >>>>>> T tmp; // Allocate temporary dummy metadata object to get >>>>>> to the original vtable. >>>>>> >>>>>> As we are not really "initializing a vtable" here. >>>>>> >>>>>> Same with 'patch'. It'd be so much faster and easier to >>>>>> read this code with more comments please. >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>>>>> >>>>>> >>>>>> Why are these testers here? >>>>>> >>>>>> >>>>>> I updated the comment: >>>>>> >>>>>> // Used by CDS. These classes need to access the private >>>>>> ConstantPool() constructor. >>>>>> template friend class CppVtableTesterA; >>>>>> template friend class CppVtableTesterB; >>>>>> template friend class CppVtableCloner; >>>>>> >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> This is the official review (follow up of the >>>>>> "Determining the size of C++ vtables" thread >>>>>> onhotspot-dev at openjdk.java.net >>>>>> ). >>>>>> >>>>>> The new code has the same assumption as the >>>>>> existing code in JDK 10: for a C++ object that >>>>>> contains virtual methods (e.g., ConstantPool), >>>>>> we assume the first intptr_t slot of the >>>>>> object is a _vptr, which points to a vtable, >>>>>> which consists of no more than 150 intptr_t's. >>>>>> >>>>>> ConstantPool*p -->[ _vptr ] -------> [ >>>>>> vtable slot 0 ] >>>>>> [ field #0 ] [ >>>>>> vtable slot 1 ] >>>>>> [ field #1 ] [ >>>>>> vtable slot 2 ] >>>>>> [ field #2 ] [ >>>>>> vtable slot 3 ] >>>>>> [ .... ] [ >>>>>> vtable slot 4] >>>>>> [ >>>>>> vtable slot 5 ] >>>>>> [ >>>>>> ... ] >>>>>> >>>>>> + In the existing code, we were pointing the >>>>>> vtable slots to >>>>>> code that's generated by HotSpot. >>>>>> >>>>>> + In the new code, we copy the vtable slots >>>>>> from an existing >>>>>> vtable (generated by the C++ linker). >>>>>> >>>>>> Per Thomas St?fe's advice, I don't try to >>>>>> determine the size of the vtable (as that >>>>>> would add one more compiler requirement where >>>>>> new virtual methods added by a subclass must >>>>>> be placed at a higher offset in the vtable). >>>>>> >>>>>> Instead, I have added code in non-product >>>>>> builds to ensure that the vtables are no >>>>>> longer than 150 entries. You can run with >>>>>> "-XX:+PrintSharedSpaces -Xshare:dump" to print >>>>>> out the actual size of the vtables for your >>>>>> particular platform: >>>>>> >>>>>> ConstantPool has 12 virtual methods >>>>>> InstanceKlass has 113 virtual methods >>>>>> InstanceClassLoaderKlass has 113 virtual methods >>>>>> InstanceMirrorKlass has 113 virtual methods >>>>>> InstanceRefKlass has 113 virtual methods >>>>>> Method has 12 virtual methods >>>>>> ObjArrayKlass has 114 virtual methods >>>>>> TypeArrayKlass has 114 virtual methods >>>>>> >>>>>> As mentioned in the code comments, if you have >>>>>> an esoteric C++ compiler, the >>>>>> verify_sufficient_size() function will >>>>>> probably fail, but hopefully that would give >>>>>> you some hints for porting this code. >>>>>> >>>>>> To avoid accidentally touching an unmapped >>>>>> page, the code uses SafeFetchN for copying the >>>>>> vtable contents, and would shrink the vtable >>>>>> to less than 150 entries if necessary. I can't >>>>>> test this for real, but I've added some code >>>>>> to simulate an error: >>>>>> >>>>>> for (int i=0; i>>>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>> if (num == bad >>>>>> // || i > 120 /* uncomment this line >>>>>> to test */ >>>>>> ) { >>>>>> _info->set_vtab_size(i-1); >>>>>> break; >>>>>> } >>>>>> dstvtab[i] = num; >>>>>> } >>>>>> >>>>>> Results: >>>>>> >>>>>> + Removed 850 lines of CPU-dependent code >>>>>> >>>>>> + CDS image is about 50K smaller >>>>>> >>>>>> + Previously Metadata objects must live in the >>>>>> read-write section in the CDS >>>>>> archive, because their _vptr was updated at >>>>>> run time. Now _vptr is no longer >>>>>> updated, so ConstantPool can be moved to the >>>>>> read-only section (see JDK-8171392). >>>>>> >>>>>> Thanks >>>>>> - Ioi From ioi.lam at oracle.com Wed Mar 8 05:39:43 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 07 Mar 2017 21:39:43 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <3B78A8C3-216D-43F2-A777-A6DF85828651@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> <58BF24F0.6090704@oracle.com> <58BF636B.7050609@oracle.com> <3B78A8C3-216D-43F2-A777-A6DF85828651@oracle.com> Message-ID: <58BF991F.6020200@oracle.com> On 3/7/17 8:12 PM, Jiangli Zhou wrote: > Hi Ioi, > >> On Mar 7, 2017, at 5:50 PM, Ioi Lam > > wrote: >> >> >> >> On 3/7/17 4:04 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> Some minor comments/suggestions for the latest changes: >>> >>> You added MetaspaceShared::is_valid_method(). There is an existing >>> Method::is_valid_method() function. It might cause confusion for >>> anyone who?s not familiar with the CDS code when trying to determine >>> which is_valid_method() to use in the future. How about renaming >>> MetaspaceShared::is_valid_method() to is_valid_archived_method() and >>> move it to method.*? >> >> I renamed it to MetaspaceShared::is_valid_shared_method(), to be >> consistent with existing names such as >> MetaspaceShared::is_in_shared_space(). >> >> I don't want to move it to method.cpp because the logic of >> determining whether the Method's vptr points to the cloned vtable is >> inside metaspaceShared.cpp > > Ok. > >> >>> Also, MetaspaceShared::is_valid_shared_object() might be more >>> meaningful if it is rename as >>> MetaspaceShared::is_valid_shared_metadata or >>> MetaspaceShared::is_valid_archive_metadata. >> There's no MetaspaceShared::is_valid_shared_object(). I think you >> mean CppVtableCloner::is_valid_shared_object. >> The meaning of the name is pretty clear: is it a valid shared object >> of type > > Using ?shared_object? might be a bit too broad here. We have both > shared metadata and shared String objects. This API is used to > determine if it is a valid data of the Metadata types that have C++ > vtable. Also, I just noticed the implementation doesn?t do any > specific ?is_shared? check. How about changing it to > CppVtableCloner::is_valid()? How about this: template class CppVtableCloner : public T { ... static bool is_valid_shared_object(const T* obj) { intptr_t* vptr = *(intptr_t**)obj; return vptr == _info->cloned_vtable(); } }; bool MetaspaceShared::is_valid_shared_method(const Method* m) { assert(is_in_shared_space(m), "must be"); return CppVtableCloner::is_valid_shared_object(m); } You can pass in only a Method* to CppVtableCloner::is_valid_shared_object(const Method* obj), so it's pretty clear what "obj" is in this case. I also added the "is_shared" check as an assert. The caller should call this only after checking that the method is in shared space. What do you think? Thanks - Ioi > > Thanks, > Jiangli > >> >> Thanks >> - Ioi >>> >>> Thanks, >>> Jiangli >>> >>>> On Mar 7, 2017, at 1:24 PM, Ioi Lam >>> > wrote: >>>> >>>> Hi Jiangli, >>>> >>>> Good catch! >>>> >>>> When I try to remove the code, I actually found my last version >>>> (v04) would not work with Method::is_valid_method(). The following >>>> assert would fail. This means that shared methods be skipped when >>>> we walk the stack (such as when doing a crash dump): >>>> >>>> void Method::restore_unshareable_info(TRAPS) { >>>> - assert(is_method(), "ensure C++ vtable is restored"); >>>> + assert(is_method() && is_valid_method(), "ensure C++ vtable is >>>> restored"); >>>> >>>> So I made a more comprehensive fix to make sure >>>> Method::is_valid_method works. Here's a diff from the previous version: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v05/ >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 3/7/17 9:58 AM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> I think there is no other usage of >>>>> Universe::init_self_patching_vtbl_list() after you removed the >>>>> ones from metaspaceShared.cpp. You can remove the function from >>>>> universe.*. >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On Mar 6, 2017, at 5:25 PM, Ioi Lam wrote: >>>>>> >>>>>> Hi Thomas & Coleen, >>>>>> >>>>>> Thanks for your comments. I have updated the rev: >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ >>>>>> >>>>>> [1] Switched back to computing the exact vtable size >>>>>> [2] Move non-trivial functions outside of their class declaration >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> On 3/6/17 8:51 AM, Thomas St?fe wrote: >>>>>>> Hi Coleen and Ioi, >>>>>>> >>>>>>> I had to port C++ code to platforms with terrible compilers for >>>>>>> a time in my life, that is why I like code to be as portable as >>>>>>> possible. That said, you are right in your argumentation, the >>>>>>> SafeFetch solution is not terribly elegant and Ioi's original >>>>>>> way of determining the vtable size is cleaner. >>>>>>> >>>>>>> I did some checks on some of our architectures with a test >>>>>>> similar to Ioi's and on a first glance it seems to work for >>>>>>> simple cases (single and public inheritance) on ppc (AIX) and >>>>>>> Linux ia64. Although the vtables seemed to me to contain >>>>>>> function descriptors, not real pointers to code, so this is >>>>>>> something to keep in mind. But if the live vtable are copied, >>>>>>> the function descriptors they contain should point to valid code >>>>>>> too, so it should not matter. Just to remember to not expect >>>>>>> every slot in the array to be a valid code pointer. >>>>>>> >>>>>>> So, in short, I remove my objection to Ioi's original solution, >>>>>>> as far as that matters. >>>>>>> >>>>>>> I still think we rely on a lot here: Contiguous vtable >>>>>>> containing absolute memory addresses, vtable pointer at start of >>>>>>> object and vtable entries to be ordered from base->derived >>>>>>> class. So I wonder how much effort it would be (now or in the >>>>>>> future as a separate change) to have a fallback where - at >>>>>>> loading time - instead of copying vtables the vtable pointers in >>>>>>> the objects were fixed up to point to the new live vtables? I >>>>>>> know this would be more expensive and potentially defeat the >>>>>>> point of shared classes. But maybe not, it depends on how many >>>>>>> objects are there, no? >>>>>>> >>>>>>> Kind Regards, Thomas >>>>>>> >>>>>>> On Sun, Mar 5, 2017 at 4:17 PM, >>>>>> > wrote: >>>>>>> >>>>>>> >>>>>>> Ioi, Some comments inline (where no comments, insert "ok") :) >>>>>>> >>>>>>> >>>>>>> On 3/2/17 10:37 PM, Ioi Lam wrote: >>>>>>> >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Thanks for the comments. I have updated the webrev. See >>>>>>> in-line for responses. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> Ioi >>>>>>> I like the concept of this a lot but have some stylistic >>>>>>> comments to help people reading this code later. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>>>>>> >>>>>>> >>>>>>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >>>>>>> save many characters, especially in CppVtableInfo/Testers >>>>>>> >>>>>>> Done. >>>>>>> >>>>>>> + // Start at slot 1, because slot 0 may be RTTI (on >>>>>>> Solaris/Sparc) >>>>>>> + int i; >>>>>>> + for (i=1; ; i++) { >>>>>>> Since you're using 'i' later, can you rename it to >>>>>>> something descriptive. Or have another variable >>>>>>> "vtable_length" to use later. This looks like an old >>>>>>> style for loop. >>>>>>> >>>>>>> Done >>>>>>> >>>>>>> Can the functions for CppVtableInfo be declared >>>>>>> outside of >>>>>>> the class declaration? They don't need to be inline and >>>>>>> then the debug code for testing the vtable size can >>>>>>> be not >>>>>>> in the middle of the class declaration. Then you can >>>>>>> move the Tester classes to inside the same #ifndef >>>>>>> PRODUCT >>>>>>> block. >>>>>>> >>>>>>> Can you put #endif // PRODUCT when the ifdef covers >>>>>>> several lines of code? >>>>>>> >>>>>>> Done >>>>>>> >>>>>>> vtab_of could be more descriptive, like cpp_vtable_for(). >>>>>>> >>>>>>> I changed to vtable_of(). Because the class name is already >>>>>>> CppVtableCloner, repeating the word "cpp" seems >>>>>>> repetitive to me. >>>>>>> >>>>>>> Was PrintSharedSpaces was never converted to UL? >>>>>>> >>>>>>> Right. I've filed >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176132 >>>>>>> >>>>>>> (-XX:+PrintSharedSpaces should be converted to use Unified >>>>>>> Logging.) >>>>>>> >>>>>>> + int n = MAX_VTABLE_SIZE; >>>>>>> >>>>>>> Can you propagate MAX_VTABLE_SIZE to the places where >>>>>>> it's >>>>>>> used. n isn't descriptive. This starts out with >>>>>>> max_vtable_size and then changes the size. Reusing 'n' >>>>>>> makes this really hard to follow. Not having a comment >>>>>>> that we only allocate enough slots for the vtable >>>>>>> makes it >>>>>>> hard too. >>>>>>> >>>>>>> + // allocate CppVtableInfo in the MD section + _info = >>>>>>> (CppVtabInfo*)md_top; >>>>>>> + _info->set_vtab_size(n); // initially set to >>>>>>> max_vtable_size >>>>>>> + + // allocate temporary local instance of the metadata >>>>>>> type T + T tmp; >>>>>>> + intptr_t* srcvtab = vtab_of(tmp); >>>>>>> + intptr_t* dstvtab = _info->vtab(); >>>>>>> + >>>>>>> >>>>>>> Fixed. >>>>>>> >>>>>>> Something like that for comments. dstvtab is the >>>>>>> destination_vtable in the MD section. >>>>>>> >>>>>>> >>>>>>> I've dropped the md_ prefix from the functions that deal with >>>>>>> the vtables, since they shouldn't care whether it's the "MD" >>>>>>> section or not. Now it looks like this: >>>>>>> >>>>>>> // Allocate and initialize the C++ vtables, starting from >>>>>>> top, >>>>>>> but do not go past end. >>>>>>> intptr_t* >>>>>>> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>>>>>> intptr_t* end) { >>>>>>> assert(DumpSharedSpaces, "dump-time only"); >>>>>>> // Layout (each slot is a intptr_t): >>>>>>> // [number of slots in the first vtable = n1] >>>>>>> // [ slots for the first vtable] >>>>>>> // [number of slots in the first second = n2] >>>>>>> // [ slots for the second vtable] >>>>>>> // ... >>>>>>> // The order of the vtables is the same as the >>>>>>> CPP_VTAB_PATCH_TYPES_DO macro. >>>>>>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>>>>>> return top; >>>>>>> } >>>>>>> >>>>>>> + for (int i=0; i>>>>>> + const intptr_t bad = intptr_t(0xdeadbeef); >>>>>>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>>> + if (num == bad >>>>>>> + // || i > 120 /* uncomment this line to test */ >>>>>>> + ) { >>>>>>> + _info->set_vtab_size(i-1); >>>>>>> + break; >>>>>>> + } >>>>>>> + dstvtab[i] = num; >>>>>>> + } >>>>>>> I dont understand this code. You get deadbeef for a bad >>>>>>> value if SafeFetchN gets a fault but why would it get a >>>>>>> fault at the end of the metadata's vtable? Couldn't it >>>>>>> just run onto the next vtable? I think your original way >>>>>>> of counting vtable entries might be better (sorry I >>>>>>> didn't >>>>>>> have time to study that thread). >>>>>>> >>>>>>> I've modified the comments to this. Does it make sense to >>>>>>> you? >>>>>>> >>>>>>> // It is not always safe to call memcpy(), because >>>>>>> srcvtable might be shorter than >>>>>>> // MAX_VTABLE_SIZE, and the C++ linker might have placed >>>>>>> the vtable at the very >>>>>>> // end of the last page of libjvm.so. Crossing over >>>>>>> to the >>>>>>> next page might >>>>>>> // cause a page fault. >>>>>>> >>>>>>> My fear is the JVM would suddenly start crashing because the >>>>>>> order of .o files have changed on the linker's command line, >>>>>>> or if you enable some special linker optimization flags. It's >>>>>>> better safe than sorry. >>>>>>> >>>>>>> >>>>>>> This wasn't exactly what I was not understanding. I didn't see >>>>>>> that you are copying 120 entries from the old vtable and junk >>>>>>> memory beyond the old vtable, unless you get a segv, in which >>>>>>> case >>>>>>> you copy less. I don't think you should copy random memory into >>>>>>> the vtable in the archive. This doesn't seem secure, even with >>>>>>> the segv protection. >>>>>>> >>>>>>> Since we already have assumptions about C++ vtable layout in the >>>>>>> code and it's mostly specified by various ABIs, and you have the >>>>>>> assert code, I think I would prefer that you copy only the vtable >>>>>>> entries into the archive. I guess Thomas Stuefe had a different >>>>>>> opinion. I've read the original thread. Two points: >>>>>>> >>>>>>> If new C++ compiler implementations add a discontigous vtable, >>>>>>> both the SafeFetchN and subclass additional virtual function at >>>>>>> end implementation will fail. I don't think C++ implementations >>>>>>> would do this and a contiguous vtable as first in the >>>>>>> instance has >>>>>>> been standard for years. If our metadata adds multiple >>>>>>> inheritance, the same issue would be a problem for both >>>>>>> implementations, as well as for the implementation we have before >>>>>>> Ioi's fix. >>>>>>> >>>>>>> Ioi's subclass adding virtual function method would work for any >>>>>>> esoteric C++ implementations in my memory, except the vptr >>>>>>> for the >>>>>>> old DECC++ compiler was after the nonstatic data members (which >>>>>>> would fail with all of our implementations). >>>>>>> >>>>>>> Since the code is there anyway for debug purposes, we're not >>>>>>> saving code by implementing SafeFetchN. The SafeFetchN >>>>>>> implementation isn't obvious at all what it's doing, and requires >>>>>>> better comments, especially if you don't know already what >>>>>>> SafeFetchN does. It looks really cryptic. The poisoned values >>>>>>> also bothered me in that they overload other poisoned values in >>>>>>> other parts of the jvm. >>>>>>> >>>>>>> Ioi, could you make all methods of CppVtableCloner out of line? >>>>>>> >>>>>>> The other changes look good, although I might have more requests >>>>>>> for comments. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> Would be nice to have comments here too!! >>>>>>> >>>>>>> + intptr_t* start = md_top; >>>>>>> >>>>>>> This doesn't do anything (?) >>>>>>> >>>>>>> >>>>>>> Fixed. This was left over code. >>>>>>> >>>>>>> >>>>>>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>>>>> >>>>>>> Why not zero the destination vtable in allocate? Or does >>>>>>> patching the vtable pointers call virtual functions? You >>>>>>> could prevent that so you don't need this code. >>>>>>> >>>>>>> I added this comment: >>>>>>> >>>>>>> // During patching, some virtual methods may be called, so >>>>>>> at this point >>>>>>> // the vtables must contain valid methods (as filled in by >>>>>>> CppVtableCloner::allocate). >>>>>>> MetaspaceShared::patch_cpp_vtable_pointers(); >>>>>>> >>>>>>> // The vtable clones contain addresses of the current >>>>>>> process. >>>>>>> // We don't want to write these addresses into the archive. >>>>>>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>>>>> >>>>>>> + // Restore the vtable in case we invoke any virtual >>>>>>> methods. >>>>>>> + >>>>>>> MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>>>>>> Can this be restore_cpp_vtables since that's what it's >>>>>>> doing. The first is after the dump and the second call is >>>>>>> at UseSharedSpaces. A couple of comments in this >>>>>>> clone_cpp_vtables --> restore_cpp_vtables would be >>>>>>> nice. eg: >>>>>>> >>>>>>> I prefer to use the word clone. Otherwise when you just say >>>>>>> "vtable" it's not clear whether you're talking about the >>>>>>> original one (made by the c++ linker), or the cloned one in >>>>>>> the CDS archive. >>>>>>> >>>>>>> + static intptr_t* clone_vtable(const char* name, >>>>>>> intptr_t* p) { >>>>>>> + T tmp; // Allocate temporary dummy metadata object to >>>>>>> get vtable initialized >>>>>>> + CppVtabInfo* info = (CppVtabInfo*)p; >>>>>>> + int n = info->vtab_size(); >>>>>>> + intptr_t* srcvtab = vtab_of(tmp); >>>>>>> + intptr_t* dstvtab = info->vtab(); >>>>>>> + >>>>>>> + // We already checked (and, if necessary, adjusted n) >>>>>>> when the vtables were allocated, so we are >>>>>>> + // safe to do memcpy. >>>>>>> + if (PrintSharedSpaces) { >>>>>>> + tty->print_cr("%s copying %d vtable entries", name, n); >>>>>>> + } >>>>>>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>>>>>> + return dstvtab + n; >>>>>>> + } >>>>>>> >>>>>>> Done. I changed the wording >>>>>>> >>>>>>> T tmp; // Allocate temporary dummy metadata object to get >>>>>>> to the original vtable. >>>>>>> >>>>>>> As we are not really "initializing a vtable" here. >>>>>>> >>>>>>> Same with 'patch'. It'd be so much faster and easier to >>>>>>> read this code with more comments please. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>>>>>> >>>>>>> >>>>>>> Why are these testers here? >>>>>>> >>>>>>> >>>>>>> I updated the comment: >>>>>>> >>>>>>> // Used by CDS. These classes need to access the private >>>>>>> ConstantPool() constructor. >>>>>>> template friend class CppVtableTesterA; >>>>>>> template friend class CppVtableTesterB; >>>>>>> template friend class CppVtableCloner; >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This is the official review (follow up of the >>>>>>> "Determining the size of C++ vtables" thread >>>>>>> onhotspot-dev at openjdk.java.net >>>>>>> ). >>>>>>> >>>>>>> The new code has the same assumption as the >>>>>>> existing code in JDK 10: for a C++ object >>>>>>> that >>>>>>> contains virtual methods (e.g., >>>>>>> ConstantPool), >>>>>>> we assume the first intptr_t slot of the >>>>>>> object is a _vptr, which points to a vtable, >>>>>>> which consists of no more than 150 >>>>>>> intptr_t's. >>>>>>> >>>>>>> ConstantPool*p -->[ _vptr ] -------> [ >>>>>>> vtable slot 0 ] >>>>>>> [ field #0 ] [ >>>>>>> vtable slot 1 ] >>>>>>> [ field #1 ] [ >>>>>>> vtable slot 2 ] >>>>>>> [ field #2 ] [ >>>>>>> vtable slot 3 ] >>>>>>> [ .... ] [ >>>>>>> vtable slot 4] >>>>>>> [ >>>>>>> vtable slot 5 ] >>>>>>> [ >>>>>>> ... ] >>>>>>> >>>>>>> + In the existing code, we were pointing the >>>>>>> vtable slots to >>>>>>> code that's generated by HotSpot. >>>>>>> >>>>>>> + In the new code, we copy the vtable slots >>>>>>> from an existing >>>>>>> vtable (generated by the C++ linker). >>>>>>> >>>>>>> Per Thomas St?fe's advice, I don't try to >>>>>>> determine the size of the vtable (as that >>>>>>> would add one more compiler requirement where >>>>>>> new virtual methods added by a subclass must >>>>>>> be placed at a higher offset in the vtable). >>>>>>> >>>>>>> Instead, I have added code in non-product >>>>>>> builds to ensure that the vtables are no >>>>>>> longer than 150 entries. You can run with >>>>>>> "-XX:+PrintSharedSpaces -Xshare:dump" to >>>>>>> print >>>>>>> out the actual size of the vtables for your >>>>>>> particular platform: >>>>>>> >>>>>>> ConstantPool has 12 virtual methods >>>>>>> InstanceKlass has 113 virtual methods >>>>>>> InstanceClassLoaderKlass has 113 >>>>>>> virtual methods >>>>>>> InstanceMirrorKlass has 113 virtual methods >>>>>>> InstanceRefKlass has 113 virtual methods >>>>>>> Method has 12 virtual methods >>>>>>> ObjArrayKlass has 114 virtual methods >>>>>>> TypeArrayKlass has 114 virtual methods >>>>>>> >>>>>>> As mentioned in the code comments, if you >>>>>>> have >>>>>>> an esoteric C++ compiler, the >>>>>>> verify_sufficient_size() function will >>>>>>> probably fail, but hopefully that would give >>>>>>> you some hints for porting this code. >>>>>>> >>>>>>> To avoid accidentally touching an unmapped >>>>>>> page, the code uses SafeFetchN for >>>>>>> copying the >>>>>>> vtable contents, and would shrink the vtable >>>>>>> to less than 150 entries if necessary. I >>>>>>> can't >>>>>>> test this for real, but I've added some code >>>>>>> to simulate an error: >>>>>>> >>>>>>> for (int i=0; i>>>>>> const intptr_t bad = >>>>>>> intptr_t(0xdeadbeef); >>>>>>> intptr_t num = >>>>>>> SafeFetchN(&srcvtab[i], bad); >>>>>>> if (num == bad >>>>>>> // || i > 120 /* uncomment this >>>>>>> line >>>>>>> to test */ >>>>>>> ) { >>>>>>> _info->set_vtab_size(i-1); >>>>>>> break; >>>>>>> } >>>>>>> dstvtab[i] = num; >>>>>>> } >>>>>>> >>>>>>> Results: >>>>>>> >>>>>>> + Removed 850 lines of CPU-dependent code >>>>>>> >>>>>>> + CDS image is about 50K smaller >>>>>>> >>>>>>> + Previously Metadata objects must live >>>>>>> in the >>>>>>> read-write section in the CDS >>>>>>> archive, because their _vptr was updated at >>>>>>> run time. Now _vptr is no longer >>>>>>> updated, so ConstantPool can be moved >>>>>>> to the >>>>>>> read-only section (see JDK-8171392). >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi > From jiangli.zhou at oracle.com Wed Mar 8 06:35:54 2017 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 7 Mar 2017 22:35:54 -0800 Subject: RFR[S] 8005165 Platform-independent C++ vtables for CDS In-Reply-To: <58BF991F.6020200@oracle.com> References: <58AFACB7.7020203@oracle.com> <58AFAEAE.2080205@oracle.com> <58B0805F.20205@oracle.com> <58B63114.6010806@oracle.com> <64661331-8998-44B4-97DF-D13A6EAC17E0@oracle.com> <58B8E4F7.5050301@oracle.com> <7d2c32ff-130e-8671-305f-cab1ecd9bb84@oracle.com> <58BE0C03.1040105@oracle.com> <1A6BC6C7-0457-4028-836B-1DC5B83E6C38@oracle.com> <58BF24F0.6090704@oracle.com> <58BF636B.7050609@oracle.com> <3B78A8C3-216D-43F2-A777-A6DF85828651@oracle.com> <58BF991F.6020200@oracle.com> Message-ID: Hi Ioi, > On Mar 7, 2017, at 9:39 PM, Ioi Lam wrote: > > > > On 3/7/17 8:12 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >>> On Mar 7, 2017, at 5:50 PM, Ioi Lam > wrote: >>> >>> >>> >>> On 3/7/17 4:04 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> Some minor comments/suggestions for the latest changes: >>>> >>>> You added MetaspaceShared::is_valid_method(). There is an existing Method::is_valid_method() function. It might cause confusion for anyone who?s not familiar with the CDS code when trying to determine which is_valid_method() to use in the future. How about renaming MetaspaceShared::is_valid_method() to is_valid_archived_method() and move it to method.*? >>> >>> I renamed it to MetaspaceShared::is_valid_shared_method(), to be consistent with existing names such as MetaspaceShared::is_in_shared_space(). >>> >>> I don't want to move it to method.cpp because the logic of determining whether the Method's vptr points to the cloned vtable is inside metaspaceShared.cpp >> >> Ok. >> >>> >>>> Also, MetaspaceShared::is_valid_shared_object() might be more meaningful if it is rename as MetaspaceShared::is_valid_shared_metadata or MetaspaceShared::is_valid_archive_metadata. >>> There's no MetaspaceShared::is_valid_shared_object(). I think you mean CppVtableCloner::is_valid_shared_object. >>> The meaning of the name is pretty clear: is it a valid shared object of type >> >> Using ?shared_object? might be a bit too broad here. We have both shared metadata and shared String objects. This API is used to determine if it is a valid data of the Metadata types that have C++ vtable. Also, I just noticed the implementation doesn?t do any specific ?is_shared? check. How about changing it to CppVtableCloner::is_valid()? > > How about this: > > template class CppVtableCloner : public T { > ... > static bool is_valid_shared_object(const T* obj) { > intptr_t* vptr = *(intptr_t**)obj; > return vptr == _info->cloned_vtable(); > } > }; > > bool MetaspaceShared::is_valid_shared_method(const Method* m) { > assert(is_in_shared_space(m), "must be"); > return CppVtableCloner::is_valid_shared_object(m); > } > > You can pass in only a Method* to CppVtableCloner::is_valid_shared_object(const Method* obj), so it's pretty clear what "obj" is in this case. > > I also added the "is_shared" check as an assert. The caller should call this only after checking that the method is in shared space. > > What do you think? Looks ok. Thank you for making the change. Thanks, Jiangli > > Thanks > - Ioi >> >> Thanks, >> Jiangli >> >>> >>> Thanks >>> - Ioi >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On Mar 7, 2017, at 1:24 PM, Ioi Lam > wrote: >>>>> >>>>> Hi Jiangli, >>>>> >>>>> Good catch! >>>>> >>>>> When I try to remove the code, I actually found my last version (v04) would not work with Method::is_valid_method(). The following assert would fail. This means that shared methods be skipped when we walk the stack (such as when doing a crash dump): >>>>> >>>>> void Method::restore_unshareable_info(TRAPS) { >>>>> - assert(is_method(), "ensure C++ vtable is restored"); >>>>> + assert(is_method() && is_valid_method(), "ensure C++ vtable is restored"); >>>>> >>>>> So I made a more comprehensive fix to make sure Method::is_valid_method works. Here's a diff from the previous version: >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v05/ >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 3/7/17 9:58 AM, Jiangli Zhou wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> I think there is no other usage of Universe::init_self_patching_vtbl_list() after you removed the ones from metaspaceShared.cpp. You can remove the function from universe.*. >>>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>>> On Mar 6, 2017, at 5:25 PM, Ioi Lam wrote: >>>>>>> >>>>>>> Hi Thomas & Coleen, >>>>>>> >>>>>>> Thanks for your comments. I have updated the rev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v04/ >>>>>>> >>>>>>> [1] Switched back to computing the exact vtable size >>>>>>> [2] Move non-trivial functions outside of their class declaration >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> On 3/6/17 8:51 AM, Thomas St?fe wrote: >>>>>>>> Hi Coleen and Ioi, >>>>>>>> >>>>>>>> I had to port C++ code to platforms with terrible compilers for a time in my life, that is why I like code to be as portable as possible. That said, you are right in your argumentation, the SafeFetch solution is not terribly elegant and Ioi's original way of determining the vtable size is cleaner. >>>>>>>> >>>>>>>> I did some checks on some of our architectures with a test similar to Ioi's and on a first glance it seems to work for simple cases (single and public inheritance) on ppc (AIX) and Linux ia64. Although the vtables seemed to me to contain function descriptors, not real pointers to code, so this is something to keep in mind. But if the live vtable are copied, the function descriptors they contain should point to valid code too, so it should not matter. Just to remember to not expect every slot in the array to be a valid code pointer. >>>>>>>> >>>>>>>> So, in short, I remove my objection to Ioi's original solution, as far as that matters. >>>>>>>> >>>>>>>> I still think we rely on a lot here: Contiguous vtable containing absolute memory addresses, vtable pointer at start of object and vtable entries to be ordered from base->derived class. So I wonder how much effort it would be (now or in the future as a separate change) to have a fallback where - at loading time - instead of copying vtables the vtable pointers in the objects were fixed up to point to the new live vtables? I know this would be more expensive and potentially defeat the point of shared classes. But maybe not, it depends on how many objects are there, no? >>>>>>>> >>>>>>>> Kind Regards, Thomas >>>>>>>> >>>>>>>> On Sun, Mar 5, 2017 at 4:17 PM, > wrote: >>>>>>>> >>>>>>>> >>>>>>>> Ioi, Some comments inline (where no comments, insert "ok") :) >>>>>>>> >>>>>>>> >>>>>>>> On 3/2/17 10:37 PM, Ioi Lam wrote: >>>>>>>> >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> Thanks for the comments. I have updated the webrev. See >>>>>>>> in-line for responses. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v03/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 3/2/17 8:48 AM, coleen.phillimore at oracle.com >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> Ioi >>>>>>>> I like the concept of this a lot but have some stylistic >>>>>>>> comments to help people reading this code later. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/memory/metaspaceShared.cpp.udiff.html >>>>>>>> >>>>>>>> >>>>>>>> s/vtab/vtable/g and s/Vtab/Vtable/ please. It doesn't >>>>>>>> save many characters, especially in CppVtableInfo/Testers >>>>>>>> >>>>>>>> Done. >>>>>>>> >>>>>>>> + // Start at slot 1, because slot 0 may be RTTI (on >>>>>>>> Solaris/Sparc) >>>>>>>> + int i; >>>>>>>> + for (i=1; ; i++) { >>>>>>>> Since you're using 'i' later, can you rename it to >>>>>>>> something descriptive. Or have another variable >>>>>>>> "vtable_length" to use later. This looks like an old >>>>>>>> style for loop. >>>>>>>> >>>>>>>> Done >>>>>>>> >>>>>>>> Can the functions for CppVtableInfo be declared outside of >>>>>>>> the class declaration? They don't need to be inline and >>>>>>>> then the debug code for testing the vtable size can be not >>>>>>>> in the middle of the class declaration. Then you can >>>>>>>> move the Tester classes to inside the same #ifndef PRODUCT >>>>>>>> block. >>>>>>>> >>>>>>>> Can you put #endif // PRODUCT when the ifdef covers >>>>>>>> several lines of code? >>>>>>>> >>>>>>>> Done >>>>>>>> >>>>>>>> vtab_of could be more descriptive, like cpp_vtable_for(). >>>>>>>> >>>>>>>> I changed to vtable_of(). Because the class name is already >>>>>>>> CppVtableCloner, repeating the word "cpp" seems repetitive to me. >>>>>>>> >>>>>>>> Was PrintSharedSpaces was never converted to UL? >>>>>>>> >>>>>>>> Right. I've filed >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176132 >>>>>>>> >>>>>>>> (-XX:+PrintSharedSpaces should be converted to use Unified >>>>>>>> Logging.) >>>>>>>> >>>>>>>> + int n = MAX_VTABLE_SIZE; >>>>>>>> >>>>>>>> Can you propagate MAX_VTABLE_SIZE to the places where it's >>>>>>>> used. n isn't descriptive. This starts out with >>>>>>>> max_vtable_size and then changes the size. Reusing 'n' >>>>>>>> makes this really hard to follow. Not having a comment >>>>>>>> that we only allocate enough slots for the vtable makes it >>>>>>>> hard too. >>>>>>>> >>>>>>>> + // allocate CppVtableInfo in the MD section + _info = >>>>>>>> (CppVtabInfo*)md_top; >>>>>>>> + _info->set_vtab_size(n); // initially set to max_vtable_size >>>>>>>> + + // allocate temporary local instance of the metadata >>>>>>>> type T + T tmp; >>>>>>>> + intptr_t* srcvtab = vtab_of(tmp); >>>>>>>> + intptr_t* dstvtab = _info->vtab(); >>>>>>>> + >>>>>>>> >>>>>>>> Fixed. >>>>>>>> >>>>>>>> Something like that for comments. dstvtab is the >>>>>>>> destination_vtable in the MD section. >>>>>>>> >>>>>>>> >>>>>>>> I've dropped the md_ prefix from the functions that deal with >>>>>>>> the vtables, since they shouldn't care whether it's the "MD" >>>>>>>> section or not. Now it looks like this: >>>>>>>> >>>>>>>> // Allocate and initialize the C++ vtables, starting from top, >>>>>>>> but do not go past end. >>>>>>>> intptr_t* >>>>>>>> MetaspaceShared::allocate_cpp_vtable_clones(intptr_t* top, >>>>>>>> intptr_t* end) { >>>>>>>> assert(DumpSharedSpaces, "dump-time only"); >>>>>>>> // Layout (each slot is a intptr_t): >>>>>>>> // [number of slots in the first vtable = n1] >>>>>>>> // [ slots for the first vtable] >>>>>>>> // [number of slots in the first second = n2] >>>>>>>> // [ slots for the second vtable] >>>>>>>> // ... >>>>>>>> // The order of the vtables is the same as the >>>>>>>> CPP_VTAB_PATCH_TYPES_DO macro. >>>>>>>> CPP_VTABLE_PATCH_TYPES_DO(ALLOC_CPP_VTABLE_CLONE); >>>>>>>> return top; >>>>>>>> } >>>>>>>> >>>>>>>> + for (int i=0; i>>>>>>> + const intptr_t bad = intptr_t(0xdeadbeef); >>>>>>>> + intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>>>> + if (num == bad >>>>>>>> + // || i > 120 /* uncomment this line to test */ >>>>>>>> + ) { >>>>>>>> + _info->set_vtab_size(i-1); >>>>>>>> + break; >>>>>>>> + } >>>>>>>> + dstvtab[i] = num; >>>>>>>> + } >>>>>>>> I dont understand this code. You get deadbeef for a bad >>>>>>>> value if SafeFetchN gets a fault but why would it get a >>>>>>>> fault at the end of the metadata's vtable? Couldn't it >>>>>>>> just run onto the next vtable? I think your original way >>>>>>>> of counting vtable entries might be better (sorry I didn't >>>>>>>> have time to study that thread). >>>>>>>> >>>>>>>> I've modified the comments to this. Does it make sense to you? >>>>>>>> >>>>>>>> // It is not always safe to call memcpy(), because >>>>>>>> srcvtable might be shorter than >>>>>>>> // MAX_VTABLE_SIZE, and the C++ linker might have placed >>>>>>>> the vtable at the very >>>>>>>> // end of the last page of libjvm.so. Crossing over to the >>>>>>>> next page might >>>>>>>> // cause a page fault. >>>>>>>> >>>>>>>> My fear is the JVM would suddenly start crashing because the >>>>>>>> order of .o files have changed on the linker's command line, >>>>>>>> or if you enable some special linker optimization flags. It's >>>>>>>> better safe than sorry. >>>>>>>> >>>>>>>> >>>>>>>> This wasn't exactly what I was not understanding. I didn't see >>>>>>>> that you are copying 120 entries from the old vtable and junk >>>>>>>> memory beyond the old vtable, unless you get a segv, in which case >>>>>>>> you copy less. I don't think you should copy random memory into >>>>>>>> the vtable in the archive. This doesn't seem secure, even with >>>>>>>> the segv protection. >>>>>>>> >>>>>>>> Since we already have assumptions about C++ vtable layout in the >>>>>>>> code and it's mostly specified by various ABIs, and you have the >>>>>>>> assert code, I think I would prefer that you copy only the vtable >>>>>>>> entries into the archive. I guess Thomas Stuefe had a different >>>>>>>> opinion. I've read the original thread. Two points: >>>>>>>> >>>>>>>> If new C++ compiler implementations add a discontigous vtable, >>>>>>>> both the SafeFetchN and subclass additional virtual function at >>>>>>>> end implementation will fail. I don't think C++ implementations >>>>>>>> would do this and a contiguous vtable as first in the instance has >>>>>>>> been standard for years. If our metadata adds multiple >>>>>>>> inheritance, the same issue would be a problem for both >>>>>>>> implementations, as well as for the implementation we have before >>>>>>>> Ioi's fix. >>>>>>>> >>>>>>>> Ioi's subclass adding virtual function method would work for any >>>>>>>> esoteric C++ implementations in my memory, except the vptr for the >>>>>>>> old DECC++ compiler was after the nonstatic data members (which >>>>>>>> would fail with all of our implementations). >>>>>>>> >>>>>>>> Since the code is there anyway for debug purposes, we're not >>>>>>>> saving code by implementing SafeFetchN. The SafeFetchN >>>>>>>> implementation isn't obvious at all what it's doing, and requires >>>>>>>> better comments, especially if you don't know already what >>>>>>>> SafeFetchN does. It looks really cryptic. The poisoned values >>>>>>>> also bothered me in that they overload other poisoned values in >>>>>>>> other parts of the jvm. >>>>>>>> >>>>>>>> Ioi, could you make all methods of CppVtableCloner out of line? >>>>>>>> >>>>>>>> The other changes look good, although I might have more requests >>>>>>>> for comments. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> >>>>>>>> Would be nice to have comments here too!! >>>>>>>> >>>>>>>> + intptr_t* start = md_top; >>>>>>>> >>>>>>>> This doesn't do anything (?) >>>>>>>> >>>>>>>> >>>>>>>> Fixed. This was left over code. >>>>>>>> >>>>>>>> >>>>>>>> + MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>>>>>> >>>>>>>> Why not zero the destination vtable in allocate? Or does >>>>>>>> patching the vtable pointers call virtual functions? You >>>>>>>> could prevent that so you don't need this code. >>>>>>>> >>>>>>>> I added this comment: >>>>>>>> >>>>>>>> // During patching, some virtual methods may be called, so >>>>>>>> at this point >>>>>>>> // the vtables must contain valid methods (as filled in by >>>>>>>> CppVtableCloner::allocate). >>>>>>>> MetaspaceShared::patch_cpp_vtable_pointers(); >>>>>>>> >>>>>>>> // The vtable clones contain addresses of the current process. >>>>>>>> // We don't want to write these addresses into the archive. >>>>>>>> MetaspaceShared::zero_cpp_vtable_clones_for_writing(); >>>>>>>> >>>>>>>> + // Restore the vtable in case we invoke any virtual methods. >>>>>>>> + MetaspaceShared::clone_cpp_vtables((intptr_t*)vtbl_list); >>>>>>>> Can this be restore_cpp_vtables since that's what it's >>>>>>>> doing. The first is after the dump and the second call is >>>>>>>> at UseSharedSpaces. A couple of comments in this >>>>>>>> clone_cpp_vtables --> restore_cpp_vtables would be nice. eg: >>>>>>>> >>>>>>>> I prefer to use the word clone. Otherwise when you just say >>>>>>>> "vtable" it's not clear whether you're talking about the >>>>>>>> original one (made by the c++ linker), or the cloned one in >>>>>>>> the CDS archive. >>>>>>>> >>>>>>>> + static intptr_t* clone_vtable(const char* name, >>>>>>>> intptr_t* p) { >>>>>>>> + T tmp; // Allocate temporary dummy metadata object to >>>>>>>> get vtable initialized >>>>>>>> + CppVtabInfo* info = (CppVtabInfo*)p; >>>>>>>> + int n = info->vtab_size(); >>>>>>>> + intptr_t* srcvtab = vtab_of(tmp); >>>>>>>> + intptr_t* dstvtab = info->vtab(); >>>>>>>> + >>>>>>>> + // We already checked (and, if necessary, adjusted n) >>>>>>>> when the vtables were allocated, so we are >>>>>>>> + // safe to do memcpy. >>>>>>>> + if (PrintSharedSpaces) { >>>>>>>> + tty->print_cr("%s copying %d vtable entries", name, n); >>>>>>>> + } >>>>>>>> + memcpy(dstvtab, srcvtab, sizeof(intptr_t) * n); >>>>>>>> + return dstvtab + n; >>>>>>>> + } >>>>>>>> >>>>>>>> Done. I changed the wording >>>>>>>> >>>>>>>> T tmp; // Allocate temporary dummy metadata object to get >>>>>>>> to the original vtable. >>>>>>>> >>>>>>>> As we are not really "initializing a vtable" here. >>>>>>>> >>>>>>>> Same with 'patch'. It'd be so much faster and easier to >>>>>>>> read this code with more comments please. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/src/share/vm/oops/constantPool.hpp.udiff.html >>>>>>>> >>>>>>>> >>>>>>>> Why are these testers here? >>>>>>>> >>>>>>>> >>>>>>>> I updated the comment: >>>>>>>> >>>>>>>> // Used by CDS. These classes need to access the private >>>>>>>> ConstantPool() constructor. >>>>>>>> template friend class CppVtableTesterA; >>>>>>>> template friend class CppVtableTesterB; >>>>>>>> template friend class CppVtableCloner; >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> On 3/1/17 3:25 AM, Ioi Lam wrote: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8005165 >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk10/8005165-platform-independent-cds-vtable.v02/ >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is the official review (follow up of the >>>>>>>> "Determining the size of C++ vtables" thread >>>>>>>> onhotspot-dev at openjdk.java.net >>>>>>>> ). >>>>>>>> >>>>>>>> The new code has the same assumption as the >>>>>>>> existing code in JDK 10: for a C++ object that >>>>>>>> contains virtual methods (e.g., ConstantPool), >>>>>>>> we assume the first intptr_t slot of the >>>>>>>> object is a _vptr, which points to a vtable, >>>>>>>> which consists of no more than 150 intptr_t's. >>>>>>>> >>>>>>>> ConstantPool*p -->[ _vptr ] -------> [ >>>>>>>> vtable slot 0 ] >>>>>>>> [ field #0 ] [ >>>>>>>> vtable slot 1 ] >>>>>>>> [ field #1 ] [ >>>>>>>> vtable slot 2 ] >>>>>>>> [ field #2 ] [ >>>>>>>> vtable slot 3 ] >>>>>>>> [ .... ] [ >>>>>>>> vtable slot 4] >>>>>>>> [ >>>>>>>> vtable slot 5 ] >>>>>>>> [ >>>>>>>> ... ] >>>>>>>> >>>>>>>> + In the existing code, we were pointing the >>>>>>>> vtable slots to >>>>>>>> code that's generated by HotSpot. >>>>>>>> >>>>>>>> + In the new code, we copy the vtable slots >>>>>>>> from an existing >>>>>>>> vtable (generated by the C++ linker). >>>>>>>> >>>>>>>> Per Thomas St?fe's advice, I don't try to >>>>>>>> determine the size of the vtable (as that >>>>>>>> would add one more compiler requirement where >>>>>>>> new virtual methods added by a subclass must >>>>>>>> be placed at a higher offset in the vtable). >>>>>>>> >>>>>>>> Instead, I have added code in non-product >>>>>>>> builds to ensure that the vtables are no >>>>>>>> longer than 150 entries. You can run with >>>>>>>> "-XX:+PrintSharedSpaces -Xshare:dump" to print >>>>>>>> out the actual size of the vtables for your >>>>>>>> particular platform: >>>>>>>> >>>>>>>> ConstantPool has 12 virtual methods >>>>>>>> InstanceKlass has 113 virtual methods >>>>>>>> InstanceClassLoaderKlass has 113 virtual methods >>>>>>>> InstanceMirrorKlass has 113 virtual methods >>>>>>>> InstanceRefKlass has 113 virtual methods >>>>>>>> Method has 12 virtual methods >>>>>>>> ObjArrayKlass has 114 virtual methods >>>>>>>> TypeArrayKlass has 114 virtual methods >>>>>>>> >>>>>>>> As mentioned in the code comments, if you have >>>>>>>> an esoteric C++ compiler, the >>>>>>>> verify_sufficient_size() function will >>>>>>>> probably fail, but hopefully that would give >>>>>>>> you some hints for porting this code. >>>>>>>> >>>>>>>> To avoid accidentally touching an unmapped >>>>>>>> page, the code uses SafeFetchN for copying the >>>>>>>> vtable contents, and would shrink the vtable >>>>>>>> to less than 150 entries if necessary. I can't >>>>>>>> test this for real, but I've added some code >>>>>>>> to simulate an error: >>>>>>>> >>>>>>>> for (int i=0; i>>>>>>> const intptr_t bad = intptr_t(0xdeadbeef); >>>>>>>> intptr_t num = SafeFetchN(&srcvtab[i], bad); >>>>>>>> if (num == bad >>>>>>>> // || i > 120 /* uncomment this line >>>>>>>> to test */ >>>>>>>> ) { >>>>>>>> _info->set_vtab_size(i-1); >>>>>>>> break; >>>>>>>> } >>>>>>>> dstvtab[i] = num; >>>>>>>> } >>>>>>>> >>>>>>>> Results: >>>>>>>> >>>>>>>> + Removed 850 lines of CPU-dependent code >>>>>>>> >>>>>>>> + CDS image is about 50K smaller >>>>>>>> >>>>>>>> + Previously Metadata objects must live in the >>>>>>>> read-write section in the CDS >>>>>>>> archive, because their _vptr was updated at >>>>>>>> run time. Now _vptr is no longer >>>>>>>> updated, so ConstantPool can be moved to the >>>>>>>> read-only section (see JDK-8171392). >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >> > From mikael.gerdin at oracle.com Wed Mar 8 14:16:35 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 8 Mar 2017 15:16:35 +0100 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks Message-ID: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> Hi all, Please review this change to the lock rank of the locks taken in the G1 pre and post write barriers. The rank of these locks have long been a problem since even though they are leaf locks semantically they have been ranked as "nonleaf" locks in the lock rank system and this has caused several issues over the years where a thread holding a VM mutex and attempting to write an oop would in some rare cases hit a deadlock warning due to it acquiring one of the CBL monitors. Now this problem has come up yet again with the weak JNI handles bugfix where a lock rank assertion was hit yet again due to the fact that some code was holding a leaf lock while resolving a weak JNI handle. I suggest that the ranks of the involved locks are changed to "leaf - 1", allowing them to be acquired by threads holding "leaf" locks. This should not cause any further problems since reducing the ranks only allows the locks to be taken in more places. Both pairs of locks (the SATB and DirtyCardQ ones) have an associated FL_lock which protects a free list of buffers which is acquired while holding the respective CBL monitors but the FL_locks have the "special" lock rank and so they will still order correctly with the new "leaf - 1" ranks. There is some horrible stuff going on in locking_enqueue_completed_buffer but I've chosen to not change that at this point in time even though it relates to the relative ranks of each pair of locks. Testing: JPRT, HS tiers 2 and 3 were tested with a patch to acquire the locks even more often than what is normal to make sure that the code was tested properly. Some testing on HS tiers 4, 5 and 6 (Linux x64) JDK tiers 1, 2 and 3 Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 Webrev: http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ Thanks /Mikael From markus.gronlund at oracle.com Wed Mar 8 14:33:07 2017 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 8 Mar 2017 06:33:07 -0800 (PST) Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> Message-ID: <088bd929-4088-47df-b4fb-0b65416e9fe0@default> Hi Mikael, I think this looks good, thanks for addressing. Markus -----Original Message----- From: Mikael Gerdin Sent: den 8 mars 2017 15:17 To: hotspot-dev Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks Hi all, Please review this change to the lock rank of the locks taken in the G1 pre and post write barriers. The rank of these locks have long been a problem since even though they are leaf locks semantically they have been ranked as "nonleaf" locks in the lock rank system and this has caused several issues over the years where a thread holding a VM mutex and attempting to write an oop would in some rare cases hit a deadlock warning due to it acquiring one of the CBL monitors. Now this problem has come up yet again with the weak JNI handles bugfix where a lock rank assertion was hit yet again due to the fact that some code was holding a leaf lock while resolving a weak JNI handle. I suggest that the ranks of the involved locks are changed to "leaf - 1", allowing them to be acquired by threads holding "leaf" locks. This should not cause any further problems since reducing the ranks only allows the locks to be taken in more places. Both pairs of locks (the SATB and DirtyCardQ ones) have an associated FL_lock which protects a free list of buffers which is acquired while holding the respective CBL monitors but the FL_locks have the "special" lock rank and so they will still order correctly with the new "leaf - 1" ranks. There is some horrible stuff going on in locking_enqueue_completed_buffer but I've chosen to not change that at this point in time even though it relates to the relative ranks of each pair of locks. Testing: JPRT, HS tiers 2 and 3 were tested with a patch to acquire the locks even more often than what is normal to make sure that the code was tested properly. Some testing on HS tiers 4, 5 and 6 (Linux x64) JDK tiers 1, 2 and 3 Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 Webrev: http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ Thanks /Mikael From mikael.gerdin at oracle.com Wed Mar 8 15:09:40 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 8 Mar 2017 16:09:40 +0100 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <088bd929-4088-47df-b4fb-0b65416e9fe0@default> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> <088bd929-4088-47df-b4fb-0b65416e9fe0@default> Message-ID: <0aeef3ad-4b89-5d6a-fd07-92547ceab43e@oracle.com> Hi Markus, On 2017-03-08 15:33, Markus Gronlund wrote: > Hi Mikael, > > I think this looks good, thanks for addressing. Thanks for the review! /Mikael > > Markus > > -----Original Message----- > From: Mikael Gerdin > Sent: den 8 mars 2017 15:17 > To: hotspot-dev > Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks > > Hi all, > > Please review this change to the lock rank of the locks taken in the G1 pre and post write barriers. > > The rank of these locks have long been a problem since even though they are leaf locks semantically they have been ranked as "nonleaf" locks in the lock rank system and this has caused several issues over the years where a thread holding a VM mutex and attempting to write an oop would in some rare cases hit a deadlock warning due to it acquiring one of the CBL monitors. > > Now this problem has come up yet again with the weak JNI handles bugfix where a lock rank assertion was hit yet again due to the fact that some code was holding a leaf lock while resolving a weak JNI handle. > > I suggest that the ranks of the involved locks are changed to "leaf - 1", allowing them to be acquired by threads holding "leaf" locks. This should not cause any further problems since reducing the ranks only allows the locks to be taken in more places. Both pairs of locks (the SATB and DirtyCardQ ones) have an associated FL_lock which protects a free list of buffers which is acquired while holding the respective CBL monitors but the FL_locks have the "special" lock rank and so they will still order correctly with the new "leaf - 1" ranks. > > There is some horrible stuff going on in locking_enqueue_completed_buffer but I've chosen to not change that at this point in time even though it relates to the relative ranks of each pair of locks. > > Testing: > JPRT, HS tiers 2 and 3 were tested with a patch to acquire the locks even more often than what is normal to make sure that the code was tested properly. > Some testing on HS tiers 4, 5 and 6 (Linux x64) JDK tiers 1, 2 and 3 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 > Webrev: http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ > > Thanks > /Mikael > From shafi.s.ahmad at oracle.com Wed Mar 8 15:21:10 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 8 Mar 2017 07:21:10 -0800 (PST) Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> Message-ID: <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> Hi, May I get the review done for this. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Wednesday, March 01, 2017 4:27 PM > To: hotspot-dev at openjdk.java.net > Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method > PerfLongVariant::sample > > Hi, > > Summary: > It's a very small change to a single file. > > void PerfLongVariant::sample() { > > - assert(_sample_helper != NULL || _sampled != NULL, "unexpected > state"); > + // JJJ - This should not happen. Maybe the first sample is taken // > + while the _sample_helper is being null'ed out. > + // assert(_sample_helper != NULL || _sampled != NULL, "unexpected > + state"); if (_sample_helper == NULL) return; > > if (_sample_helper != NULL) { > *(jlong*)_valuep = _sample_helper->take_sample(); > } > else if (_sampled != NULL) { > *(jlong*)_valuep = *_sampled; > } > } > Above five lines are modified in changeset > http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.12 > Due to the addition of NULL check and return > if (_sample_helper == NULL) return; > the else-if block becomes redundant and statement 'if (_sample_helper != > NULL)' is not needed. > > Webrev link: http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ > Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 > > Testing: jprt. > > Thanks, > Shafi From coleen.phillimore at oracle.com Wed Mar 8 15:42:39 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 8 Mar 2017 10:42:39 -0500 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> Message-ID: <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> This doesn't look right. Don't you need to still check for _sampled != NULL? Can you remove the JJJ in the comment? I don't know what tests were run where Jon (aka JJJ in the code) found this, do you? Thanks, Coleen On 3/8/17 10:21 AM, Shafi Ahmad wrote: > Hi, > > May I get the review done for this. > > Regards, > Shafi > >> -----Original Message----- >> From: Shafi Ahmad >> Sent: Wednesday, March 01, 2017 4:27 PM >> To: hotspot-dev at openjdk.java.net >> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >> PerfLongVariant::sample >> >> Hi, >> >> Summary: >> It's a very small change to a single file. >> >> void PerfLongVariant::sample() { >> >> - assert(_sample_helper != NULL || _sampled != NULL, "unexpected >> state"); >> + // JJJ - This should not happen. Maybe the first sample is taken // >> + while the _sample_helper is being null'ed out. >> + // assert(_sample_helper != NULL || _sampled != NULL, "unexpected >> + state"); if (_sample_helper == NULL) return; >> >> if (_sample_helper != NULL) { >> *(jlong*)_valuep = _sample_helper->take_sample(); >> } >> else if (_sampled != NULL) { >> *(jlong*)_valuep = *_sampled; >> } >> } >> Above five lines are modified in changeset >> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.12 >> Due to the addition of NULL check and return >> if (_sample_helper == NULL) return; >> the else-if block becomes redundant and statement 'if (_sample_helper != >> NULL)' is not needed. >> >> Webrev link: http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >> >> Testing: jprt. >> >> Thanks, >> Shafi From coleen.phillimore at oracle.com Wed Mar 8 18:50:07 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 8 Mar 2017 13:50:07 -0500 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> Message-ID: <1af7231f-c017-2932-7979-28ac17efba66@oracle.com> This looks good. I filed https://bugs.openjdk.java.net/browse/JDK-8176393 Can you add some info to it? thanks, Coleen On 3/8/17 9:16 AM, Mikael Gerdin wrote: > Hi all, > > Please review this change to the lock rank of the locks taken in the > G1 pre and post write barriers. > > The rank of these locks have long been a problem since even though > they are leaf locks semantically they have been ranked as "nonleaf" > locks in the lock rank system and this has caused several issues over > the years where a thread holding a VM mutex and attempting to write an > oop would in some rare cases hit a deadlock warning due to it > acquiring one of the CBL monitors. > > Now this problem has come up yet again with the weak JNI handles > bugfix where a lock rank assertion was hit yet again due to the fact > that some code was holding a leaf lock while resolving a weak JNI handle. > > I suggest that the ranks of the involved locks are changed to "leaf - > 1", allowing them to be acquired by threads holding "leaf" locks. This > should not cause any further problems since reducing the ranks only > allows the locks to be taken in more places. Both pairs of locks (the > SATB and DirtyCardQ ones) have an associated FL_lock which protects a > free list of buffers which is acquired while holding the respective > CBL monitors but the FL_locks have the "special" lock rank and so they > will still order correctly with the new "leaf - 1" ranks. > > There is some horrible stuff going on in > locking_enqueue_completed_buffer but I've chosen to not change that at > this point in time even though it relates to the relative ranks of > each pair of locks. > > Testing: > JPRT, HS tiers 2 and 3 were tested with a patch to acquire the locks > even more often than what is normal to make sure that the code was > tested properly. > Some testing on HS tiers 4, 5 and 6 (Linux x64) > JDK tiers 1, 2 and 3 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 > Webrev: http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ > > Thanks > /Mikael From kim.barrett at oracle.com Wed Mar 8 22:55:52 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 8 Mar 2017 17:55:52 -0500 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> Message-ID: <73B19D8E-55B9-4C66-98F5-CEFE1A15A3D9@oracle.com> > On Mar 8, 2017, at 9:16 AM, Mikael Gerdin wrote: > > Hi all, > > Please review this change to the lock rank of the locks taken in the G1 pre and post write barriers. > > The rank of these locks have long been a problem since even though they are leaf locks semantically they have been ranked as "nonleaf" locks in the lock rank system and this has caused several issues over the years where a thread holding a VM mutex and attempting to write an oop would in some rare cases hit a deadlock warning due to it acquiring one of the CBL monitors. > > Now this problem has come up yet again with the weak JNI handles bugfix where a lock rank assertion was hit yet again due to the fact that some code was holding a leaf lock while resolving a weak JNI handle. > > I suggest that the ranks of the involved locks are changed to "leaf - 1", allowing them to be acquired by threads holding "leaf" locks. This should not cause any further problems since reducing the ranks only allows the locks to be taken in more places. Both pairs of locks (the SATB and DirtyCardQ ones) have an associated FL_lock which protects a free list of buffers which is acquired while holding the respective CBL monitors but the FL_locks have the "special" lock rank and so they will still order correctly with the new "leaf - 1" ranks. Not that it matters, since, as you say, the lock ranking orders are still okay, but I don?t recall any free list locks while holding the corresponding completed buffer lock. > There is some horrible stuff going on in locking_enqueue_completed_buffer but I've chosen to not change that at this point in time even though it relates to the relative ranks of each pair of locks. Thank you for not messing with that! I have some changes in that area that have been waiting for JDK 10; I plan to start putting those out for review soon. > > Testing: > JPRT, HS tiers 2 and 3 were tested with a patch to acquire the locks even more often than what is normal to make sure that the code was tested properly. > Some testing on HS tiers 4, 5 and 6 (Linux x64) > JDK tiers 1, 2 and 3 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 > Webrev: http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ Looks good. From zhuyong.me at gmail.com Thu Mar 9 01:25:30 2017 From: zhuyong.me at gmail.com (Zhu Yong) Date: Thu, 9 Mar 2017 09:25:30 +0800 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> <1d2586eb-0257-a330-2ff6-5028de510b4b@oracle.com> Message-ID: Here is my latest update with good news Yesterday pulled tips code from jdk9-dev and built with my current toolchain (no change), this time client and minimal VM finished the test code without any issue. The issue might relate to https://bugs.openjdk.java.net/browse/JDK-8175567, the final solution in http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/d6a0fe7ae0e8 is different from the very initial proposed solution (which I used in my previous build). On Tue, Mar 7, 2017 at 5:54 AM, Chris Plummer wrote: > I haven't been following this too closely, but one thing to consider is > that you are not getting proper floating point calling conventions. You > mentioned your configure target ABI is arm-vfp-sflt. This means you have > VFP support, but use the old softfloat calling conventions (FP values > passed in GPRs). Most platforms these days with VFP support use the new > hardfloat calling conventions (FP values passed in FPRs). So this could > simply be the case of building the wrong binary for your platform. > > - Use arm-vfp-hflt is you have VFP support and want to use hard float > calling > conventions. > - Use arm-vfp-sflt if you have VFP support and want to use the older soft > float > calling conventions. > - Use arm-sflt is you have a true soft float platform. > > There's also a very old arm sflt ABI that I don't think we support. The > main difference from arm-sflt is the word endianess of of 64-bit values. I > believe the presence of -mfpu=vfp is what is used to get the more modern > endianess, which to say the least is very non-obvious option for doing > this, but is the reason you will see -mfpu=vfp for all our supported arm > platforms (arm-vfp-hflt, arm-vfp-sflt, and arm-sflt). Here's the logic from > the configure script: > > if test "x$ARM_FLOAT_TYPE" = xvfp-sflt; then > ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=softfp -mfpu=vfp > -DFLOAT_ARCH=-vfp-sflt' > elif test "x$ARM_FLOAT_TYPE" = xvfp-hflt; then > ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=hard -mfpu=vfp > -DFLOAT_ARCH=-vfp-hflt' > elif test "x$ARM_FLOAT_TYPE" = xsflt; then > ARM_FLOAT_TYPE_FLAGS='-msoft-float -mfpu=vfp' > fi > > BTW, for the arm-sflt case, FLOAT_ARCH gets set internally to "-sflt". I'm > not sure why for arm-sflt it's handled in the source and the other two > platforms set it in the configure script. > > cheers, > > Chris > > > On 3/5/17 8:30 PM, Zhu Yong wrote: > >> Sometime later this week, I might get newer version of Marvell toolchain, >> I >> will do a build and test again then. >> I will update this thread when I get result. >> >> Thank you very much for helping. >> >> On Mon, Mar 6, 2017 at 9:09 AM, David Holmes >> wrote: >> >> On 3/03/2017 6:59 PM, Zhu Yong wrote: >>> >>> Thank you very much for helping. >>>> >>>> Today I tried to debug the test code, but without luck, still can't >>>> figure out why. >>>> >>>> First, I followed the instruction >>>> http://stackoverflow.com/questions/29916995/intellij-idea- >>>> remotely-debug-java-console-program, >>>> changed suspend=y because the test code is short, it will exit before >>>> debug start if not suspend first. Unfortunately, with remote debug, I >>>> was not able to reproduce the issue. Same class file, directly run, will >>>> produce the issue, remote debug won't. >>>> >>>> Then, I tried *jdb *from my OpenJDK9 client vm, run directly on my >>>> target system to debug the code. If I set breakpoint, program run >>>> without issue, however, if don't set breakpoint, just "run" in jdb, >>>> problem reproduced. >>>> >>>> This suggests to me that a floating-point register is not being >>> preserved >>> when needed. When you run with the breakpoint you will get full >>> save/restore of all the register state. Just a theory. >>> >>> Next, I tried to use *javap *to decompile the output class file, run >>> >>>> OpenJDK9 client vm javap and host side javap on same class file, the >>>> output is same for getTime function. >>>> >>>> It is a native code issue not a bytecode issue, so javap wont help here. >>> >>> One interesting point I noticed today is the output like below when >>> >>>> problem happen, 2nd call of getTime() has value of 1st call result / >>>> 1000. Looks like 2nd call getTime(), double variable q was not updated >>>> startTime: 284092.063000 endTime: 284.092063 runTime: -283807.970937 >>>> >>>> What should I do so that I can come a conclusion that it's due to >>>> compiler issue? >>>> >>>> Can you get access to a different gcc to build with? >>> >>> David >>> ----- >>> >>> >>> On Thu, Mar 2, 2017 at 12:32 PM, David Holmes >>> > wrote: >>>> >>>> I built your sample program with latest JDK 9 for arm-vfp-sflt and >>>> ran on an emulator I have available, and I had no issues. >>>> >>>> So I still have concerns this may be a build/compiler issue. >>>> >>>> David >>>> >>>> >>>> On 2/03/2017 12:51 PM, David Holmes wrote: >>>> >>>> I've moved this discussion over to hotspot-dev as this seems no >>>> longer a >>>> build issue but a C1 soft-float issue. >>>> >>>> David >>>> >>>> On 2/03/2017 12:35 PM, Zhu Yong wrote: >>>> >>>> If run with -Xint, it works. >>>> >>>> I have created simplified FP test by remove all non-related >>>> code from >>>> Whetstone test code. >>>> The test code is here: >>>> https://gist.github.com/yongz >>>> hy/d65c26d39fe5d549d1b406c7c84e >>>> acd4 >>>> >>> zhy/d65c26d39fe5d549d1b406c7c84 >>>> eacd4> >>>> >>>> I am not sure if the test code can help or not. The >>>> behaviour is weird : >>>> - If I print both itime and q, program will run OK >>>> - inside condition block if(q<1.0f), if I use exit code >>>> 2,3,4,5, problem >>>> appears, however, if I use exit code >=6, program run OK. >>>> >>>> I always get exit code 5, the line q = (double)itime is >>>> wrong? >>>> >>>> public static double getTime() >>>> { >>>> double q; >>>> long itime; >>>> itime = System.currentTimeMillis(); >>>> if(itime<1000000) { >>>> System.exit(1); >>>> } >>>> //System.out.printf("time long value %d\n", >>>> itime); >>>> q = (double) itime; >>>> if(q < 1.0f) { >>>> System.exit(5); // I always get exit code >>>> 5 >>>> } >>>> //System.out.printf("time float value %f\n", >>>> q); >>>> return q / 1000.0; >>>> } >>>> >>>> >>>> >>>> On Wed, Mar 1, 2017 at 5:31 PM, David Holmes >>>> >>>> >>> >> wrote: >>>> >>>> On 1/03/2017 6:45 PM, Zhu Yong wrote: >>>> >>>> OK, for the Whetstone Benchmark, I added debug >>>> printing and >>>> found it's >>>> due to function getTime(), it only get good value >>>> on >>>> 1st call, >>>> all >>>> following calls get 0s. >>>> Debug printing of itime value is correct. So reason >>>> should be >>>> the return >>>> division line. Could it because toolchain's >>>> floating >>>> point >>>> operation??? >>>> But why server VM Ok? >>>> >>>> >>>> Server and client implement FP logic differently, and >>>> particularly >>>> as this is soft-fp, they may well not be in sync. Does >>>> -Xint work? >>>> >>>> Can you isolate to a simple FP test? >>>> >>>> David >>>> >>>> public static double getTime() >>>> { >>>> double q; >>>> long itime; >>>> itime = System.currentTimeMillis(); >>>> q = (double) itime; >>>> return q / 1000.0; >>>> } >>>> >>>> On Wed, Mar 1, 2017 at 3:14 PM, David Holmes >>>> >>> >>>> >>> > >>>> >>> >>>> >>> >>> wrote: >>>> >>>> On 1/03/2017 4:26 PM, Zhu Yong wrote: >>>> >>>> We use armv7-marvell-linux-gnueabi-softfp >>>> toolchain >>>> >>>> gcc version 4.6.4 (Marvell GCC release >>>> 20150204-c4af733b 64K >>>> MAXPAGESIZE >>>> ALIGN CVE-2015-0235) >>>> >>>> Is this toolchain too old? I am asking >>>> because I saw >>>> other strange >>>> issues as well: >>>> >>>> >>>> We last used gcc 4.7.2. I can't really say if >>>> 4.6.4 is "too >>>> old". >>>> >>>> I have successfully build server, client >>>> and >>>> minimal VM, >>>> when I run >>>> Dhrystone benchmark file (the same jar file >>>> from >>>> http://www.okayan.jp/DhrystoneApplet/ >>>> >>>> >>> > >>>> >>> >>>> >>> >>), the >>>> performance >>>> from >>>> server VM >>>> is much better than client and minimal VM. >>>> (minimal: 1629852, client: 1615508, server: >>>> 2338871 ) >>>> >>>> >>>> That's expected. The server VM is high >>>> performance. >>>> >>>> And when run Whetstone Benchmark >>>> from >>>> >>>> http://www.roylongbottom.org.uk/online/whetjava2.html >>>> >>>> >>>> >>> > >>>> >>>> >>> >>>> >>>> >>> >>, >>>> server VM >>>> finished with good result, client and >>>> minimal VM not >>>> able to finish >>>> (program stuck there forever after print >>>> 1st >>>> line of >>>> output). >>>> >>>> >>>> That needs investigating. You need to try and >>>> generate a >>>> stackdump >>>> to see where things are stuck. >>>> >>>> David >>>> >>>> >>>> On Wed, Mar 1, 2017 at 1:56 PM, David >>>> Holmes >>>> >>> >>>> >>> > >>>> >>> david.holmes at oracle.com> >>>> >>> >> >>>> >>> >>>> >>> > >>>> >>> >>>> >>> >>>> wrote: >>>> >>>> On 1/03/2017 3:39 PM, Zhu Yong wrote: >>>> >>>> Thank you for the information, I >>>> managed to make >>>> minimal >>>> build >>>> pass now. >>>> >>>> Initially I though >>>> JVM_EXCLUDE_FILES >>>> need to add >>>> additional 3 >>>> generated >>>> files (they appeared >>>> in _BUILD_LIBJVM_objectfilenames. >>>> txt) >>>> : >>>> bytecodeInterpreterWithChecks.cpp >>>> jvmtiEnter.cpp >>>> jvmtiEnterTrace.cpp >>>> But build still fail with same >>>> error >>>> message. >>>> >>>> Finally I figured out it's due to >>>> those 8 doit() >>>> functions >>>> implementation in jvmtiEnvBase.hpp >>>> file. After >>>> move all >>>> those 8 >>>> doit() >>>> implementations to >>>> jvmtiEnvBase.cpp, >>>> build of >>>> minimal VM >>>> passed >>>> without >>>> any issue. >>>> >>>> >>>> That seems to be a compiler issue. >>>> jvmtiEnvBase.hpp is >>>> unconditionally included in a couple of >>>> places >>>> because we >>>> have to >>>> have enough of the JVMTI code to say >>>> JVMTI is not >>>> available. >>>> Those >>>> doit() implementations, though arguably >>>> could be >>>> conditional on >>>> INCLUDE_JVMTI, are not referenced by >>>> any >>>> called code >>>> and so >>>> should >>>> not linked in. But in your case it >>>> seems >>>> they are. >>>> >>>> What toolchain are you using? >>>> >>>> David >>>> ----- >>>> >>>> Changes appended >>>> ============= >>>> >>>> --- >>>> .../src/share/vm/prims/jvmtiE >>>> nvBase.cpp >>>> | 74 >>>> ++++++++++++++++++++++ >>>> .../src/share/vm/prims/jvmtiE >>>> nvBase.hpp >>>> | 68 >>>> +++----------------- >>>> 2 files changed, 82 >>>> insertions(+), 60 >>>> deletions(-) >>>> mode change 100644 => 100755 >>>> >>>> hotspot-9211c2e89c1c/src/share >>>> /vm/prims/jvmtiEnvBase.cpp >>>> mode change 100644 => 100755 >>>> >>>> hotspot-9211c2e89c1c/src/share >>>> /vm/prims/jvmtiEnvBase.hpp >>>> >>>> diff --git >>>> >>>> >>>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>>> >>>> >>>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>>> old mode 100644 >>>> new mode 100755 >>>> index dd241a0..e5832ba >>>> --- >>>> >>>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>>> +++ >>>> >>>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp >>>> @@ -1283,6 +1283,80 @@ >>>> >>>> VM_GetMultipleStackTraces::all >>>> ocate_and_fill_stacks(jint >>>> thread_count) { >>>> "the last copied frame >>>> info must be >>>> the last >>>> record"); >>>> } >>>> >>>> +void >>>> +VM_UpdateForPopTopFrame::doit() { >>>> + JavaThread* jt = >>>> _state->get_thread(); >>>> + if (Threads::includes(jt) && >>>> !jt->is_exiting() && >>>> jt->threadObj() != >>>> NULL) { >>>> + _state->update_for_pop_top_fra >>>> me(); >>>> + } else { >>>> + _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> + } >>>> +} >>>> + >>>> +void >>>> +VM_SetFramePop::doit() { >>>> + JavaThread* jt = >>>> _state->get_thread(); >>>> + if (Threads::includes(jt) && >>>> !jt->is_exiting() && >>>> jt->threadObj() != >>>> NULL) { >>>> + int frame_number = >>>> _state->count_frames() - >>>> _depth; >>>> + >>>> >>>> >>>> >>>> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ >>>> pop(frame_number); >>>> >>>> + } else { >>>> + _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> + } >>>> +} >>>> + >>>> +void >>>> +VM_GetOwnedMonitorInfo::doit() { >>>> + _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> + if >>>> (Threads::includes(_java_thread) && >>>> !_java_thread->is_exiting() >>>> + >>>> && >>>> _java_thread->threadObj() != >>>> NULL) { >>>> + _result = ((JvmtiEnvBase >>>> >>>> *)_env)->get_owned_monitors(_calling_thread, >>>> _java_thread, >>>> + >>>> _owned_monitors_list); >>>> + } >>>> +} >>>> + >>>> +void >>>> +VM_GetObjectMonitorUsage::doit() >>>> { >>>> + _result = ((JvmtiEnvBase*) >>>> >>>> _env)->get_object_monitor_usage(_calling_thread, >>>> _object, >>>> _info_ptr); >>>> +} >>>> + >>>> +void >>>> +VM_GetCurrentContendedMonito >>>> r::doit() >>>> { >>>> + _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> + if >>>> (Threads::includes(_java_thread) && >>>> !_java_thread->is_exiting() && >>>> + _java_thread->threadObj() != >>>> NULL) { >>>> + _result = ((JvmtiEnvBase >>>> >>>> >>>> >>>> *)_env)->get_current_contended_monitor(_calling_thread,_ >>>> java_thread,_owned_monitor_ptr); >>>> >>>> + } >>>> +} >>>> + >>>> +void >>>> +VM_GetStackTrace::doit() { >>>> + _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> + if >>>> (Threads::includes(_java_thread) && >>>> !_java_thread->is_exiting() >>>> + >>>> && >>>> _java_thread->threadObj() != >>>> NULL) { >>>> + _result = ((JvmtiEnvBase >>>> *)_env)->get_stack_trace(_java_thread, >>>> + >>>> _start_depth, >>>> _max_count, >>>> + >>>> _frame_buffer, >>>> _count_ptr); >>>> + } >>>> +} >>>> + >>>> +void >>>> +VM_GetFrameCount::doit() { >>>> + _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> + JavaThread* jt = >>>> _state->get_thread(); >>>> + if (Threads::includes(jt) && >>>> !jt->is_exiting() && >>>> jt->threadObj() != >>>> NULL) { >>>> + _result = >>>> ((JvmtiEnvBase*)_env)->get_fra >>>> me_count(_state, >>>> _count_ptr); >>>> + } >>>> +} >>>> + >>>> +void >>>> +VM_GetFrameLocation::doit() { >>>> + _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> + if >>>> (Threads::includes(_java_thread) && >>>> !_java_thread->is_exiting() && >>>> + _java_thread->threadObj() != >>>> NULL) { >>>> + _result = >>>> >>>> ((JvmtiEnvBase*)_env)->get_fra >>>> me_location(_java_thread, >>>> _depth, >>>> + >>>> _method_ptr, >>>> _location_ptr); >>>> + } >>>> +} >>>> >>>> void >>>> VM_GetThreadListStackTraces::d >>>> oit() >>>> { >>>> diff --git >>>> >>>> >>>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>>> >>>> >>>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>>> old mode 100644 >>>> new mode 100755 >>>> index 04e6869..00b9890 >>>> --- >>>> >>>> a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>>> +++ >>>> >>>> b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp >>>> @@ -359,14 +359,7 @@ public: >>>> } >>>> VMOp_Type type() const { return >>>> VMOp_UpdateForPopTopFrame; } >>>> jvmtiError result() { return >>>> _result; } >>>> - void doit() { >>>> - JavaThread* jt = >>>> _state->get_thread(); >>>> - if (Threads::includes(jt) && >>>> !jt->is_exiting() && >>>> jt->threadObj() >>>> != NULL) { >>>> - >>>> _state->update_for_pop_top_frame(); >>>> - } else { >>>> - _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> - } >>>> - } >>>> + void doit(); >>>> }; >>>> >>>> // VM operation to set frame pop. >>>> @@ -389,15 +382,7 @@ public: >>>> bool >>>> allow_nested_vm_operations() >>>> const { >>>> return true; } >>>> VMOp_Type type() const { return >>>> VMOp_SetFramePop; } >>>> jvmtiError result() { return >>>> _result; } >>>> - void doit() { >>>> - JavaThread* jt = >>>> _state->get_thread(); >>>> - if (Threads::includes(jt) && >>>> !jt->is_exiting() && >>>> jt->threadObj() >>>> != NULL) { >>>> - int frame_number = >>>> _state->count_frames() >>>> - _depth; >>>> - >>>> >>>> >>>> >>>> _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ >>>> pop(frame_number); >>>> >>>> - } else { >>>> - _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> - } >>>> - } >>>> + void doit(); >>>> }; >>>> >>>> >>>> @@ -421,14 +406,7 @@ public: >>>> _result = JVMTI_ERROR_NONE; >>>> } >>>> VMOp_Type type() const { return >>>> VMOp_GetOwnedMonitorInfo; } >>>> - void doit() { >>>> - _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> - if >>>> (Threads::includes(_java_thread) && >>>> !_java_thread->is_exiting() >>>> - >>>> && >>>> _java_thread->threadObj() != >>>> NULL) { >>>> - _result = ((JvmtiEnvBase >>>> >>>> *)_env)->get_owned_monitors(_calling_thread, >>>> _java_thread, >>>> - >>>> _owned_monitors_list); >>>> - } >>>> - } >>>> + void doit(); >>>> jvmtiError result() { return >>>> _result; } >>>> }; >>>> >>>> @@ -451,10 +429,7 @@ public: >>>> } >>>> VMOp_Type type() const { return >>>> VMOp_GetObjectMonitorUsage; } >>>> jvmtiError result() { return >>>> _result; } >>>> - void doit() { >>>> - _result = ((JvmtiEnvBase*) >>>> >>>> _env)->get_object_monitor_usage(_calling_thread, >>>> _object, >>>> _info_ptr); >>>> - } >>>> - >>>> + void doit(); >>>> }; >>>> >>>> // VM operation to get current >>>> contended >>>> monitor. >>>> @@ -475,13 +450,7 @@ public: >>>> } >>>> VMOp_Type type() const { return >>>> VMOp_GetCurrentContendedMonitor; } >>>> jvmtiError result() { return >>>> _result; } >>>> - void doit() { >>>> - _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> - if >>>> (Threads::includes(_java_thread) && >>>> !_java_thread->is_exiting() && >>>> - _java_thread->threadObj() >>>> != NULL) { >>>> - _result = ((JvmtiEnvBase >>>> >>>> >>>> >>>> *)_env)->get_current_contended_monitor(_calling_thread,_ >>>> java_thread,_owned_monitor_ptr); >>>> >>>> - } >>>> - } >>>> + void doit(); >>>> }; >>>> >>>> // VM operation to get stack trace >>>> at safepoint. >>>> @@ -508,15 +477,7 @@ public: >>>> } >>>> jvmtiError result() { return >>>> _result; } >>>> VMOp_Type type() const { return >>>> VMOp_GetStackTrace; } >>>> - void doit() { >>>> - _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> - if >>>> (Threads::includes(_java_thread) && >>>> !_java_thread->is_exiting() >>>> - >>>> && >>>> _java_thread->threadObj() != >>>> NULL) { >>>> - _result = ((JvmtiEnvBase >>>> *)_env)->get_stack_trace(_java >>>> _thread, >>>> - >>>> _start_depth, >>>> _max_count, >>>> - >>>> _frame_buffer, >>>> _count_ptr); >>>> - } >>>> - } >>>> + void doit(); >>>> }; >>>> >>>> // forward declaration >>>> @@ -606,13 +567,7 @@ public: >>>> } >>>> VMOp_Type type() const { return >>>> VMOp_GetFrameCount; } >>>> jvmtiError result() { return >>>> _result; } >>>> - void doit() { >>>> - _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> - JavaThread* jt = >>>> _state->get_thread(); >>>> - if (Threads::includes(jt) && >>>> !jt->is_exiting() && >>>> jt->threadObj() >>>> != NULL) { >>>> - _result = >>>> ((JvmtiEnvBase*)_env)->get_fra >>>> me_count(_state, >>>> _count_ptr); >>>> - } >>>> - } >>>> + void doit(); >>>> }; >>>> >>>> // VM operation to frame location >>>> at safepoint. >>>> @@ -636,14 +591,7 @@ public: >>>> } >>>> VMOp_Type type() const { return >>>> VMOp_GetFrameLocation; } >>>> jvmtiError result() { return >>>> _result; } >>>> - void doit() { >>>> - _result = >>>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>>> - if >>>> (Threads::includes(_java_thread) && >>>> !_java_thread->is_exiting() && >>>> - _java_thread->threadObj() >>>> != NULL) { >>>> - _result = >>>> >>>> ((JvmtiEnvBase*)_env)->get_fra >>>> me_location(_java_thread, >>>> _depth, >>>> - >>>> _method_ptr, >>>> _location_ptr); >>>> - } >>>> - } >>>> + void doit(); >>>> }; >>>> >>>> >>>> -- >>>> 2.1.4 >>>> >>>> >>>> On Tue, Feb 28, 2017 at 6:11 PM, >>>> David Holmes >>>> >>> >>>> >>> > >>>> >>> >>>> >>> >> >>>> >>> >>>> >>> > >>>> >>> >>>> >>> >>> >>>> >>> >>>> >>> > >>>> >>> >>>> >>> >> >>>> >>>> >>> >>>> >>> > >>>> >>> >>>> >>> >>>>> wrote: >>>> >>>> On 28/02/2017 7:35 PM, David >>>> Holmes wrote: >>>> >>>> On 28/02/2017 6:51 PM, Zhu >>>> Yong wrote: >>>> >>>> Dear All, >>>> >>>> I am testing cross >>>> compile >>>> OpenJDK-9+158 for our >>>> embedded >>>> system using >>>> buildroot, I can build >>>> server and >>>> client >>>> variants >>>> successfully, but the >>>> output compact1 profile >>>> file size >>>> is too >>>> big, then I >>>> tried >>>> to build >>>> minimal >>>> variant, failed when >>>> linking >>>> libjvm.so. >>>> >>>> I checked >>>> >>>> >>>> >>>> >>>> build/linux-arm-normal-minimal-release/hotspot/variant- >>>> minimal/ >>>> >>>> libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt, >>>> jvmtiEnvBase.o >>>> and >>>> jvmtiEnvThreadState.o are not >>>> listed in >>>> the file >>>> (which >>>> is required >>>> from the error message >>>> below). >>>> >>>> >>>> Right - JVM TI is not part >>>> of the >>>> Minimal VM. At the >>>> moment it looks >>>> like some service has >>>> either >>>> been >>>> enabled when >>>> it should not >>>> have been, >>>> or has not been correctly >>>> if'def in the >>>> source. >>>> >>>> >>>> As far as I can see your error >>>> messages are >>>> complaining about >>>> missing functions that are >>>> called from the >>>> same >>>> sources as the >>>> functions that are missing! ie. >>>> >>>> undefined reference to >>>> >>>> >>>> `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, >>>> JavaThread*, >>>> _jobject**)' >>>> /tmp/cc27HS5M.ltrans0.ltrans. >>>> o: >>>> In function >>>> `VM_GetOwnedMonitorInfo:: >>>> doit() >>>> >>>> but VM_GetOwnedMonitorInfo is >>>> defined in >>>> jvmtiEnv.cpp which >>>> should >>>> be included or excluded the >>>> same >>>> as >>>> jvmtiEnBase.cpp. >>>> Here's the >>>> logic in the makefile that >>>> controls this: >>>> >>>> ifneq ($(call >>>> check-jvm-feature, >>>> jvmti), >>> >>> From david.holmes at oracle.com Thu Mar 9 01:49:28 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 9 Mar 2017 11:49:28 +1000 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> <1d2586eb-0257-a330-2ff6-5028de510b4b@oracle.com> Message-ID: <289fc22c-3faa-26f8-5cce-affd3b184998@oracle.com> On 9/03/2017 11:25 AM, Zhu Yong wrote: > Here is my latest update with good news > > Yesterday pulled tips code from jdk9-dev and built with my current > toolchain (no change), this time client and minimal VM finished the test > code without any issue. > > The issue might relate to > https://bugs.openjdk.java.net/browse/JDK-8175567, the final solution > in http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/d6a0fe7ae0e8 is > different from the very initial proposed solution (which I used in my > previous build). Easy enough to test that theory. :) Glad things are now working. David ----- > On Tue, Mar 7, 2017 at 5:54 AM, Chris Plummer > wrote: > > I haven't been following this too closely, but one thing to consider > is that you are not getting proper floating point calling > conventions. You mentioned your configure target ABI is > arm-vfp-sflt. This means you have VFP support, but use the old > softfloat calling conventions (FP values passed in GPRs). Most > platforms these days with VFP support use the new hardfloat calling > conventions (FP values passed in FPRs). So this could simply be the > case of building the wrong binary for your platform. > > - Use arm-vfp-hflt is you have VFP support and want to use hard > float calling > conventions. > - Use arm-vfp-sflt if you have VFP support and want to use the > older soft float > calling conventions. > - Use arm-sflt is you have a true soft float platform. > > There's also a very old arm sflt ABI that I don't think we support. > The main difference from arm-sflt is the word endianess of of 64-bit > values. I believe the presence of -mfpu=vfp is what is used to get > the more modern endianess, which to say the least is very > non-obvious option for doing this, but is the reason you will see > -mfpu=vfp for all our supported arm platforms (arm-vfp-hflt, > arm-vfp-sflt, and arm-sflt). Here's the logic from the configure script: > > if test "x$ARM_FLOAT_TYPE" = xvfp-sflt; then > ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=softfp -mfpu=vfp > -DFLOAT_ARCH=-vfp-sflt' > elif test "x$ARM_FLOAT_TYPE" = xvfp-hflt; then > ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=hard -mfpu=vfp > -DFLOAT_ARCH=-vfp-hflt' > elif test "x$ARM_FLOAT_TYPE" = xsflt; then > ARM_FLOAT_TYPE_FLAGS='-msoft-float -mfpu=vfp' > fi > > BTW, for the arm-sflt case, FLOAT_ARCH gets set internally to > "-sflt". I'm not sure why for arm-sflt it's handled in the source > and the other two platforms set it in the configure script. > > cheers, > > Chris > > > On 3/5/17 8:30 PM, Zhu Yong wrote: > > Sometime later this week, I might get newer version of Marvell > toolchain, I > will do a build and test again then. > I will update this thread when I get result. > > Thank you very much for helping. > > On Mon, Mar 6, 2017 at 9:09 AM, David Holmes > > > wrote: > > On 3/03/2017 6:59 PM, Zhu Yong wrote: > > Thank you very much for helping. > > Today I tried to debug the test code, but without luck, > still can't > figure out why. > > First, I followed the instruction > http://stackoverflow.com/questions/29916995/intellij-idea- > > remotely-debug-java-console-program, > changed suspend=y because the test code is short, it > will exit before > debug start if not suspend first. Unfortunately, with > remote debug, I > was not able to reproduce the issue. Same class file, > directly run, will > produce the issue, remote debug won't. > > Then, I tried *jdb *from my OpenJDK9 client vm, run > directly on my > target system to debug the code. If I set breakpoint, > program run > without issue, however, if don't set breakpoint, just > "run" in jdb, > problem reproduced. > > This suggests to me that a floating-point register is not > being preserved > when needed. When you run with the breakpoint you will get full > save/restore of all the register state. Just a theory. > > Next, I tried to use *javap *to decompile the output class > file, run > > OpenJDK9 client vm javap and host side javap on same > class file, the > output is same for getTime function. > > It is a native code issue not a bytecode issue, so javap > wont help here. > > One interesting point I noticed today is the output like > below when > > problem happen, 2nd call of getTime() has value of 1st > call result / > 1000. Looks like 2nd call getTime(), double variable q > was not updated > startTime: 284092.063000 endTime: 284.092063 runTime: > -283807.970937 > > What should I do so that I can come a conclusion that > it's due to > compiler issue? > > Can you get access to a different gcc to build with? > > David > ----- > > > On Thu, Mar 2, 2017 at 12:32 PM, David Holmes > > >> wrote: > > I built your sample program with latest JDK 9 for > arm-vfp-sflt and > ran on an emulator I have available, and I had no > issues. > > So I still have concerns this may be a > build/compiler issue. > > David > > > On 2/03/2017 12:51 PM, David Holmes wrote: > > I've moved this discussion over to hotspot-dev > as this seems no > longer a > build issue but a C1 soft-float issue. > > David > > On 2/03/2017 12:35 PM, Zhu Yong wrote: > > If run with -Xint, it works. > > I have created simplified FP test by remove > all non-related > code from > Whetstone test code. > The test code is here: > > https://gist.github.com/yongzhy/d65c26d39fe5d549d1b406c7c84e > > acd4 > > > eacd4> > > I am not sure if the test code can help or > not. The > behaviour is weird : > - If I print both itime and q, program will > run OK > - inside condition block if(q<1.0f), if I > use exit code > 2,3,4,5, problem > appears, however, if I use exit code >=6, > program run OK. > > I always get exit code 5, the line q = > (double)itime is wrong? > > public static double getTime() > { > double q; > long itime; > itime = System.currentTimeMillis(); > if(itime<1000000) { > System.exit(1); > } > //System.out.printf("time long > value %d\n", > itime); > q = (double) itime; > if(q < 1.0f) { > System.exit(5); // I > always get exit code 5 > } > //System.out.printf("time float > value %f\n", q); > return q / 1000.0; > } > > > > On Wed, Mar 1, 2017 at 5:31 PM, David Holmes > > > > > >>> wrote: > > On 1/03/2017 6:45 PM, Zhu Yong wrote: > > OK, for the Whetstone Benchmark, I > added debug > printing and > found it's > due to function getTime(), it only > get good value on > 1st call, > all > following calls get 0s. > Debug printing of itime value is > correct. So reason > should be > the return > division line. Could it because > toolchain's floating > point > operation??? > But why server VM Ok? > > > Server and client implement FP logic > differently, and > particularly > as this is soft-fp, they may well not > be in sync. Does > -Xint work? > > Can you isolate to a simple FP test? > > David > > public static double getTime() > { > double q; > long itime; > itime = > System.currentTimeMillis(); > q = (double) itime; > return q / 1000.0; > } > > On Wed, Mar 1, 2017 at 3:14 PM, > David Holmes > > > > > >> > > > > > >>>> wrote: > > On 1/03/2017 4:26 PM, Zhu Yong > wrote: > > We use > armv7-marvell-linux-gnueabi-softfp > toolchain > > gcc version 4.6.4 (Marvell > GCC release > 20150204-c4af733b 64K > MAXPAGESIZE > ALIGN CVE-2015-0235) > > Is this toolchain too old? > I am asking > because I saw > other strange > issues as well: > > > We last used gcc 4.7.2. I can't > really say if > 4.6.4 is "too > old". > > I have successfully build > server, client and > minimal VM, > when I run > Dhrystone benchmark file > (the same jar file > from > > http://www.okayan.jp/DhrystoneApplet/ > > > > > > >> > > > > > > > >>>), the performance > from > server VM > is much better than client > and minimal VM. > (minimal: 1629852, client: > 1615508, server: > 2338871 ) > > > That's expected. The server VM > is high > performance. > > And when run Whetstone > Benchmark > from > > > http://www.roylongbottom.org.uk/online/whetjava2.html > > > > > > > > > >> > > > > > > > > > > > >>>, > server VM > finished with good result, > client and > minimal VM not > able to finish > (program stuck there > forever after print 1st > line of > output). > > > That needs investigating. You > need to try and > generate a > stackdump > to see where things are stuck. > > David > > > On Wed, Mar 1, 2017 at 1:56 > PM, David Holmes > > > > > >> > david.holmes at oracle.com > > > >>> > > > > > > >> > > > > > > >>>>> wrote: > > On 1/03/2017 3:39 PM, > Zhu Yong wrote: > > Thank you for the > information, I > managed to make > minimal > build > pass now. > > Initially I though > JVM_EXCLUDE_FILES > need to add > additional 3 > generated > files (they appeared > in > _BUILD_LIBJVM_objectfilenames.txt) > : > > bytecodeInterpreterWithChecks.cpp > jvmtiEnter.cpp > jvmtiEnterTrace.cpp > But build still > fail with same error > message. > > Finally I figured > out it's due to > those 8 doit() > functions > implementation in > jvmtiEnvBase.hpp > file. After > move all > those 8 > doit() > implementations to > jvmtiEnvBase.cpp, > build of > minimal VM > passed > without > any issue. > > > That seems to be a > compiler issue. > jvmtiEnvBase.hpp is > unconditionally > included in a couple of > places > because we > have to > have enough of the > JVMTI code to say > JVMTI is not > available. > Those > doit() implementations, > though arguably > could be > conditional on > INCLUDE_JVMTI, are not > referenced by any > called code > and so > should > not linked in. But in > your case it seems > they are. > > What toolchain are you > using? > > David > ----- > > Changes appended > ============= > > --- > > .../src/share/vm/prims/jvmtiE > nvBase.cpp > | 74 > ++++++++++++++++++++++ > > .../src/share/vm/prims/jvmtiE > nvBase.hpp > | 68 > +++----------------- > 2 files changed, > 82 insertions(+), 60 > deletions(-) > mode change 100644 > => 100755 > > hotspot-9211c2e89c1c/src/share > /vm/prims/jvmtiEnvBase.cpp > mode change 100644 > => 100755 > > hotspot-9211c2e89c1c/src/share > /vm/prims/jvmtiEnvBase.hpp > > diff --git > > > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > > > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > old mode 100644 > new mode 100755 > index dd241a0..e5832ba > --- > > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > +++ > > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.cpp > @@ -1283,6 +1283,80 @@ > > VM_GetMultipleStackTraces::all > ocate_and_fill_stacks(jint > thread_count) { > "the last > copied frame > info must be > the last > record"); > } > > +void > > +VM_UpdateForPopTopFrame::doit() { > + JavaThread* jt = > _state->get_thread(); > + if > (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() != > NULL) { > + > _state->update_for_pop_top_fra > me(); > + } else { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + } > +} > + > +void > > +VM_SetFramePop::doit() { > + JavaThread* jt = > _state->get_thread(); > + if > (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() != > NULL) { > + int frame_number = > _state->count_frames() - > _depth; > + > > > > > _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ > pop(frame_number); > > + } else { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + } > +} > + > +void > > +VM_GetOwnedMonitorInfo::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + if > (Threads::includes(_java_thread) && > > !_java_thread->is_exiting() > + > && > > _java_thread->threadObj() != > NULL) { > + _result = > ((JvmtiEnvBase > > *)_env)->get_owned_monitors(_calling_thread, > _java_thread, > + > _owned_monitors_list); > + } > +} > + > +void > > +VM_GetObjectMonitorUsage::doit() { > + _result = > ((JvmtiEnvBase*) > > > _env)->get_object_monitor_usage(_calling_thread, > _object, > _info_ptr); > +} > + > +void > > +VM_GetCurrentContendedMonitor::doit() > { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + if > (Threads::includes(_java_thread) && > > !_java_thread->is_exiting() && > + > _java_thread->threadObj() != > NULL) { > + _result = > ((JvmtiEnvBase > > > > > *)_env)->get_current_contended_monitor(_calling_thread,_ > java_thread,_owned_monitor_ptr); > > + } > +} > + > +void > > +VM_GetStackTrace::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + if > (Threads::includes(_java_thread) && > > !_java_thread->is_exiting() > + > && > > _java_thread->threadObj() != > NULL) { > + _result = > ((JvmtiEnvBase > > *)_env)->get_stack_trace(_java_thread, > + > _start_depth, > _max_count, > + > _frame_buffer, > _count_ptr); > + } > +} > + > +void > > +VM_GetFrameCount::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + JavaThread* jt = > _state->get_thread(); > + if > (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() != > NULL) { > + _result = > ((JvmtiEnvBase*)_env)->get_fra > me_count(_state, > _count_ptr); > + } > +} > + > +void > > +VM_GetFrameLocation::doit() { > + _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > + if > (Threads::includes(_java_thread) && > > !_java_thread->is_exiting() && > + > _java_thread->threadObj() != > NULL) { > + _result = > > ((JvmtiEnvBase*)_env)->get_fra > me_location(_java_thread, > _depth, > + > _method_ptr, > _location_ptr); > + } > +} > > void > > VM_GetThreadListStackTraces::doit() > { > diff --git > > > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > > > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > old mode 100644 > new mode 100755 > index 04e6869..00b9890 > --- > > > a/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > +++ > > > b/hotspot-9211c2e89c1c/src/share/vm/prims/jvmtiEnvBase.hpp > @@ -359,14 +359,7 > @@ public: > } > VMOp_Type type() > const { return > VMOp_UpdateForPopTopFrame; } > jvmtiError > result() { return > _result; } > - void doit() { > - JavaThread* jt = > _state->get_thread(); > - if > (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - > _state->update_for_pop_top_frame(); > - } else { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - } > - } > + void doit(); > }; > > // VM operation to > set frame pop. > @@ -389,15 +382,7 > @@ public: > bool > allow_nested_vm_operations() > const { > return true; } > VMOp_Type type() > const { return > VMOp_SetFramePop; } > jvmtiError > result() { return > _result; } > - void doit() { > - JavaThread* jt = > _state->get_thread(); > - if > (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - int > frame_number = > _state->count_frames() > - _depth; > - > > > > > _state->env_thread_state((JvmtiEnvBase*)_env)->set_frame_ > pop(frame_number); > > - } else { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - } > - } > + void doit(); > }; > > > @@ -421,14 +406,7 > @@ public: > _result = > JVMTI_ERROR_NONE; > } > VMOp_Type type() > const { return > VMOp_GetOwnedMonitorInfo; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - if > (Threads::includes(_java_thread) && > > !_java_thread->is_exiting() > - > && > > _java_thread->threadObj() != > NULL) { > - _result = > ((JvmtiEnvBase > > *)_env)->get_owned_monitors(_calling_thread, > _java_thread, > - > _owned_monitors_list); > - } > - } > + void doit(); > jvmtiError > result() { return > _result; } > }; > > @@ -451,10 +429,7 > @@ public: > } > VMOp_Type type() > const { return > VMOp_GetObjectMonitorUsage; } > jvmtiError > result() { return > _result; } > - void doit() { > - _result = > ((JvmtiEnvBase*) > > > _env)->get_object_monitor_usage(_calling_thread, > _object, > _info_ptr); > - } > - > + void doit(); > }; > > // VM operation to > get current > contended > monitor. > @@ -475,13 +450,7 > @@ public: > } > VMOp_Type type() > const { return > > VMOp_GetCurrentContendedMonitor; } > jvmtiError > result() { return > _result; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - if > (Threads::includes(_java_thread) && > > !_java_thread->is_exiting() && > - > _java_thread->threadObj() > != NULL) { > - _result = > ((JvmtiEnvBase > > > > > *)_env)->get_current_contended_monitor(_calling_thread,_ > java_thread,_owned_monitor_ptr); > > - } > - } > + void doit(); > }; > > // VM operation to > get stack trace > at safepoint. > @@ -508,15 +477,7 > @@ public: > } > jvmtiError > result() { return > _result; } > VMOp_Type type() > const { return > VMOp_GetStackTrace; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - if > (Threads::includes(_java_thread) && > > !_java_thread->is_exiting() > - > && > > _java_thread->threadObj() != > NULL) { > - _result = > ((JvmtiEnvBase > > *)_env)->get_stack_trace(_java > _thread, > - > _start_depth, > _max_count, > - > _frame_buffer, > _count_ptr); > - } > - } > + void doit(); > }; > > // forward declaration > @@ -606,13 +567,7 > @@ public: > } > VMOp_Type type() > const { return > VMOp_GetFrameCount; } > jvmtiError > result() { return > _result; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - JavaThread* jt = > _state->get_thread(); > - if > (Threads::includes(jt) && > !jt->is_exiting() && > jt->threadObj() > != NULL) { > - _result = > ((JvmtiEnvBase*)_env)->get_fra > me_count(_state, > _count_ptr); > - } > - } > + void doit(); > }; > > // VM operation to > frame location > at safepoint. > @@ -636,14 +591,7 > @@ public: > } > VMOp_Type type() > const { return > VMOp_GetFrameLocation; } > jvmtiError > result() { return > _result; } > - void doit() { > - _result = > JVMTI_ERROR_THREAD_NOT_ALIVE; > - if > (Threads::includes(_java_thread) && > > !_java_thread->is_exiting() && > - > _java_thread->threadObj() > != NULL) { > - _result = > > ((JvmtiEnvBase*)_env)->get_fra > me_location(_java_thread, > _depth, > - > _method_ptr, > _location_ptr); > - } > - } > + void doit(); > }; > > > -- > 2.1.4 > > > On Tue, Feb 28, > 2017 at 6:11 PM, > David Holmes > > > > > > >> > > > > > > >>> > > > > > >> > > > > > > >>>> > > > > > > >> > > > > > > >>> > > > > > > > >> > > > > > > >>>>>> wrote: > > On 28/02/2017 > 7:35 PM, David > Holmes wrote: > > On > 28/02/2017 6:51 PM, Zhu > Yong wrote: > > Dear All, > > I am > testing cross compile > OpenJDK-9+158 for our > embedded > system > using > > buildroot, I can build > server and > client > variants > > successfully, but the > output > compact1 profile > file size > is too > big, then I > tried > to build > minimal > > variant, failed when > linking > libjvm.so. > > I checked > > > > > > build/linux-arm-normal-minimal-release/hotspot/variant- > minimal/ > > > libjvm/objs/_BUILD_LIBJVM_objectfilenames.txt, > jvmtiEnvBase.o > and > jvmtiEnvThreadState.o are not > listed in > the file > (which > is required > from > the error message > below). > > > Right - JVM > TI is not part > of the > Minimal VM. At the > moment it looks > like some > service has either > been > enabled when > it should not > have been, > or has not > been correctly > if'def in the > source. > > > As far as I can > see your error > messages are > complaining about > missing > functions that are > called from the > same > sources as the > functions that > are missing! ie. > > undefined > reference to > > > > `JvmtiEnvBase::get_current_contended_monitor(JavaThread*, > JavaThread*, > _jobject**)' > > /tmp/cc27HS5M.ltrans0.ltrans.o: > In function > > `VM_GetOwnedMonitorInfo::doit() > > but > VM_GetOwnedMonitorInfo is > defined in > jvmtiEnv.cpp which > should > be included or > excluded the same > as > jvmtiEnBase.cpp. > Here's the > logic in the > makefile that > controls this: > > ifneq ($(call > check-jvm-feature, > jvmti), > > From david.holmes at oracle.com Thu Mar 9 02:05:14 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 9 Mar 2017 12:05:14 +1000 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> Message-ID: <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> Hi Coleen, On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: > > This doesn't look right. > > Don't you need to still check for _sampled != NULL? That use of _sampled was in dead code that is now removed. if (_sample_helper == NULL) return; - if (_sample_helper != NULL) { *(jlong*)_valuep = _sample_helper->take_sample(); - } - else if (_sampled != NULL) { - *(jlong*)_valuep = *_sampled; - } } This change looks fine to me. > Can you remove the JJJ in the comment? I second that :) Thanks, David ----- > I don't know what tests were run > where Jon (aka JJJ in the code) found this, do you? > > Thanks, > Coleen > > > On 3/8/17 10:21 AM, Shafi Ahmad wrote: >> Hi, >> >> May I get the review done for this. >> >> Regards, >> Shafi >> >>> -----Original Message----- >>> From: Shafi Ahmad >>> Sent: Wednesday, March 01, 2017 4:27 PM >>> To: hotspot-dev at openjdk.java.net >>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >>> PerfLongVariant::sample >>> >>> Hi, >>> >>> Summary: >>> It's a very small change to a single file. >>> >>> void PerfLongVariant::sample() { >>> >>> - assert(_sample_helper != NULL || _sampled != NULL, "unexpected >>> state"); >>> + // JJJ - This should not happen. Maybe the first sample is taken // >>> + while the _sample_helper is being null'ed out. >>> + // assert(_sample_helper != NULL || _sampled != NULL, "unexpected >>> + state"); if (_sample_helper == NULL) return; >>> >>> if (_sample_helper != NULL) { >>> *(jlong*)_valuep = _sample_helper->take_sample(); >>> } >>> else if (_sampled != NULL) { >>> *(jlong*)_valuep = *_sampled; >>> } >>> } >>> Above five lines are modified in changeset >>> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.12 >>> Due to the addition of NULL check and return >>> if (_sample_helper == NULL) return; >>> the else-if block becomes redundant and statement 'if (_sample_helper != >>> NULL)' is not needed. >>> >>> Webrev link: http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >>> >>> Testing: jprt. >>> >>> Thanks, >>> Shafi > From david.holmes at oracle.com Thu Mar 9 02:16:45 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 9 Mar 2017 12:16:45 +1000 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> Message-ID: <9b8e33eb-e961-db9d-d0c4-e322799d6a18@oracle.com> Hi Mikael, On 9/03/2017 12:16 AM, Mikael Gerdin wrote: > Hi all, > > Please review this change to the lock rank of the locks taken in the G1 > pre and post write barriers. > > The rank of these locks have long been a problem since even though they > are leaf locks semantically they have been ranked as "nonleaf" locks in > the lock rank system and this has caused several issues over the years > where a thread holding a VM mutex and attempting to write an oop would > in some rare cases hit a deadlock warning due to it acquiring one of the > CBL monitors. > > Now this problem has come up yet again with the weak JNI handles bugfix > where a lock rank assertion was hit yet again due to the fact that some > code was holding a leaf lock while resolving a weak JNI handle. > > I suggest that the ranks of the involved locks are changed to "leaf - > 1", allowing them to be acquired by threads holding "leaf" locks. This Seems like a reasonable change. Hopefully in 10 we may be able to return some sanity to the notion of "leaf", "leaf-1" and "special" ranks. :) Thanks, David ----- > should not cause any further problems since reducing the ranks only > allows the locks to be taken in more places. Both pairs of locks (the > SATB and DirtyCardQ ones) have an associated FL_lock which protects a > free list of buffers which is acquired while holding the respective CBL > monitors but the FL_locks have the "special" lock rank and so they will > still order correctly with the new "leaf - 1" ranks. > > There is some horrible stuff going on in > locking_enqueue_completed_buffer but I've chosen to not change that at > this point in time even though it relates to the relative ranks of each > pair of locks. > > Testing: > JPRT, HS tiers 2 and 3 were tested with a patch to acquire the locks > even more often than what is normal to make sure that the code was > tested properly. > Some testing on HS tiers 4, 5 and 6 (Linux x64) > JDK tiers 1, 2 and 3 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 > Webrev: http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ > > Thanks > /Mikael From zhuyong.me at gmail.com Thu Mar 9 02:17:38 2017 From: zhuyong.me at gmail.com (Zhu Yong) Date: Thu, 9 Mar 2017 10:17:38 +0800 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: <289fc22c-3faa-26f8-5cce-affd3b184998@oracle.com> References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> <1d2586eb-0257-a330-2ff6-5028de510b4b@oracle.com> <289fc22c-3faa-26f8-5cce-affd3b184998@oracle.com> Message-ID: Yes, it's easy to test. I am doing benchmark (SciMark and SPECjvm2008) on target system with the newly built server, client and minimal VM. After done the benchmarks, I will build a test code and update again. On Thu, Mar 9, 2017 at 9:49 AM, David Holmes wrote: > On 9/03/2017 11:25 AM, Zhu Yong wrote: > >> Here is my latest update with good news >> >> Yesterday pulled tips code from jdk9-dev and built with my current >> toolchain (no change), this time client and minimal VM finished the test >> code without any issue. >> >> The issue might relate to >> https://bugs.openjdk.java.net/browse/JDK-8175567, the final solution >> in http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/d6a0fe7ae0e8 is >> different from the very initial proposed solution (which I used in my >> previous build). >> > > Easy enough to test that theory. :) > > Glad things are now working. > > David > ----- > > On Tue, Mar 7, 2017 at 5:54 AM, Chris Plummer > > wrote: >> >> I haven't been following this too closely, but one thing to consider >> is that you are not getting proper floating point calling >> conventions. You mentioned your configure target ABI is >> arm-vfp-sflt. This means you have VFP support, but use the old >> softfloat calling conventions (FP values passed in GPRs). Most >> platforms these days with VFP support use the new hardfloat calling >> conventions (FP values passed in FPRs). So this could simply be the >> case of building the wrong binary for your platform. >> >> - Use arm-vfp-hflt is you have VFP support and want to use hard >> float calling >> conventions. >> - Use arm-vfp-sflt if you have VFP support and want to use the >> older soft float >> calling conventions. >> - Use arm-sflt is you have a true soft float platform. >> >> There's also a very old arm sflt ABI that I don't think we support. >> The main difference from arm-sflt is the word endianess of of 64-bit >> values. I believe the presence of -mfpu=vfp is what is used to get >> the more modern endianess, which to say the least is very >> non-obvious option for doing this, but is the reason you will see >> -mfpu=vfp for all our supported arm platforms (arm-vfp-hflt, >> arm-vfp-sflt, and arm-sflt). Here's the logic from the configure >> script: >> >> if test "x$ARM_FLOAT_TYPE" = xvfp-sflt; then >> ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=softfp -mfpu=vfp >> -DFLOAT_ARCH=-vfp-sflt' >> elif test "x$ARM_FLOAT_TYPE" = xvfp-hflt; then >> ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=hard -mfpu=vfp >> -DFLOAT_ARCH=-vfp-hflt' >> elif test "x$ARM_FLOAT_TYPE" = xsflt; then >> ARM_FLOAT_TYPE_FLAGS='-msoft-float -mfpu=vfp' >> fi >> >> BTW, for the arm-sflt case, FLOAT_ARCH gets set internally to >> "-sflt". I'm not sure why for arm-sflt it's handled in the source >> and the other two platforms set it in the configure script. >> >> cheers, >> >> Chris >> >> >> On 3/5/17 8:30 PM, Zhu Yong wrote: >> >> Sometime later this week, I might get newer version of Marvell >> toolchain, I >> will do a build and test again then. >> I will update this thread when I get result. >> >> Thank you very much for helping. >> >> On Mon, Mar 6, 2017 at 9:09 AM, David Holmes >> > >> wrote: >> >> On 3/03/2017 6:59 PM, Zhu Yong wrote: >> >> Thank you very much for helping. >> >> Today I tried to debug the test code, but without luck, >> still can't >> figure out why. >> >> First, I followed the instruction >> http://stackoverflow.com/quest >> ions/29916995/intellij-idea- >> > tions/29916995/intellij-idea-> >> remotely-debug-java-console-program, >> changed suspend=y because the test code is short, it >> will exit before >> debug start if not suspend first. Unfortunately, with >> remote debug, I >> was not able to reproduce the issue. Same class file, >> directly run, will >> produce the issue, remote debug won't. >> >> Then, I tried *jdb *from my OpenJDK9 client vm, run >> directly on my >> target system to debug the code. If I set breakpoint, >> program run >> without issue, however, if don't set breakpoint, just >> "run" in jdb, >> problem reproduced. >> >> This suggests to me that a floating-point register is not >> being preserved >> when needed. When you run with the breakpoint you will get >> full >> save/restore of all the register state. Just a theory. >> >> Next, I tried to use *javap *to decompile the output class >> file, run >> >> OpenJDK9 client vm javap and host side javap on same >> class file, the >> output is same for getTime function. >> >> It is a native code issue not a bytecode issue, so javap >> wont help here. >> >> One interesting point I noticed today is the output like >> below when >> >> problem happen, 2nd call of getTime() has value of 1st >> call result / >> 1000. Looks like 2nd call getTime(), double variable q >> was not updated >> startTime: 284092.063000 endTime: 284.092063 runTime: >> -283807.970937 >> >> What should I do so that I can come a conclusion that >> it's due to >> compiler issue? >> >> Can you get access to a different gcc to build with? >> >> David >> ----- >> >> >> On Thu, Mar 2, 2017 at 12:32 PM, David Holmes >> >> > >> wrote: >> >> I built your sample program with latest JDK 9 for >> arm-vfp-sflt and >> ran on an emulator I have available, and I had no >> issues. >> >> So I still have concerns this may be a >> build/compiler issue. >> >> David >> >> >> On 2/03/2017 12:51 PM, David Holmes wrote: >> >> I've moved this discussion over to hotspot-dev >> as this seems no >> longer a >> build issue but a C1 soft-float issue. >> >> David >> >> On 2/03/2017 12:35 PM, Zhu Yong wrote: >> >> If run with -Xint, it works. >> >> I have created simplified FP test by remove >> all non-related >> code from >> Whetstone test code. >> The test code is here: >> >> https://gist.github.com/yongz >> hy/d65c26d39fe5d549d1b406c7c84e >> > hy/d65c26d39fe5d549d1b406c7c84e> >> acd4 >> >> > zhy/d65c26d39fe5d549d1b406c7c84 >> > hy/d65c26d39fe5d549d1b406c7c84> >> eacd4> >> >> I am not sure if the test code can help or >> not. The >> behaviour is weird : >> - If I print both itime and q, program will >> run OK >> - inside condition block if(q<1.0f), if I >> use exit code >> 2,3,4,5, problem >> appears, however, if I use exit code >=6, >> program run OK. >> >> I always get exit code 5, the line q = >> (double)itime is wrong? >> >> public static double getTime() >> { >> double q; >> long itime; >> itime = >> System.currentTimeMillis(); >> if(itime<1000000) { >> System.exit(1); >> } >> //System.out.printf("time long >> value %d\n", >> itime); >> q = (double) itime; >> if(q < 1.0f) { >> System.exit(5); // I >> always get exit code 5 >> } >> //System.out.printf("time float >> value %f\n", q); >> return q / 1000.0; >> } >> >> >> >> On Wed, Mar 1, 2017 at 5:31 PM, David Holmes >> > >> > > >> > >> > >>> wrote: >> >> On 1/03/2017 6:45 PM, Zhu Yong wrote: >> >> OK, for the Whetstone Benchmark, I >> added debug >> printing and >> found it's >> due to function getTime(), it only >> get good value on >> 1st call, >> all >> following calls get 0s. >> Debug printing of itime value is >> correct. So reason >> should be >> the return >> division line. Could it because >> toolchain's floating >> point >> operation??? >> But why server VM Ok? >> >> >> Server and client implement FP logic >> differently, and >> particularly >> as this is soft-fp, they may well not >> be in sync. Does >> -Xint work? >> >> Can you isolate to a simple FP test? >> >> David >> >> public static double getTime() >> { >> double q; >> long itime; >> itime = >> System.currentTimeMillis(); >> q = (double) itime; >> return q / 1000.0; >> } >> >> On Wed, Mar 1, 2017 at 3:14 PM, >> David Holmes >> > >> > > >> > >> > >> >> > >> > > >> > >> > >>>> wrote: >> >> On 1/03/2017 4:26 PM, Zhu Yong >> wrote: >> >> We use >> armv7-marvell-linux-gnueabi-softfp >> toolchain >> >> gcc version 4.6.4 (Marvell >> GCC release >> 20150204-c4af733b 64K >> MAXPAGESIZE >> ALIGN CVE-2015-0235) >> >> Is this toolchain too old? >> I am asking >> because I saw >> other strange >> issues as well: >> >> >> We last used gcc 4.7.2. I can't >> really say if >> 4.6.4 is "too >> old". >> >> I have successfully build >> server, client and >> minimal VM, >> when I run >> Dhrystone benchmark file >> (the same jar file >> from >> >> http://www.okayan.jp/DhrystoneApplet/ >> >> > > >> >> > >> > >> >> >> > >> > > >> >> > >> > >>>), the >> performance >> from >> server VM >> is much better than client >> and minimal VM. >> (minimal: 1629852, client: >> 1615508, server: >> 2338871 ) >> >> >> That's expected. The server VM >> is high >> performance. >> >> And when run Whetstone >> Benchmark >> from >> >> >> http://www.roylongbottom.org.uk/online/whetjava2.html >> >> >> > > >> >> >> > >> >> > >> >> >> >> > >> >> > > >> >> >> > >> >> > > >>>>, >> server VM >> finished with good result, >> client and >> minimal VM not >> able to finish >> (program stuck there >> forever after print 1st >> line of >> output). >> >> >> That needs investigating. You >> need to try and >> generate a >> stackdump >> to see where things are stuck. >> >> David >> >> >> On Wed, Mar 1, 2017 at 1:56 >> PM, David Holmes >> > >> > > >> > >> > >> >> > > david.holmes at oracle.com > >> > >> > >>> >> >> > >> > > >> > >> > >> >> >> > >> > > >> > >> > >>>>> wrote: >> >> On 1/03/2017 3:39 PM, >> Zhu Yong wrote: >> >> Thank you for the >> information, I >> managed to make >> minimal >> build >> pass now. >> >> Initially I though >> JVM_EXCLUDE_FILES >> need to add >> additional 3 >> generated >> files (they appeared >> in >> _BUILD_LIBJVM_objectfilenames.txt) >> : >> >> bytecodeInterpreterWithChecks.cpp >> jvmtiEnter.cpp >> jvmtiEnterTrace.cpp >> But build still >> fail with same error >> message. >> >> Finally I figured >> out it's due to >> those 8 doit() >> functions >> implementation in >> jvmtiEnvBase.hpp >> file. After >> move all >> those 8 >> doit() >> implementations to >> jvmtiEnvBase.cpp, >> build of >> minimal VM >> passed >> without >> any issue. >> >> >> That seems to be a >> compiler issue. >> jvmtiEnvBase.hpp is >> unconditionally >> included in a couple of >> places >> because we >> have to >> have enough of the >> JVMTI code to say >> JVMTI is not >> available. >> Those >> doit() implementations, >> though arguably >> could be >> conditional on >> INCLUDE_JVMTI, are not >> referenced by any >> called code >> and so >> should >> not linked in. But in >> your case it seems >> they are. >> >> What toolchain are you >> using? >> >> David >> ----- >> >> Changes appended >> ============= >> >> --- >> >> .../src/share/vm/prims/jvmtiE >> nvBase.cpp >> | 74 >> >> ++++++++++++++++++++++ >> >> .../src/share/vm/prims/jvmtiE >> nvBase.hpp >> | 68 >> +++----------------- >> 2 files changed, >> 82 insertions(+), 60 >> deletions(-) >> mode change 100644 >> => 100755 >> >> hotspot-9211c2e89c1c/src/share >> /vm/prims/jvmtiEnvBase.cpp >> mode change 100644 >> => 100755 >> >> hotspot-9211c2e89c1c/src/share >> /vm/prims/jvmtiEnvBase.hpp >> >> diff --git >> >> >> >> a/hotspot-9211c2e89c1c/src/sh >> are/vm/prims/jvmtiEnvBase.cpp >> >> >> >> b/hotspot-9211c2e89c1c/src/sh >> are/vm/prims/jvmtiEnvBase.cpp >> old mode 100644 >> new mode 100755 >> index >> dd241a0..e5832ba >> --- >> >> >> a/hotspot-9211c2e89c1c/src/sh >> are/vm/prims/jvmtiEnvBase.cpp >> +++ >> >> >> b/hotspot-9211c2e89c1c/src/sh >> are/vm/prims/jvmtiEnvBase.cpp >> @@ -1283,6 +1283,80 >> @@ >> >> VM_GetMultipleStackTraces::all >> ocate_and_fill_stacks(jint >> thread_count) { >> "the last >> copied frame >> info must be >> the last >> record"); >> } >> >> +void >> >> +VM_UpdateForPopTopFrame::doit() { >> + JavaThread* jt = >> _state->get_thread(); >> + if >> (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + >> _state->update_for_pop_top_fra >> me(); >> + } else { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + } >> +} >> + >> +void >> >> +VM_SetFramePop::doit() { >> + JavaThread* jt = >> _state->get_thread(); >> + if >> (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + int >> frame_number = >> _state->count_frames() - >> _depth; >> + >> >> >> >> >> _state->env_thread_state((Jvm >> tiEnvBase*)_env)->set_frame_ >> pop(frame_number); >> >> + } else { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + } >> +} >> + >> +void >> >> +VM_GetOwnedMonitorInfo::doit() { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if >> (Threads::includes(_java_thread) && >> >> !_java_thread->is_exiting() >> + >> && >> >> _java_thread->threadObj() != >> NULL) { >> + _result = >> ((JvmtiEnvBase >> >> *)_env)->get_owned_monitors(_calling_thread, >> _java_thread, >> + >> >> _owned_monitors_list); >> + } >> +} >> + >> +void >> >> +VM_GetObjectMonitorUsage::doit() { >> + _result = >> ((JvmtiEnvBase*) >> >> >> _env)->get_object_monitor_usage(_calling_thread, >> _object, >> _info_ptr); >> +} >> + >> +void >> >> +VM_GetCurrentContendedMonitor::doit() >> { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if >> (Threads::includes(_java_thread) && >> >> !_java_thread->is_exiting() && >> + >> _java_thread->threadObj() != >> NULL) { >> + _result = >> ((JvmtiEnvBase >> >> >> >> >> *)_env)->get_current_contended_monitor(_calling_thread,_ >> java_thread,_owned_monitor_ptr); >> >> + } >> +} >> + >> +void >> >> +VM_GetStackTrace::doit() { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if >> (Threads::includes(_java_thread) && >> >> !_java_thread->is_exiting() >> + >> && >> >> _java_thread->threadObj() != >> NULL) { >> + _result = >> ((JvmtiEnvBase >> >> *)_env)->get_stack_trace(_java_thread, >> + >> _start_depth, >> _max_count, >> + >> _frame_buffer, >> _count_ptr); >> + } >> +} >> + >> +void >> >> +VM_GetFrameCount::doit() { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + JavaThread* jt = >> _state->get_thread(); >> + if >> (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() != >> NULL) { >> + _result = >> ((JvmtiEnvBase*)_env)->get_ >> fra >> me_count(_state, >> _count_ptr); >> + } >> +} >> + >> +void >> >> +VM_GetFrameLocation::doit() { >> + _result = >> JVMTI_ERROR_THREAD_NOT_ALIVE; >> + if >> (Threads::includes(_java_thread) && >> >> !_java_thread->is_exiting() && >> + >> _java_thread->threadObj() != >> NULL) { >> + _result = >> >> ((JvmtiEnvBase*)_env)->get_fra >> me_location(_java_thread, >> _depth, >> + >> _method_ptr, >> _location_ptr); >> + } >> +} >> >> void >> >> VM_GetThreadListStackTraces::doit() >> { >> diff --git >> >> >> >> a/hotspot-9211c2e89c1c/src/sh >> are/vm/prims/jvmtiEnvBase.hpp >> >> >> >> b/hotspot-9211c2e89c1c/src/sh >> are/vm/prims/jvmtiEnvBase.hpp >> old mode 100644 >> new mode 100755 >> index >> 04e6869..00b9890 >> --- >> >> >> a/hotspot-9211c2e89c1c/src/sh >> are/vm/prims/jvmtiEnvBase.hpp >> +++ >> >> >> b/hotspot-9211c2e89c1c/src/sh >> are/vm/prims/jvmtiEnvBase.hpp >> @@ -359,14 +359,7 >> @@ public: >> } >> VMOp_Type type() >> const { return >> VMOp_UpdateForPopTopFrame; } >> jvmtiError >> result() { return >> _result; } >> - void doit() { >> - JavaThread* jt = >> _state->get_thread(); >> - if >> (Threads::includes(jt) && >> !jt->is_exiting() && >> jt->threadObj() >> != NULL) { >> - >> _state->update_for_pop_top_frame(); >> - } else { >> - _result = > > From shafi.s.ahmad at oracle.com Thu Mar 9 05:03:03 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 8 Mar 2017 21:03:03 -0800 (PST) Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> Message-ID: Hi David, Coleen, Thank you for the review. I will send the updated webrev link. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Thursday, March 09, 2017 7:35 AM > To: Coleen Phillimore ; hotspot- > dev at openjdk.java.net > Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > PerfLongVariant::sample > > Hi Coleen, > > On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: > > > > This doesn't look right. > > > > Don't you need to still check for _sampled != NULL? > > That use of _sampled was in dead code that is now removed. > > if (_sample_helper == NULL) return; > - if (_sample_helper != NULL) { > *(jlong*)_valuep = _sample_helper->take_sample(); > - } > - else if (_sampled != NULL) { > - *(jlong*)_valuep = *_sampled; > - } > } > > This change looks fine to me. > > > Can you remove the JJJ in the comment? > > I second that :) > > Thanks, > David > ----- > > > I don't know what tests were run > > where Jon (aka JJJ in the code) found this, do you? > > > > Thanks, > > Coleen > > > > > > On 3/8/17 10:21 AM, Shafi Ahmad wrote: > >> Hi, > >> > >> May I get the review done for this. > >> > >> Regards, > >> Shafi > >> > >>> -----Original Message----- > >>> From: Shafi Ahmad > >>> Sent: Wednesday, March 01, 2017 4:27 PM > >>> To: hotspot-dev at openjdk.java.net > >>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method > >>> PerfLongVariant::sample > >>> > >>> Hi, > >>> > >>> Summary: > >>> It's a very small change to a single file. > >>> > >>> void PerfLongVariant::sample() { > >>> > >>> - assert(_sample_helper != NULL || _sampled != NULL, "unexpected > >>> state"); > >>> + // JJJ - This should not happen. Maybe the first sample is taken > >>> + // while the _sample_helper is being null'ed out. > >>> + // assert(_sample_helper != NULL || _sampled != NULL, "unexpected > >>> + state"); if (_sample_helper == NULL) return; > >>> > >>> if (_sample_helper != NULL) { > >>> *(jlong*)_valuep = _sample_helper->take_sample(); > >>> } > >>> else if (_sampled != NULL) { > >>> *(jlong*)_valuep = *_sampled; > >>> } > >>> } > >>> Above five lines are modified in changeset > >>> > http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.1 > >>> 2 Due to the addition of NULL check and return > >>> if (_sample_helper == NULL) return; the else-if block becomes > >>> redundant and statement 'if (_sample_helper != NULL)' is not needed. > >>> > >>> Webrev link: > http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ > >>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 > >>> > >>> Testing: jprt. > >>> > >>> Thanks, > >>> Shafi > > From shafi.s.ahmad at oracle.com Thu Mar 9 05:08:02 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 8 Mar 2017 21:08:02 -0800 (PST) Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> Message-ID: Hi, Please find updated webrev link. http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Thursday, March 09, 2017 7:35 AM > To: Coleen Phillimore ; hotspot- > dev at openjdk.java.net > Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > PerfLongVariant::sample > > Hi Coleen, > > On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: > > > > This doesn't look right. > > > > Don't you need to still check for _sampled != NULL? > > That use of _sampled was in dead code that is now removed. > > if (_sample_helper == NULL) return; > - if (_sample_helper != NULL) { > *(jlong*)_valuep = _sample_helper->take_sample(); > - } > - else if (_sampled != NULL) { > - *(jlong*)_valuep = *_sampled; > - } > } > > This change looks fine to me. > > > Can you remove the JJJ in the comment? > > I second that :) > > Thanks, > David > ----- > > > I don't know what tests were run > > where Jon (aka JJJ in the code) found this, do you? > > > > Thanks, > > Coleen > > > > > > On 3/8/17 10:21 AM, Shafi Ahmad wrote: > >> Hi, > >> > >> May I get the review done for this. > >> > >> Regards, > >> Shafi > >> > >>> -----Original Message----- > >>> From: Shafi Ahmad > >>> Sent: Wednesday, March 01, 2017 4:27 PM > >>> To: hotspot-dev at openjdk.java.net > >>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method > >>> PerfLongVariant::sample > >>> > >>> Hi, > >>> > >>> Summary: > >>> It's a very small change to a single file. > >>> > >>> void PerfLongVariant::sample() { > >>> > >>> - assert(_sample_helper != NULL || _sampled != NULL, "unexpected > >>> state"); > >>> + // JJJ - This should not happen. Maybe the first sample is taken > >>> + // while the _sample_helper is being null'ed out. > >>> + // assert(_sample_helper != NULL || _sampled != NULL, "unexpected > >>> + state"); if (_sample_helper == NULL) return; > >>> > >>> if (_sample_helper != NULL) { > >>> *(jlong*)_valuep = _sample_helper->take_sample(); > >>> } > >>> else if (_sampled != NULL) { > >>> *(jlong*)_valuep = *_sampled; > >>> } > >>> } > >>> Above five lines are modified in changeset > >>> > http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.1 > >>> 2 Due to the addition of NULL check and return > >>> if (_sample_helper == NULL) return; the else-if block becomes > >>> redundant and statement 'if (_sample_helper != NULL)' is not needed. > >>> > >>> Webrev link: > http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ > >>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 > >>> > >>> Testing: jprt. > >>> > >>> Thanks, > >>> Shafi > > From david.holmes at oracle.com Thu Mar 9 05:19:25 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 9 Mar 2017 15:19:25 +1000 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> Message-ID: Hi Shafi, On 9/03/2017 3:08 PM, Shafi Ahmad wrote: > Hi, > > Please find updated webrev link. > http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ The more I read this code block, the comment and the commented-out assertion: 215 void PerfLongVariant::sample() { 216 217 // This should not happen. Maybe the first sample is taken 218 // while the _sample_helper is being null'ed out. 219 // assert(_sample_helper != NULL || _sampled != NULL, "unexpected state"); 220 if (_sample_helper == NULL) return; 221 222 *(jlong*)_valuep = _sample_helper->take_sample(); 223 } the less it makes sense to me. ??? David > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Thursday, March 09, 2017 7:35 AM >> To: Coleen Phillimore ; hotspot- >> dev at openjdk.java.net >> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >> PerfLongVariant::sample >> >> Hi Coleen, >> >> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: >>> >>> This doesn't look right. >>> >>> Don't you need to still check for _sampled != NULL? >> >> That use of _sampled was in dead code that is now removed. >> >> if (_sample_helper == NULL) return; >> - if (_sample_helper != NULL) { >> *(jlong*)_valuep = _sample_helper->take_sample(); >> - } >> - else if (_sampled != NULL) { >> - *(jlong*)_valuep = *_sampled; >> - } >> } >> >> This change looks fine to me. >> >>> Can you remove the JJJ in the comment? >> >> I second that :) >> >> Thanks, >> David >> ----- >> >>> I don't know what tests were run >>> where Jon (aka JJJ in the code) found this, do you? >>> >>> Thanks, >>> Coleen >>> >>> >>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: >>>> Hi, >>>> >>>> May I get the review done for this. >>>> >>>> Regards, >>>> Shafi >>>> >>>>> -----Original Message----- >>>>> From: Shafi Ahmad >>>>> Sent: Wednesday, March 01, 2017 4:27 PM >>>>> To: hotspot-dev at openjdk.java.net >>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >>>>> PerfLongVariant::sample >>>>> >>>>> Hi, >>>>> >>>>> Summary: >>>>> It's a very small change to a single file. >>>>> >>>>> void PerfLongVariant::sample() { >>>>> >>>>> - assert(_sample_helper != NULL || _sampled != NULL, "unexpected >>>>> state"); >>>>> + // JJJ - This should not happen. Maybe the first sample is taken >>>>> + // while the _sample_helper is being null'ed out. >>>>> + // assert(_sample_helper != NULL || _sampled != NULL, "unexpected >>>>> + state"); if (_sample_helper == NULL) return; >>>>> >>>>> if (_sample_helper != NULL) { >>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>> } >>>>> else if (_sampled != NULL) { >>>>> *(jlong*)_valuep = *_sampled; >>>>> } >>>>> } >>>>> Above five lines are modified in changeset >>>>> >> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.1 >>>>> 2 Due to the addition of NULL check and return >>>>> if (_sample_helper == NULL) return; the else-if block becomes >>>>> redundant and statement 'if (_sample_helper != NULL)' is not needed. >>>>> >>>>> Webrev link: >> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >>>>> >>>>> Testing: jprt. >>>>> >>>>> Thanks, >>>>> Shafi >>> From shafi.s.ahmad at oracle.com Thu Mar 9 05:30:51 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 8 Mar 2017 21:30:51 -0800 (PST) Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> Message-ID: Hi David, Yes, you are right. The comments are not at all needed because of if (_sample_helper == NULL) return; Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Thursday, March 09, 2017 10:49 AM > To: Shafi Ahmad ; Coleen Phillimore > ; hotspot-dev at openjdk.java.net > Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > PerfLongVariant::sample > > Hi Shafi, > > On 9/03/2017 3:08 PM, Shafi Ahmad wrote: > > Hi, > > > > Please find updated webrev link. > > http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ > > The more I read this code block, the comment and the commented-out > assertion: > > 215 void PerfLongVariant::sample() { > 216 > 217 // This should not happen. Maybe the first sample is taken > 218 // while the _sample_helper is being null'ed out. > 219 // assert(_sample_helper != NULL || _sampled != NULL, > "unexpected state"); > 220 if (_sample_helper == NULL) return; > 221 > 222 *(jlong*)_valuep = _sample_helper->take_sample(); > 223 } > > the less it makes sense to me. ??? > > David > > > > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Thursday, March 09, 2017 7:35 AM > >> To: Coleen Phillimore ; hotspot- > >> dev at openjdk.java.net > >> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > >> PerfLongVariant::sample > >> > >> Hi Coleen, > >> > >> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: > >>> > >>> This doesn't look right. > >>> > >>> Don't you need to still check for _sampled != NULL? > >> > >> That use of _sampled was in dead code that is now removed. > >> > >> if (_sample_helper == NULL) return; > >> - if (_sample_helper != NULL) { > >> *(jlong*)_valuep = _sample_helper->take_sample(); > >> - } > >> - else if (_sampled != NULL) { > >> - *(jlong*)_valuep = *_sampled; > >> - } > >> } > >> > >> This change looks fine to me. > >> > >>> Can you remove the JJJ in the comment? > >> > >> I second that :) > >> > >> Thanks, > >> David > >> ----- > >> > >>> I don't know what tests were run > >>> where Jon (aka JJJ in the code) found this, do you? > >>> > >>> Thanks, > >>> Coleen > >>> > >>> > >>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: > >>>> Hi, > >>>> > >>>> May I get the review done for this. > >>>> > >>>> Regards, > >>>> Shafi > >>>> > >>>>> -----Original Message----- > >>>>> From: Shafi Ahmad > >>>>> Sent: Wednesday, March 01, 2017 4:27 PM > >>>>> To: hotspot-dev at openjdk.java.net > >>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method > >>>>> PerfLongVariant::sample > >>>>> > >>>>> Hi, > >>>>> > >>>>> Summary: > >>>>> It's a very small change to a single file. > >>>>> > >>>>> void PerfLongVariant::sample() { > >>>>> > >>>>> - assert(_sample_helper != NULL || _sampled != NULL, "unexpected > >>>>> state"); > >>>>> + // JJJ - This should not happen. Maybe the first sample is > >>>>> + taken // while the _sample_helper is being null'ed out. > >>>>> + // assert(_sample_helper != NULL || _sampled != NULL, > >>>>> + "unexpected state"); if (_sample_helper == NULL) return; > >>>>> > >>>>> if (_sample_helper != NULL) { > >>>>> *(jlong*)_valuep = _sample_helper->take_sample(); > >>>>> } > >>>>> else if (_sampled != NULL) { > >>>>> *(jlong*)_valuep = *_sampled; > >>>>> } > >>>>> } > >>>>> Above five lines are modified in changeset > >>>>> > >> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.1 > >>>>> 2 Due to the addition of NULL check and return > >>>>> if (_sample_helper == NULL) return; the else-if block becomes > >>>>> redundant and statement 'if (_sample_helper != NULL)' is not > needed. > >>>>> > >>>>> Webrev link: > >> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ > >>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 > >>>>> > >>>>> Testing: jprt. > >>>>> > >>>>> Thanks, > >>>>> Shafi > >>> From shafi.s.ahmad at oracle.com Thu Mar 9 05:39:44 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 8 Mar 2017 21:39:44 -0800 (PST) Subject: [8u-dev] RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field Message-ID: Hi, Please review the backport of "JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field" to jdk8u-dev. The backport is not clean even though the changes are very small. Webrev: http://cr.openjdk.java.net/~shshahma/8176150/webrev.00/ Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8171194 Jdk10 review: http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025875.html Testing: jprt and jtreg test. Regards, Shafi From david.holmes at oracle.com Thu Mar 9 06:15:46 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 9 Mar 2017 16:15:46 +1000 Subject: [8u-dev] RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: References: Message-ID: <9d0fe9e5-21e4-b88d-6a9a-263c9eb26d1e@oracle.com> Hi Shafi, On 9/03/2017 3:39 PM, Shafi Ahmad wrote: > Hi, > > Please review the backport of "JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field" to jdk8u-dev. > The backport is not clean even though the changes are very small. > > Webrev: http://cr.openjdk.java.net/~shshahma/8176150/webrev.00/ The code changes look fine to me. But note that as this is an enhancement you will need to get an enhancement approval for it to into 8u, as well as the actual push approval: http://openjdk.java.net/projects/jdk8u/enhancement-template.html Thanks, David > Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8171194 > Jdk10 review: http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025875.html > > Testing: jprt and jtreg test. > > > Regards, > Shafi > From thomas.schatzl at oracle.com Thu Mar 9 09:42:03 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 09 Mar 2017 10:42:03 +0100 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> Message-ID: <1489052523.3514.0.camel@oracle.com> Hi Mikael, On Wed, 2017-03-08 at 15:16 +0100, Mikael Gerdin wrote: > Hi all, > > Please review this change to the lock rank of the locks taken in the > G1 pre and post write barriers. > > The rank of these locks have long been a problem since even though > they are leaf locks semantically they have been ranked as "nonleaf" > locks in the lock rank system and this has caused several issues over > the years where a thread holding a VM mutex and attempting to write > an oop would in some rare cases hit a deadlock warning due to it > acquiring one of the CBL monitors. > > Now this problem has come up yet again with the weak JNI handles > bugfix where a lock rank assertion was hit yet again due to the fact > that some code was holding a leaf lock while resolving a weak JNI > handle. > > I suggest that the ranks of the involved locks are changed to "leaf - > 1", allowing them to be acquired by threads holding "leaf" locks. Looks good to me. Thomas From shafi.s.ahmad at oracle.com Thu Mar 9 10:09:02 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 9 Mar 2017 02:09:02 -0800 (PST) Subject: [8u-dev] RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: <9d0fe9e5-21e4-b88d-6a9a-263c9eb26d1e@oracle.com> References: <9d0fe9e5-21e4-b88d-6a9a-263c9eb26d1e@oracle.com> Message-ID: Hi David, Thank you for the review and letting me know about the enhancement approval process. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Thursday, March 09, 2017 11:46 AM > To: Shafi Ahmad ; hotspot- > dev at openjdk.java.net > Subject: Re: [8u-dev] RFR JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field > > Hi Shafi, > > On 9/03/2017 3:39 PM, Shafi Ahmad wrote: > > Hi, > > > > Please review the backport of "JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field" to jdk8u-dev. > > The backport is not clean even though the changes are very small. > > > > Webrev: http://cr.openjdk.java.net/~shshahma/8176150/webrev.00/ > > The code changes look fine to me. > > But note that as this is an enhancement you will need to get an > enhancement approval for it to into 8u, as well as the actual push approval: > > http://openjdk.java.net/projects/jdk8u/enhancement-template.html > > Thanks, > David > > > Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8171194 > > Jdk10 review: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/02587 > > 5.html > > > > Testing: jprt and jtreg test. > > > > > > Regards, > > Shafi > > From shafi.s.ahmad at oracle.com Thu Mar 9 10:55:32 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 9 Mar 2017 02:55:32 -0800 (PST) Subject: [8u-dev] RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: References: <9d0fe9e5-21e4-b88d-6a9a-263c9eb26d1e@oracle.com> Message-ID: <8b0b3217-fdbf-4f44-bf2c-9586f9130b93@default> Hi All, Let me know if I have to remove the comment completely. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Thursday, March 09, 2017 3:39 PM > To: David Holmes ; hotspot- > dev at openjdk.java.net > Subject: RE: [8u-dev] RFR JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field > > Hi David, > > Thank you for the review and letting me know about the enhancement > approval process. > > Regards, > Shafi > > > -----Original Message----- > > From: David Holmes > > Sent: Thursday, March 09, 2017 11:46 AM > > To: Shafi Ahmad ; hotspot- > > dev at openjdk.java.net > > Subject: Re: [8u-dev] RFR JDK-8171194: Exception "Duplicate field > > name&signature in class file" should report the name and signature of > > the field > > > > Hi Shafi, > > > > On 9/03/2017 3:39 PM, Shafi Ahmad wrote: > > > Hi, > > > > > > Please review the backport of "JDK-8171194: Exception "Duplicate > > > field > > name&signature in class file" should report the name and signature of > > the field" to jdk8u-dev. > > > The backport is not clean even though the changes are very small. > > > > > > Webrev: http://cr.openjdk.java.net/~shshahma/8176150/webrev.00/ > > > > The code changes look fine to me. > > > > But note that as this is an enhancement you will need to get an > > enhancement approval for it to into 8u, as well as the actual push approval: > > > > http://openjdk.java.net/projects/jdk8u/enhancement-template.html > > > > Thanks, > > David > > > > > Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8171194 > > > Jdk10 review: > > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025 > > > 87 > > > 5.html > > > > > > Testing: jprt and jtreg test. > > > > > > > > > Regards, > > > Shafi > > > From shafi.s.ahmad at oracle.com Thu Mar 9 11:21:41 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 9 Mar 2017 03:21:41 -0800 (PST) Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> Message-ID: <135d4f81-42dd-42ee-bcc9-9c81d177a1db@default> Hi All, Let me know if I have to remove the comment completely. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Thursday, March 09, 2017 11:01 AM > To: David Holmes ; Coleen Phillimore > ; hotspot-dev at openjdk.java.net > Subject: RE: JDK 10 RFR JDK-8167425: Redundant code in method > PerfLongVariant::sample > > Hi David, > > Yes, you are right. > > The comments are not at all needed because of if (_sample_helper == NULL) > return; > > Regards, > Shafi > > -----Original Message----- > > From: David Holmes > > Sent: Thursday, March 09, 2017 10:49 AM > > To: Shafi Ahmad ; Coleen Phillimore > > ; hotspot-dev at openjdk.java.net > > Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > > PerfLongVariant::sample > > > > Hi Shafi, > > > > On 9/03/2017 3:08 PM, Shafi Ahmad wrote: > > > Hi, > > > > > > Please find updated webrev link. > > > http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ > > > > The more I read this code block, the comment and the commented-out > > assertion: > > > > 215 void PerfLongVariant::sample() { > > 216 > > 217 // This should not happen. Maybe the first sample is taken > > 218 // while the _sample_helper is being null'ed out. > > 219 // assert(_sample_helper != NULL || _sampled != NULL, > > "unexpected state"); > > 220 if (_sample_helper == NULL) return; > > 221 > > 222 *(jlong*)_valuep = _sample_helper->take_sample(); > > 223 } > > > > the less it makes sense to me. ??? > > > > David > > > > > > > > > > > Regards, > > > Shafi > > > > > >> -----Original Message----- > > >> From: David Holmes > > >> Sent: Thursday, March 09, 2017 7:35 AM > > >> To: Coleen Phillimore ; hotspot- > > >> dev at openjdk.java.net > > >> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > > >> PerfLongVariant::sample > > >> > > >> Hi Coleen, > > >> > > >> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: > > >>> > > >>> This doesn't look right. > > >>> > > >>> Don't you need to still check for _sampled != NULL? > > >> > > >> That use of _sampled was in dead code that is now removed. > > >> > > >> if (_sample_helper == NULL) return; > > >> - if (_sample_helper != NULL) { > > >> *(jlong*)_valuep = _sample_helper->take_sample(); > > >> - } > > >> - else if (_sampled != NULL) { > > >> - *(jlong*)_valuep = *_sampled; > > >> - } > > >> } > > >> > > >> This change looks fine to me. > > >> > > >>> Can you remove the JJJ in the comment? > > >> > > >> I second that :) > > >> > > >> Thanks, > > >> David > > >> ----- > > >> > > >>> I don't know what tests were run > > >>> where Jon (aka JJJ in the code) found this, do you? > > >>> > > >>> Thanks, > > >>> Coleen > > >>> > > >>> > > >>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: > > >>>> Hi, > > >>>> > > >>>> May I get the review done for this. > > >>>> > > >>>> Regards, > > >>>> Shafi > > >>>> > > >>>>> -----Original Message----- > > >>>>> From: Shafi Ahmad > > >>>>> Sent: Wednesday, March 01, 2017 4:27 PM > > >>>>> To: hotspot-dev at openjdk.java.net > > >>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method > > >>>>> PerfLongVariant::sample > > >>>>> > > >>>>> Hi, > > >>>>> > > >>>>> Summary: > > >>>>> It's a very small change to a single file. > > >>>>> > > >>>>> void PerfLongVariant::sample() { > > >>>>> > > >>>>> - assert(_sample_helper != NULL || _sampled != NULL, > > >>>>> "unexpected state"); > > >>>>> + // JJJ - This should not happen. Maybe the first sample is > > >>>>> + taken // while the _sample_helper is being null'ed out. > > >>>>> + // assert(_sample_helper != NULL || _sampled != NULL, > > >>>>> + "unexpected state"); if (_sample_helper == NULL) return; > > >>>>> > > >>>>> if (_sample_helper != NULL) { > > >>>>> *(jlong*)_valuep = _sample_helper->take_sample(); > > >>>>> } > > >>>>> else if (_sampled != NULL) { > > >>>>> *(jlong*)_valuep = *_sampled; > > >>>>> } > > >>>>> } > > >>>>> Above five lines are modified in changeset > > >>>>> > > >> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802. > > >> 1 > > >>>>> 2 Due to the addition of NULL check and return > > >>>>> if (_sample_helper == NULL) return; the else-if block becomes > > >>>>> redundant and statement 'if (_sample_helper != NULL)' is not > > needed. > > >>>>> > > >>>>> Webrev link: > > >> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ > > >>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 > > >>>>> > > >>>>> Testing: jprt. > > >>>>> > > >>>>> Thanks, > > >>>>> Shafi > > >>> From shafi.s.ahmad at oracle.com Thu Mar 9 11:23:18 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 9 Mar 2017 03:23:18 -0800 (PST) Subject: [8u-dev] RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: <8b0b3217-fdbf-4f44-bf2c-9586f9130b93@default> References: <9d0fe9e5-21e4-b88d-6a9a-263c9eb26d1e@oracle.com> <8b0b3217-fdbf-4f44-bf2c-9586f9130b93@default> Message-ID: <5a1151dc-31b2-4c85-b1b0-a3c2262b816b@default> Hi All, Please ignore my previous mail. Sorry, by mistake I replied to this mail thread. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Thursday, March 09, 2017 4:26 PM > To: hotspot-dev at openjdk.java.net > Subject: RE: [8u-dev] RFR JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field > > Hi All, > > Let me know if I have to remove the comment completely. > > Regards, > Shafi > > > -----Original Message----- > > From: Shafi Ahmad > > Sent: Thursday, March 09, 2017 3:39 PM > > To: David Holmes ; hotspot- > > dev at openjdk.java.net > > Subject: RE: [8u-dev] RFR JDK-8171194: Exception "Duplicate field > > name&signature in class file" should report the name and signature of > > the field > > > > Hi David, > > > > Thank you for the review and letting me know about the enhancement > > approval process. > > > > Regards, > > Shafi > > > > > -----Original Message----- > > > From: David Holmes > > > Sent: Thursday, March 09, 2017 11:46 AM > > > To: Shafi Ahmad ; hotspot- > > > dev at openjdk.java.net > > > Subject: Re: [8u-dev] RFR JDK-8171194: Exception "Duplicate field > > > name&signature in class file" should report the name and signature > > > of the field > > > > > > Hi Shafi, > > > > > > On 9/03/2017 3:39 PM, Shafi Ahmad wrote: > > > > Hi, > > > > > > > > Please review the backport of "JDK-8171194: Exception "Duplicate > > > > field > > > name&signature in class file" should report the name and signature > > > of the field" to jdk8u-dev. > > > > The backport is not clean even though the changes are very small. > > > > > > > > Webrev: http://cr.openjdk.java.net/~shshahma/8176150/webrev.00/ > > > > > > The code changes look fine to me. > > > > > > But note that as this is an enhancement you will need to get an > > > enhancement approval for it to into 8u, as well as the actual push > approval: > > > > > > http://openjdk.java.net/projects/jdk8u/enhancement-template.html > > > > > > Thanks, > > > David > > > > > > > Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8171194 > > > > Jdk10 review: > > > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/0 > > > > 25 > > > > 87 > > > > 5.html > > > > > > > > Testing: jprt and jtreg test. > > > > > > > > > > > > Regards, > > > > Shafi > > > > From david.holmes at oracle.com Thu Mar 9 11:54:06 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 9 Mar 2017 21:54:06 +1000 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <135d4f81-42dd-42ee-bcc9-9c81d177a1db@default> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> <135d4f81-42dd-42ee-bcc9-9c81d177a1db@default> Message-ID: Hi Shafi, On 9/03/2017 9:21 PM, Shafi Ahmad wrote: > Hi All, > > Let me know if I have to remove the comment completely. I don't think the comment makes much sense, so assuming the code is logically correct I would remove it. I'd then suggest simplifying the method to just: void PerfLongVariant::sample() { if (_sample_helper != NULL) { *(jlong*)_valuep = _sample_helper->take_sample(); } } But I have to say that I don't really understand what this code is doing. David > Regards, > Shafi > >> -----Original Message----- >> From: Shafi Ahmad >> Sent: Thursday, March 09, 2017 11:01 AM >> To: David Holmes ; Coleen Phillimore >> ; hotspot-dev at openjdk.java.net >> Subject: RE: JDK 10 RFR JDK-8167425: Redundant code in method >> PerfLongVariant::sample >> >> Hi David, >> >> Yes, you are right. >> >> The comments are not at all needed because of if (_sample_helper == NULL) >> return; >> >> Regards, >> Shafi >>> -----Original Message----- >>> From: David Holmes >>> Sent: Thursday, March 09, 2017 10:49 AM >>> To: Shafi Ahmad ; Coleen Phillimore >>> ; hotspot-dev at openjdk.java.net >>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>> PerfLongVariant::sample >>> >>> Hi Shafi, >>> >>> On 9/03/2017 3:08 PM, Shafi Ahmad wrote: >>>> Hi, >>>> >>>> Please find updated webrev link. >>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ >>> >>> The more I read this code block, the comment and the commented-out >>> assertion: >>> >>> 215 void PerfLongVariant::sample() { >>> 216 >>> 217 // This should not happen. Maybe the first sample is taken >>> 218 // while the _sample_helper is being null'ed out. >>> 219 // assert(_sample_helper != NULL || _sampled != NULL, >>> "unexpected state"); >>> 220 if (_sample_helper == NULL) return; >>> 221 >>> 222 *(jlong*)_valuep = _sample_helper->take_sample(); >>> 223 } >>> >>> the less it makes sense to me. ??? >>> >>> David >>> >>> >>> >>> >>>> Regards, >>>> Shafi >>>> >>>>> -----Original Message----- >>>>> From: David Holmes >>>>> Sent: Thursday, March 09, 2017 7:35 AM >>>>> To: Coleen Phillimore ; hotspot- >>>>> dev at openjdk.java.net >>>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>>>> PerfLongVariant::sample >>>>> >>>>> Hi Coleen, >>>>> >>>>> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> This doesn't look right. >>>>>> >>>>>> Don't you need to still check for _sampled != NULL? >>>>> >>>>> That use of _sampled was in dead code that is now removed. >>>>> >>>>> if (_sample_helper == NULL) return; >>>>> - if (_sample_helper != NULL) { >>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>> - } >>>>> - else if (_sampled != NULL) { >>>>> - *(jlong*)_valuep = *_sampled; >>>>> - } >>>>> } >>>>> >>>>> This change looks fine to me. >>>>> >>>>>> Can you remove the JJJ in the comment? >>>>> >>>>> I second that :) >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> I don't know what tests were run >>>>>> where Jon (aka JJJ in the code) found this, do you? >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: >>>>>>> Hi, >>>>>>> >>>>>>> May I get the review done for this. >>>>>>> >>>>>>> Regards, >>>>>>> Shafi >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Shafi Ahmad >>>>>>>> Sent: Wednesday, March 01, 2017 4:27 PM >>>>>>>> To: hotspot-dev at openjdk.java.net >>>>>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >>>>>>>> PerfLongVariant::sample >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Summary: >>>>>>>> It's a very small change to a single file. >>>>>>>> >>>>>>>> void PerfLongVariant::sample() { >>>>>>>> >>>>>>>> - assert(_sample_helper != NULL || _sampled != NULL, >>>>>>>> "unexpected state"); >>>>>>>> + // JJJ - This should not happen. Maybe the first sample is >>>>>>>> + taken // while the _sample_helper is being null'ed out. >>>>>>>> + // assert(_sample_helper != NULL || _sampled != NULL, >>>>>>>> + "unexpected state"); if (_sample_helper == NULL) return; >>>>>>>> >>>>>>>> if (_sample_helper != NULL) { >>>>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>>>>> } >>>>>>>> else if (_sampled != NULL) { >>>>>>>> *(jlong*)_valuep = *_sampled; >>>>>>>> } >>>>>>>> } >>>>>>>> Above five lines are modified in changeset >>>>>>>> >>>>> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802. >>>>> 1 >>>>>>>> 2 Due to the addition of NULL check and return >>>>>>>> if (_sample_helper == NULL) return; the else-if block becomes >>>>>>>> redundant and statement 'if (_sample_helper != NULL)' is not >>> needed. >>>>>>>> >>>>>>>> Webrev link: >>>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >>>>>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >>>>>>>> >>>>>>>> Testing: jprt. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Shafi >>>>>> From mikael.gerdin at oracle.com Thu Mar 9 12:01:17 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 9 Mar 2017 13:01:17 +0100 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <73B19D8E-55B9-4C66-98F5-CEFE1A15A3D9@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> <73B19D8E-55B9-4C66-98F5-CEFE1A15A3D9@oracle.com> Message-ID: <81ec5642-e536-8399-53f1-3b4b4443306f@oracle.com> Hi Kim, On 2017-03-08 23:55, Kim Barrett wrote: >> On Mar 8, 2017, at 9:16 AM, Mikael Gerdin >> wrote: >> >> Hi all, >> >> Please review this change to the lock rank of the locks taken in >> the G1 pre and post write barriers. >> >> The rank of these locks have long been a problem since even though >> they are leaf locks semantically they have been ranked as "nonleaf" >> locks in the lock rank system and this has caused several issues >> over the years where a thread holding a VM mutex and attempting to >> write an oop would in some rare cases hit a deadlock warning due to >> it acquiring one of the CBL monitors. >> >> Now this problem has come up yet again with the weak JNI handles >> bugfix where a lock rank assertion was hit yet again due to the >> fact that some code was holding a leaf lock while resolving a weak >> JNI handle. >> >> I suggest that the ranks of the involved locks are changed to "leaf >> - 1", allowing them to be acquired by threads holding "leaf" locks. >> This should not cause any further problems since reducing the ranks >> only allows the locks to be taken in more places. Both pairs of >> locks (the SATB and DirtyCardQ ones) have an associated FL_lock >> which protects a free list of buffers which is acquired while >> holding the respective CBL monitors but the FL_locks have the >> "special" lock rank and so they will still order correctly with the >> new "leaf - 1" ranks. > > Not that it matters, since, as you say, the lock ranking orders are > still okay, but I don?t recall any free list locks while holding the > corresponding completed buffer lock. Right, but the free list lock is taken when holding the corresponding shared lock. > >> There is some horrible stuff going on in >> locking_enqueue_completed_buffer but I've chosen to not change that >> at this point in time even though it relates to the relative ranks >> of each pair of locks. > > Thank you for not messing with that! I have some changes in that > area that have been waiting for JDK 10; I plan to start putting those > out for review soon. Great! > >> >> Testing: JPRT, HS tiers 2 and 3 were tested with a patch to acquire >> the locks even more often than what is normal to make sure that the >> code was tested properly. Some testing on HS tiers 4, 5 and 6 >> (Linux x64) JDK tiers 1, 2 and 3 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 Webrev: >> http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ > > Looks good. Thanks for the review, Kim. /Mikael > From mikael.gerdin at oracle.com Thu Mar 9 12:02:32 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 9 Mar 2017 13:02:32 +0100 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <9b8e33eb-e961-db9d-d0c4-e322799d6a18@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> <9b8e33eb-e961-db9d-d0c4-e322799d6a18@oracle.com> Message-ID: Hi David, On 2017-03-09 03:16, David Holmes wrote: > Hi Mikael, > > On 9/03/2017 12:16 AM, Mikael Gerdin wrote: >> Hi all, >> >> Please review this change to the lock rank of the locks taken in the G1 >> pre and post write barriers. >> >> The rank of these locks have long been a problem since even though they >> are leaf locks semantically they have been ranked as "nonleaf" locks in >> the lock rank system and this has caused several issues over the years >> where a thread holding a VM mutex and attempting to write an oop would >> in some rare cases hit a deadlock warning due to it acquiring one of the >> CBL monitors. >> >> Now this problem has come up yet again with the weak JNI handles bugfix >> where a lock rank assertion was hit yet again due to the fact that some >> code was holding a leaf lock while resolving a weak JNI handle. >> >> I suggest that the ranks of the involved locks are changed to "leaf - >> 1", allowing them to be acquired by threads holding "leaf" locks. This > > Seems like a reasonable change. Thanks :) > > Hopefully in 10 we may be able to return some sanity to the notion of > "leaf", "leaf-1" and "special" ranks. :) That would be really nice. /Mikael > > Thanks, > David > ----- > >> should not cause any further problems since reducing the ranks only >> allows the locks to be taken in more places. Both pairs of locks (the >> SATB and DirtyCardQ ones) have an associated FL_lock which protects a >> free list of buffers which is acquired while holding the respective CBL >> monitors but the FL_locks have the "special" lock rank and so they will >> still order correctly with the new "leaf - 1" ranks. >> >> There is some horrible stuff going on in >> locking_enqueue_completed_buffer but I've chosen to not change that at >> this point in time even though it relates to the relative ranks of each >> pair of locks. >> >> Testing: >> JPRT, HS tiers 2 and 3 were tested with a patch to acquire the locks >> even more often than what is normal to make sure that the code was >> tested properly. >> Some testing on HS tiers 4, 5 and 6 (Linux x64) >> JDK tiers 1, 2 and 3 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 >> Webrev: http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ >> >> Thanks >> /Mikael From coleen.phillimore at oracle.com Thu Mar 9 14:06:53 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 9 Mar 2017 09:06:53 -0500 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> Message-ID: <68bf5baf-bb75-fbb7-1533-246dbc626b48@oracle.com> On 3/8/17 9:05 PM, David Holmes wrote: > Hi Coleen, > > On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: >> >> This doesn't look right. >> >> Don't you need to still check for _sampled != NULL? > > That use of _sampled was in dead code that is now removed. I see it all now. Coleen > > if (_sample_helper == NULL) return; > - if (_sample_helper != NULL) { > *(jlong*)_valuep = _sample_helper->take_sample(); > - } > - else if (_sampled != NULL) { > - *(jlong*)_valuep = *_sampled; > - } > } > > This change looks fine to me. > >> Can you remove the JJJ in the comment? > > I second that :) > > Thanks, > David > ----- > >> I don't know what tests were run >> where Jon (aka JJJ in the code) found this, do you? >> >> Thanks, >> Coleen >> >> >> On 3/8/17 10:21 AM, Shafi Ahmad wrote: >>> Hi, >>> >>> May I get the review done for this. >>> >>> Regards, >>> Shafi >>> >>>> -----Original Message----- >>>> From: Shafi Ahmad >>>> Sent: Wednesday, March 01, 2017 4:27 PM >>>> To: hotspot-dev at openjdk.java.net >>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >>>> PerfLongVariant::sample >>>> >>>> Hi, >>>> >>>> Summary: >>>> It's a very small change to a single file. >>>> >>>> void PerfLongVariant::sample() { >>>> >>>> - assert(_sample_helper != NULL || _sampled != NULL, "unexpected >>>> state"); >>>> + // JJJ - This should not happen. Maybe the first sample is >>>> taken // >>>> + while the _sample_helper is being null'ed out. >>>> + // assert(_sample_helper != NULL || _sampled != NULL, "unexpected >>>> + state"); if (_sample_helper == NULL) return; >>>> >>>> if (_sample_helper != NULL) { >>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>> } >>>> else if (_sampled != NULL) { >>>> *(jlong*)_valuep = *_sampled; >>>> } >>>> } >>>> Above five lines are modified in changeset >>>> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.12 >>>> Due to the addition of NULL check and return >>>> if (_sample_helper == NULL) return; >>>> the else-if block becomes redundant and statement 'if >>>> (_sample_helper != >>>> NULL)' is not needed. >>>> >>>> Webrev link: http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >>>> >>>> Testing: jprt. >>>> >>>> Thanks, >>>> Shafi >> From coleen.phillimore at oracle.com Thu Mar 9 14:09:35 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 9 Mar 2017 09:09:35 -0500 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> <135d4f81-42dd-42ee-bcc9-9c81d177a1db@default> Message-ID: <13815949-0075-e5e9-f762-a3947653180f@oracle.com> On 3/9/17 6:54 AM, David Holmes wrote: > Hi Shafi, > > On 9/03/2017 9:21 PM, Shafi Ahmad wrote: >> Hi All, >> >> Let me know if I have to remove the comment completely. > > I don't think the comment makes much sense, so assuming the code is > logically correct I would remove it. I'd then suggest simplifying the > method to just: > > void PerfLongVariant::sample() { > if (_sample_helper != NULL) { > *(jlong*)_valuep = _sample_helper->take_sample(); > } > } > > But I have to say that I don't really understand what this code is doing. I don't either. I think the version without comments is probably the least confusing and looks correct. If you change it to this, I don't need to see a webrev again. thanks, Coleen > > David > >> Regards, >> Shafi >> >>> -----Original Message----- >>> From: Shafi Ahmad >>> Sent: Thursday, March 09, 2017 11:01 AM >>> To: David Holmes ; Coleen Phillimore >>> ; hotspot-dev at openjdk.java.net >>> Subject: RE: JDK 10 RFR JDK-8167425: Redundant code in method >>> PerfLongVariant::sample >>> >>> Hi David, >>> >>> Yes, you are right. >>> >>> The comments are not at all needed because of if (_sample_helper == >>> NULL) >>> return; >>> >>> Regards, >>> Shafi >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Thursday, March 09, 2017 10:49 AM >>>> To: Shafi Ahmad ; Coleen Phillimore >>>> ; hotspot-dev at openjdk.java.net >>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>>> PerfLongVariant::sample >>>> >>>> Hi Shafi, >>>> >>>> On 9/03/2017 3:08 PM, Shafi Ahmad wrote: >>>>> Hi, >>>>> >>>>> Please find updated webrev link. >>>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ >>>> >>>> The more I read this code block, the comment and the commented-out >>>> assertion: >>>> >>>> 215 void PerfLongVariant::sample() { >>>> 216 >>>> 217 // This should not happen. Maybe the first sample is taken >>>> 218 // while the _sample_helper is being null'ed out. >>>> 219 // assert(_sample_helper != NULL || _sampled != NULL, >>>> "unexpected state"); >>>> 220 if (_sample_helper == NULL) return; >>>> 221 >>>> 222 *(jlong*)_valuep = _sample_helper->take_sample(); >>>> 223 } >>>> >>>> the less it makes sense to me. ??? >>>> >>>> David >>>> >>>> >>>> >>>> >>>>> Regards, >>>>> Shafi >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Thursday, March 09, 2017 7:35 AM >>>>>> To: Coleen Phillimore ; hotspot- >>>>>> dev at openjdk.java.net >>>>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>>>>> PerfLongVariant::sample >>>>>> >>>>>> Hi Coleen, >>>>>> >>>>>> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> This doesn't look right. >>>>>>> >>>>>>> Don't you need to still check for _sampled != NULL? >>>>>> >>>>>> That use of _sampled was in dead code that is now removed. >>>>>> >>>>>> if (_sample_helper == NULL) return; >>>>>> - if (_sample_helper != NULL) { >>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>>> - } >>>>>> - else if (_sampled != NULL) { >>>>>> - *(jlong*)_valuep = *_sampled; >>>>>> - } >>>>>> } >>>>>> >>>>>> This change looks fine to me. >>>>>> >>>>>>> Can you remove the JJJ in the comment? >>>>>> >>>>>> I second that :) >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> I don't know what tests were run >>>>>>> where Jon (aka JJJ in the code) found this, do you? >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> May I get the review done for this. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Shafi >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Shafi Ahmad >>>>>>>>> Sent: Wednesday, March 01, 2017 4:27 PM >>>>>>>>> To: hotspot-dev at openjdk.java.net >>>>>>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >>>>>>>>> PerfLongVariant::sample >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Summary: >>>>>>>>> It's a very small change to a single file. >>>>>>>>> >>>>>>>>> void PerfLongVariant::sample() { >>>>>>>>> >>>>>>>>> - assert(_sample_helper != NULL || _sampled != NULL, >>>>>>>>> "unexpected state"); >>>>>>>>> + // JJJ - This should not happen. Maybe the first sample is >>>>>>>>> + taken // while the _sample_helper is being null'ed out. >>>>>>>>> + // assert(_sample_helper != NULL || _sampled != NULL, >>>>>>>>> + "unexpected state"); if (_sample_helper == NULL) return; >>>>>>>>> >>>>>>>>> if (_sample_helper != NULL) { >>>>>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>>>>>> } >>>>>>>>> else if (_sampled != NULL) { >>>>>>>>> *(jlong*)_valuep = *_sampled; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> Above five lines are modified in changeset >>>>>>>>> >>>>>> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802. >>>>>> 1 >>>>>>>>> 2 Due to the addition of NULL check and return >>>>>>>>> if (_sample_helper == NULL) return; the else-if block becomes >>>>>>>>> redundant and statement 'if (_sample_helper != NULL)' is not >>>> needed. >>>>>>>>> >>>>>>>>> Webrev link: >>>>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >>>>>>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >>>>>>>>> >>>>>>>>> Testing: jprt. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Shafi >>>>>>> From mikael.gerdin at oracle.com Thu Mar 9 15:52:10 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 9 Mar 2017 16:52:10 +0100 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <1489052523.3514.0.camel@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> <1489052523.3514.0.camel@oracle.com> Message-ID: Hi Thomas, On 2017-03-09 10:42, Thomas Schatzl wrote: > Hi Mikael, > > On Wed, 2017-03-08 at 15:16 +0100, Mikael Gerdin wrote: >> Hi all, >> >> Please review this change to the lock rank of the locks taken in the >> G1 pre and post write barriers. >> >> The rank of these locks have long been a problem since even though >> they are leaf locks semantically they have been ranked as "nonleaf" >> locks in the lock rank system and this has caused several issues over >> the years where a thread holding a VM mutex and attempting to write >> an oop would in some rare cases hit a deadlock warning due to it >> acquiring one of the CBL monitors. >> >> Now this problem has come up yet again with the weak JNI handles >> bugfix where a lock rank assertion was hit yet again due to the fact >> that some code was holding a leaf lock while resolving a weak JNI >> handle. >> >> I suggest that the ranks of the involved locks are changed to "leaf - >> 1", allowing them to be acquired by threads holding "leaf" locks. > > Looks good to me. Thanks for the review. /Mikael > > Thomas > From mikael.gerdin at oracle.com Thu Mar 9 15:53:32 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 9 Mar 2017 16:53:32 +0100 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <1af7231f-c017-2932-7979-28ac17efba66@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> <1af7231f-c017-2932-7979-28ac17efba66@oracle.com> Message-ID: <1abef748-9cc5-01e8-c3ae-069506423d72@oracle.com> Hi Coleen, On 2017-03-08 19:50, coleen.phillimore at oracle.com wrote: > This looks good. > > I filed https://bugs.openjdk.java.net/browse/JDK-8176393 Can you add > some info to it? Thanks for the review and I'll see what I can add there. /Mikael > > thanks, > Coleen > > On 3/8/17 9:16 AM, Mikael Gerdin wrote: >> Hi all, >> >> Please review this change to the lock rank of the locks taken in the >> G1 pre and post write barriers. >> >> The rank of these locks have long been a problem since even though >> they are leaf locks semantically they have been ranked as "nonleaf" >> locks in the lock rank system and this has caused several issues over >> the years where a thread holding a VM mutex and attempting to write an >> oop would in some rare cases hit a deadlock warning due to it >> acquiring one of the CBL monitors. >> >> Now this problem has come up yet again with the weak JNI handles >> bugfix where a lock rank assertion was hit yet again due to the fact >> that some code was holding a leaf lock while resolving a weak JNI handle. >> >> I suggest that the ranks of the involved locks are changed to "leaf - >> 1", allowing them to be acquired by threads holding "leaf" locks. This >> should not cause any further problems since reducing the ranks only >> allows the locks to be taken in more places. Both pairs of locks (the >> SATB and DirtyCardQ ones) have an associated FL_lock which protects a >> free list of buffers which is acquired while holding the respective >> CBL monitors but the FL_locks have the "special" lock rank and so they >> will still order correctly with the new "leaf - 1" ranks. >> >> There is some horrible stuff going on in >> locking_enqueue_completed_buffer but I've chosen to not change that at >> this point in time even though it relates to the relative ranks of >> each pair of locks. >> >> Testing: >> JPRT, HS tiers 2 and 3 were tested with a patch to acquire the locks >> even more often than what is normal to make sure that the code was >> tested properly. >> Some testing on HS tiers 4, 5 and 6 (Linux x64) >> JDK tiers 1, 2 and 3 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8176363 >> Webrev: http://cr.openjdk.java.net/~mgerdin/8176363/webrev.0/ >> >> Thanks >> /Mikael > From christian.tornqvist at oracle.com Thu Mar 9 16:42:46 2017 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Thu, 9 Mar 2017 11:42:46 -0500 Subject: RFR(M): 8175300 - Enable artifact resolution for jtreg tests In-Reply-To: <37ec01d2977d$cfb1fc20$6f15f460$@oracle.com> References: <37ec01d2977d$cfb1fc20$6f15f460$@oracle.com> Message-ID: <428501d298f4$2ec302a0$8c4907e0$@oracle.com> Updated webrev based on feedback from Magnus Ihse Bursie: http://cr.openjdk.java.net/~ctornqvi/webrev/8175300/webrev.01/ -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Christian Tornqvist Sent: Tuesday, March 7, 2017 3:03 PM To: hotspot-dev at openjdk.java.net; 'build-dev' Subject: RFR(M): 8175300 - Enable artifact resolution for jtreg tests Hi everyone, Please review this change that adds the ability to have external dependencies in jtreg tests. As a proof of concept, I've added Scimark to the Hotspot repository. The concept is this, annotate the test with one or more @Artifact annotation describing the external dependencies, these can then either be resolved manually (using -Djdk.test.lib.artifacts.) or automatically using a custom ArtifactManager. The make changes were contributed by Erik Joelsson. These changes have been verified by running the Scimark test locally (using make) and in JPRT. Bugs: https://bugs.openjdk.java.net/browse/JDK-8175300 Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8175300/webrev.00/ Thanks, Christian From ch-hagedorn at hispeed.ch Thu Mar 9 17:25:54 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Thu, 9 Mar 2017 18:25:54 +0100 Subject: How to pin/avoid gc for an oop? Message-ID: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> Hi, I want to implement an undo functionality and thus need to access an oop even when it is out of scope in the Java code. How can I pin such an oop to avoid garbage collection of its used memory? Best regards, Christian From volker.simonis at gmail.com Thu Mar 9 17:52:28 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 9 Mar 2017 18:52:28 +0100 Subject: How to pin/avoid gc for an oop? In-Reply-To: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> Message-ID: Not quite sure what you want to achieve, but with G1 you could allocate a "humongous" object [1,2] which is just as big as a G1 region. It will be allocated into the old generation and not moved around by GC. Regards, Volker [1] http://www.oracle.com/technetwork/articles/java/g1gc-1984535.html [2] https://plumbr.eu/handbook/gc-tuning-in-practice/other-examples/humongous-allocations On Thu, Mar 9, 2017 at 6:25 PM, Christian Hagedorn wrote: > Hi, > > > > I want to implement an undo functionality and thus need to access an oop > even when it is out of scope in the Java code. How can I pin such an oop to > avoid garbage collection of its used memory? > > > > Best regards, > > Christian > From sybersnake at gmail.com Thu Mar 9 18:00:36 2017 From: sybersnake at gmail.com (Jon V.) Date: Thu, 09 Mar 2017 18:00:36 +0000 Subject: How to pin/avoid gc for an oop? In-Reply-To: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> Message-ID: Can you clarify what you mean by Undo and why you think this should be done at the VM level so we can better understand the request. If you don't want something to be garbage collected then simply don't dereference it. On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn wrote: > Hi, > > > > I want to implement an undo functionality and thus need to access an oop > even when it is out of scope in the Java code. How can I pin such an oop to > avoid garbage collection of its used memory? > > > > Best regards, > > Christian > > From kim.barrett at oracle.com Thu Mar 9 18:21:36 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 9 Mar 2017 13:21:36 -0500 Subject: RFR(S) 8176363: Incorrect lock order for G1 PtrQueue related locks In-Reply-To: <81ec5642-e536-8399-53f1-3b4b4443306f@oracle.com> References: <4b5b5844-12d7-062f-a197-c8d091d5f845@oracle.com> <73B19D8E-55B9-4C66-98F5-CEFE1A15A3D9@oracle.com> <81ec5642-e536-8399-53f1-3b4b4443306f@oracle.com> Message-ID: <730AEABC-436B-49A2-8419-83D4F4F91C8A@oracle.com> > On Mar 9, 2017, at 7:01 AM, Mikael Gerdin wrote: > > Hi Kim, > > On 2017-03-08 23:55, Kim Barrett wrote: >> >> Not that it matters, since, as you say, the lock ranking orders are >> still okay, but I don?t recall any free list locks while holding the >> corresponding completed buffer lock. > > Right, but the free list lock is taken when holding the corresponding shared lock. I forgot about that! From mikael.vidstedt at oracle.com Thu Mar 9 18:43:11 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 9 Mar 2017 10:43:11 -0800 Subject: Merging jdk9/hs and jdk9/dev Message-ID: <41F7F709-1613-4ECB-BD05-F41ED2543CA7@oracle.com> All, As you all know JDK 9 development is ramping down, and according to the JDK 9 release schedule [1] we are entering Ramp-down Phase 2 (RDP2) starting next week (March 16). While there is a small number of open issues it is reasonable to believe that the inflow of changes will approach zero over the next few days/weeks. Any changes going forward will effectively be for show stopper type bugs, and must as such be well tested and vetted before they are integrated. Given the state we?re in with the release, and in line with earlier efforts to streamline our integration processes, we propose closing down jdk9/hs [2] and moving the remaining JDK 9 (hotspot) work directly to jdk9/dev [3]. In addition to a reduction in integration overhead, this will also shorten the lead times of getting changes into master in the final latency-sensitive phase of the release. We propose that the final integration from jdk9/hs to jdk9/dev takes place on Thursday, March 23 after which jdk9/hs will be locked down/marked read-only to avoid any accidental integrations. Any work in flight would have to be rebased on jdk9/dev. Just like today, Hotspot integrations will still be made using JPRT. Earlier consolidation projects have gone smoothly, but we will be keeping an extra eye on things and ask for your patience as we work through any issues. Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Thursday, March 16, we will go ahead with the forest consolidation as described. [1] http://openjdk.java.net/projects/jdk9/ [2] http://hg.openjdk.java.net/jdk9/hs [3] http://hg.openjdk.java.net/jdk9/dev From gromero at linux.vnet.ibm.com Thu Mar 9 19:33:26 2017 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 9 Mar 2017 16:33:26 -0300 Subject: [10] RFR (S) 8175813: PPC64: "mbind: Invalid argument" when -XX:+UseNUMA is used Message-ID: <58C1AE06.9060609@linux.vnet.ibm.com> Hi, Could the following webrev be reviewed please? It improves the numa node detection when non-consecutive or memory-less nodes exist in the system. webrev: http://cr.openjdk.java.net/~gromero/8175813/v2/ bug : https://bugs.openjdk.java.net/browse/JDK-8175813 Currently, although no problem exists when the JVM detects numa nodes that are consecutive and have memory, for example in a numa topology like: available: 2 nodes (0-1) node 0 cpus: 0 8 16 24 32 node 0 size: 65258 MB node 0 free: 34 MB node 1 cpus: 40 48 56 64 72 node 1 size: 65320 MB node 1 free: 150 MB node distances: node 0 1 0: 10 20 1: 20 10, it fails on detecting numa nodes to be used in the Parallel GC in a numa topology like: available: 4 nodes (0-1,16-17) node 0 cpus: 0 8 16 24 32 node 0 size: 130706 MB node 0 free: 7729 MB node 1 cpus: 40 48 56 64 72 node 1 size: 0 MB node 1 free: 0 MB node 16 cpus: 80 88 96 104 112 node 16 size: 130630 MB node 16 free: 5282 MB node 17 cpus: 120 128 136 144 152 node 17 size: 0 MB node 17 free: 0 MB node distances: node 0 1 16 17 0: 10 20 40 40 1: 20 10 40 40 16: 40 40 10 20 17: 40 40 20 10, where node 16 is not consecutive in relation to 1 and also nodes 1 and 17 have no memory. If a topology like that exists, os::numa_make_local() will receive a local group id as a hint that is not available in the system to be bound (it will receive all nodes from 0 to 17), causing a proliferation of "mbind: Invalid argument" messages: http://cr.openjdk.java.net/~gromero/logs/jdk10_pristine.log That change improves the detection by making the JVM numa API aware of the existence of numa nodes that are non-consecutive from 0 to the highest node number and also of nodes that might be memory-less nodes, i.e. that might not be, in libnuma terms, a configured node. Hence just the configured nodes will be available: http://cr.openjdk.java.net/~gromero/logs/jdk10_numa_patched.log The change has no effect on numa topologies were the problem does not occur, i.e. no change in the number of nodes and no change in the cpu to node map. On numa topologies where memory-less nodes exist (like in the last example above), cpus from a memory-less node won't be able to bind locally so they are mapped to the closest node, otherwise they would be not associate to any node and MutableNUMASpace::cas_allocate() would pick a node randomly, compromising the performance. I found no regressions on x64 for the following numa topology: available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 8 9 10 11 node 0 size: 24102 MB node 0 free: 19806 MB node 1 cpus: 4 5 6 7 12 13 14 15 node 1 size: 24190 MB node 1 free: 21951 MB node distances: node 0 1 0: 10 21 1: 21 10 I understand that fixing the current numa detection is a prerequisite to enable UseNUMA by the default [1] and to extend the numa-aware allocation to the G1 GC [2]. Thank you. Best regards, Gustavo [1] https://bugs.openjdk.java.net/browse/JDK-8046153 (JEP 163: Enable NUMA Mode by Default When Appropriate) [2] https://bugs.openjdk.java.net/browse/JDK-8046147 (JEP 157: G1 GC: NUMA-Aware Allocation) From sybersnake at gmail.com Thu Mar 9 20:12:22 2017 From: sybersnake at gmail.com (Jon V.) Date: Thu, 09 Mar 2017 20:12:22 +0000 Subject: How to pin/avoid gc for an oop? In-Reply-To: <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> Message-ID: Are you trying to build a transactional memory system for the JVM or your own Java code? On Thu, Mar 9, 2017 at 2:33 PM Christian wrote: > Hi Jon, > > I need this for an undo log of a transactional memory system with > pessimistic locking/eager version management. > Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: > > Can you clarify what you mean by Undo and why you think this should be > done at the VM level so we can better understand the request. > > If you don't want something to be garbage collected then simply don't > dereference it. > > On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn > wrote: > > Hi, > > > > I want to implement an undo functionality and thus need to access an oop > even when it is out of scope in the Java code. How can I pin such an oop to > avoid garbage collection of its used memory? > > > > Best regards, > > Christian > > From magnus.ihse.bursie at oracle.com Fri Mar 10 07:43:16 2017 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 10 Mar 2017 08:43:16 +0100 Subject: RFR(M): 8175300 - Enable artifact resolution for jtreg tests In-Reply-To: <428501d298f4$2ec302a0$8c4907e0$@oracle.com> References: <37ec01d2977d$cfb1fc20$6f15f460$@oracle.com> <428501d298f4$2ec302a0$8c4907e0$@oracle.com> Message-ID: <0c89745b-6dbe-12a4-d66f-698c3bef6dd2@oracle.com> Looks good to me now. /Magnus On 2017-03-09 17:42, Christian Tornqvist wrote: > Updated webrev based on feedback from Magnus Ihse Bursie: > http://cr.openjdk.java.net/~ctornqvi/webrev/8175300/webrev.01/ > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of > Christian Tornqvist > Sent: Tuesday, March 7, 2017 3:03 PM > To: hotspot-dev at openjdk.java.net; 'build-dev' > Subject: RFR(M): 8175300 - Enable artifact resolution for jtreg tests > > Hi everyone, > > > > Please review this change that adds the ability to have external > dependencies in jtreg tests. As a proof of concept, I've added Scimark to > the Hotspot repository. The concept is this, annotate the test with one or > more @Artifact annotation describing the external dependencies, these can > then either be resolved manually (using -Djdk.test.lib.artifacts.) or > automatically using a custom ArtifactManager. > > > > The make changes were contributed by Erik Joelsson. > > These changes have been verified by running the Scimark test locally (using > make) and in JPRT. > > > > Bugs: > > https://bugs.openjdk.java.net/browse/JDK-8175300 > > Webrev: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8175300/webrev.00/ > > > > Thanks, > > Christian > > > > > > > > From serguei.spitsyn at oracle.com Fri Mar 10 10:59:05 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 10 Mar 2017 02:59:05 -0800 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> Message-ID: <81d75202-7012-2cae-f491-0b3b9540c7c2@oracle.com> Hi Shafi, I agree the comment is not needed. Otherwise, it looks good to me. Thanks, Serguei On 3/8/17 21:30, Shafi Ahmad wrote: > Hi David, > > Yes, you are right. > > The comments are not at all needed because of > if (_sample_helper == NULL) return; > > Regards, > Shafi >> -----Original Message----- >> From: David Holmes >> Sent: Thursday, March 09, 2017 10:49 AM >> To: Shafi Ahmad ; Coleen Phillimore >> ; hotspot-dev at openjdk.java.net >> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >> PerfLongVariant::sample >> >> Hi Shafi, >> >> On 9/03/2017 3:08 PM, Shafi Ahmad wrote: >>> Hi, >>> >>> Please find updated webrev link. >>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ >> The more I read this code block, the comment and the commented-out >> assertion: >> >> 215 void PerfLongVariant::sample() { >> 216 >> 217 // This should not happen. Maybe the first sample is taken >> 218 // while the _sample_helper is being null'ed out. >> 219 // assert(_sample_helper != NULL || _sampled != NULL, >> "unexpected state"); >> 220 if (_sample_helper == NULL) return; >> 221 >> 222 *(jlong*)_valuep = _sample_helper->take_sample(); >> 223 } >> >> the less it makes sense to me. ??? >> >> David >> >> >> >> >>> Regards, >>> Shafi >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Thursday, March 09, 2017 7:35 AM >>>> To: Coleen Phillimore ; hotspot- >>>> dev at openjdk.java.net >>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>>> PerfLongVariant::sample >>>> >>>> Hi Coleen, >>>> >>>> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: >>>>> This doesn't look right. >>>>> >>>>> Don't you need to still check for _sampled != NULL? >>>> That use of _sampled was in dead code that is now removed. >>>> >>>> if (_sample_helper == NULL) return; >>>> - if (_sample_helper != NULL) { >>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>> - } >>>> - else if (_sampled != NULL) { >>>> - *(jlong*)_valuep = *_sampled; >>>> - } >>>> } >>>> >>>> This change looks fine to me. >>>> >>>>> Can you remove the JJJ in the comment? >>>> I second that :) >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> I don't know what tests were run >>>>> where Jon (aka JJJ in the code) found this, do you? >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: >>>>>> Hi, >>>>>> >>>>>> May I get the review done for this. >>>>>> >>>>>> Regards, >>>>>> Shafi >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Shafi Ahmad >>>>>>> Sent: Wednesday, March 01, 2017 4:27 PM >>>>>>> To: hotspot-dev at openjdk.java.net >>>>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >>>>>>> PerfLongVariant::sample >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Summary: >>>>>>> It's a very small change to a single file. >>>>>>> >>>>>>> void PerfLongVariant::sample() { >>>>>>> >>>>>>> - assert(_sample_helper != NULL || _sampled != NULL, "unexpected >>>>>>> state"); >>>>>>> + // JJJ - This should not happen. Maybe the first sample is >>>>>>> + taken // while the _sample_helper is being null'ed out. >>>>>>> + // assert(_sample_helper != NULL || _sampled != NULL, >>>>>>> + "unexpected state"); if (_sample_helper == NULL) return; >>>>>>> >>>>>>> if (_sample_helper != NULL) { >>>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>>>> } >>>>>>> else if (_sampled != NULL) { >>>>>>> *(jlong*)_valuep = *_sampled; >>>>>>> } >>>>>>> } >>>>>>> Above five lines are modified in changeset >>>>>>> >>>> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802.1 >>>>>>> 2 Due to the addition of NULL check and return >>>>>>> if (_sample_helper == NULL) return; the else-if block becomes >>>>>>> redundant and statement 'if (_sample_helper != NULL)' is not >> needed. >>>>>>> Webrev link: >>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >>>>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >>>>>>> >>>>>>> Testing: jprt. >>>>>>> >>>>>>> Thanks, >>>>>>> Shafi From shafi.s.ahmad at oracle.com Fri Mar 10 11:13:22 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Fri, 10 Mar 2017 03:13:22 -0800 (PST) Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <81d75202-7012-2cae-f491-0b3b9540c7c2@oracle.com> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> <81d75202-7012-2cae-f491-0b3b9540c7c2@oracle.com> Message-ID: Hi, Thank you Coleen, David and Serguei for the review. Please find updated webrev http://cr.openjdk.java.net/~shshahma/8167425/webrev.02/ Regards, Shafi > -----Original Message----- > From: Serguei Spitsyn > Sent: Friday, March 10, 2017 4:29 PM > To: Shafi Ahmad ; David Holmes > ; Coleen Phillimore > ; hotspot-dev at openjdk.java.net > Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > PerfLongVariant::sample > > Hi Shafi, > > I agree the comment is not needed. > Otherwise, it looks good to me. > > Thanks, > Serguei > > > On 3/8/17 21:30, Shafi Ahmad wrote: > > Hi David, > > > > Yes, you are right. > > > > The comments are not at all needed because of if (_sample_helper == > > NULL) return; > > > > Regards, > > Shafi > >> -----Original Message----- > >> From: David Holmes > >> Sent: Thursday, March 09, 2017 10:49 AM > >> To: Shafi Ahmad ; Coleen Phillimore > >> ; hotspot-dev at openjdk.java.net > >> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > >> PerfLongVariant::sample > >> > >> Hi Shafi, > >> > >> On 9/03/2017 3:08 PM, Shafi Ahmad wrote: > >>> Hi, > >>> > >>> Please find updated webrev link. > >>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ > >> The more I read this code block, the comment and the commented-out > >> assertion: > >> > >> 215 void PerfLongVariant::sample() { > >> 216 > >> 217 // This should not happen. Maybe the first sample is taken > >> 218 // while the _sample_helper is being null'ed out. > >> 219 // assert(_sample_helper != NULL || _sampled != NULL, > >> "unexpected state"); > >> 220 if (_sample_helper == NULL) return; > >> 221 > >> 222 *(jlong*)_valuep = _sample_helper->take_sample(); > >> 223 } > >> > >> the less it makes sense to me. ??? > >> > >> David > >> > >> > >> > >> > >>> Regards, > >>> Shafi > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Thursday, March 09, 2017 7:35 AM > >>>> To: Coleen Phillimore ; hotspot- > >>>> dev at openjdk.java.net > >>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method > >>>> PerfLongVariant::sample > >>>> > >>>> Hi Coleen, > >>>> > >>>> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: > >>>>> This doesn't look right. > >>>>> > >>>>> Don't you need to still check for _sampled != NULL? > >>>> That use of _sampled was in dead code that is now removed. > >>>> > >>>> if (_sample_helper == NULL) return; > >>>> - if (_sample_helper != NULL) { > >>>> *(jlong*)_valuep = _sample_helper->take_sample(); > >>>> - } > >>>> - else if (_sampled != NULL) { > >>>> - *(jlong*)_valuep = *_sampled; > >>>> - } > >>>> } > >>>> > >>>> This change looks fine to me. > >>>> > >>>>> Can you remove the JJJ in the comment? > >>>> I second that :) > >>>> > >>>> Thanks, > >>>> David > >>>> ----- > >>>> > >>>>> I don't know what tests were run where Jon (aka JJJ in the code) > >>>>> found this, do you? > >>>>> > >>>>> Thanks, > >>>>> Coleen > >>>>> > >>>>> > >>>>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: > >>>>>> Hi, > >>>>>> > >>>>>> May I get the review done for this. > >>>>>> > >>>>>> Regards, > >>>>>> Shafi > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Shafi Ahmad > >>>>>>> Sent: Wednesday, March 01, 2017 4:27 PM > >>>>>>> To: hotspot-dev at openjdk.java.net > >>>>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method > >>>>>>> PerfLongVariant::sample > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> Summary: > >>>>>>> It's a very small change to a single file. > >>>>>>> > >>>>>>> void PerfLongVariant::sample() { > >>>>>>> > >>>>>>> - assert(_sample_helper != NULL || _sampled != NULL, > >>>>>>> "unexpected state"); > >>>>>>> + // JJJ - This should not happen. Maybe the first sample is > >>>>>>> + taken // while the _sample_helper is being null'ed out. > >>>>>>> + // assert(_sample_helper != NULL || _sampled != NULL, > >>>>>>> + "unexpected state"); if (_sample_helper == NULL) return; > >>>>>>> > >>>>>>> if (_sample_helper != NULL) { > >>>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); > >>>>>>> } > >>>>>>> else if (_sampled != NULL) { > >>>>>>> *(jlong*)_valuep = *_sampled; > >>>>>>> } > >>>>>>> } > >>>>>>> Above five lines are modified in changeset > >>>>>>> > >>>> > http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802. > >>>> 1 > >>>>>>> 2 Due to the addition of NULL check and return > >>>>>>> if (_sample_helper == NULL) return; the else-if block > >>>>>>> becomes redundant and statement 'if (_sample_helper != NULL)' is > >>>>>>> not > >> needed. > >>>>>>> Webrev link: > >>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ > >>>>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 > >>>>>>> > >>>>>>> Testing: jprt. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Shafi > From david.holmes at oracle.com Fri Mar 10 11:56:06 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 10 Mar 2017 21:56:06 +1000 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> <81d75202-7012-2cae-f491-0b3b9540c7c2@oracle.com> Message-ID: <57a2982e-fb0e-5d35-cbdb-a49339e90515@oracle.com> Hi Shafi, Stylistically this looks very awkward, please rewrite as: void PerfLongVariant::sample() { if (_sample_helper != NULL) { *(jlong*)_valuep = _sample_helper->take_sample(); } } No need for further webrevs. Thanks, David On 10/03/2017 9:13 PM, Shafi Ahmad wrote: > Hi, > > Thank you Coleen, David and Serguei for the review. > > Please find updated webrev > http://cr.openjdk.java.net/~shshahma/8167425/webrev.02/ > > Regards, > Shafi > >> -----Original Message----- >> From: Serguei Spitsyn >> Sent: Friday, March 10, 2017 4:29 PM >> To: Shafi Ahmad ; David Holmes >> ; Coleen Phillimore >> ; hotspot-dev at openjdk.java.net >> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >> PerfLongVariant::sample >> >> Hi Shafi, >> >> I agree the comment is not needed. >> Otherwise, it looks good to me. >> >> Thanks, >> Serguei >> >> >> On 3/8/17 21:30, Shafi Ahmad wrote: >>> Hi David, >>> >>> Yes, you are right. >>> >>> The comments are not at all needed because of if (_sample_helper == >>> NULL) return; >>> >>> Regards, >>> Shafi >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Thursday, March 09, 2017 10:49 AM >>>> To: Shafi Ahmad ; Coleen Phillimore >>>> ; hotspot-dev at openjdk.java.net >>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>>> PerfLongVariant::sample >>>> >>>> Hi Shafi, >>>> >>>> On 9/03/2017 3:08 PM, Shafi Ahmad wrote: >>>>> Hi, >>>>> >>>>> Please find updated webrev link. >>>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ >>>> The more I read this code block, the comment and the commented-out >>>> assertion: >>>> >>>> 215 void PerfLongVariant::sample() { >>>> 216 >>>> 217 // This should not happen. Maybe the first sample is taken >>>> 218 // while the _sample_helper is being null'ed out. >>>> 219 // assert(_sample_helper != NULL || _sampled != NULL, >>>> "unexpected state"); >>>> 220 if (_sample_helper == NULL) return; >>>> 221 >>>> 222 *(jlong*)_valuep = _sample_helper->take_sample(); >>>> 223 } >>>> >>>> the less it makes sense to me. ??? >>>> >>>> David >>>> >>>> >>>> >>>> >>>>> Regards, >>>>> Shafi >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Thursday, March 09, 2017 7:35 AM >>>>>> To: Coleen Phillimore ; hotspot- >>>>>> dev at openjdk.java.net >>>>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>>>>> PerfLongVariant::sample >>>>>> >>>>>> Hi Coleen, >>>>>> >>>>>> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: >>>>>>> This doesn't look right. >>>>>>> >>>>>>> Don't you need to still check for _sampled != NULL? >>>>>> That use of _sampled was in dead code that is now removed. >>>>>> >>>>>> if (_sample_helper == NULL) return; >>>>>> - if (_sample_helper != NULL) { >>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>>> - } >>>>>> - else if (_sampled != NULL) { >>>>>> - *(jlong*)_valuep = *_sampled; >>>>>> - } >>>>>> } >>>>>> >>>>>> This change looks fine to me. >>>>>> >>>>>>> Can you remove the JJJ in the comment? >>>>>> I second that :) >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> I don't know what tests were run where Jon (aka JJJ in the code) >>>>>>> found this, do you? >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> May I get the review done for this. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Shafi >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Shafi Ahmad >>>>>>>>> Sent: Wednesday, March 01, 2017 4:27 PM >>>>>>>>> To: hotspot-dev at openjdk.java.net >>>>>>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >>>>>>>>> PerfLongVariant::sample >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Summary: >>>>>>>>> It's a very small change to a single file. >>>>>>>>> >>>>>>>>> void PerfLongVariant::sample() { >>>>>>>>> >>>>>>>>> - assert(_sample_helper != NULL || _sampled != NULL, >>>>>>>>> "unexpected state"); >>>>>>>>> + // JJJ - This should not happen. Maybe the first sample is >>>>>>>>> + taken // while the _sample_helper is being null'ed out. >>>>>>>>> + // assert(_sample_helper != NULL || _sampled != NULL, >>>>>>>>> + "unexpected state"); if (_sample_helper == NULL) return; >>>>>>>>> >>>>>>>>> if (_sample_helper != NULL) { >>>>>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>>>>>> } >>>>>>>>> else if (_sampled != NULL) { >>>>>>>>> *(jlong*)_valuep = *_sampled; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> Above five lines are modified in changeset >>>>>>>>> >>>>>> >> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802. >>>>>> 1 >>>>>>>>> 2 Due to the addition of NULL check and return >>>>>>>>> if (_sample_helper == NULL) return; the else-if block >>>>>>>>> becomes redundant and statement 'if (_sample_helper != NULL)' is >>>>>>>>> not >>>> needed. >>>>>>>>> Webrev link: >>>>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >>>>>>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >>>>>>>>> >>>>>>>>> Testing: jprt. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Shafi >> From serguei.spitsyn at oracle.com Fri Mar 10 11:58:12 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 10 Mar 2017 03:58:12 -0800 Subject: JDK 10 RFR JDK-8167425: Redundant code in method PerfLongVariant::sample In-Reply-To: <57a2982e-fb0e-5d35-cbdb-a49339e90515@oracle.com> References: <4f8503cd-0552-425c-a9e5-0e7d7b830d52@default> <48fca1ec-9898-473f-8c79-c1a7c3194c70@default> <2e11be0f-7a01-d712-1978-d5dc03703b16@oracle.com> <0ae24e28-61db-8464-6305-36ae29096694@oracle.com> <81d75202-7012-2cae-f491-0b3b9540c7c2@oracle.com> <57a2982e-fb0e-5d35-cbdb-a49339e90515@oracle.com> Message-ID: <16ff0a23-3bcc-766c-5b29-4cc936114a8d@oracle.com> +1 Thanks, Serguei On 3/10/17 03:56, David Holmes wrote: > Hi Shafi, > > Stylistically this looks very awkward, please rewrite as: > > void PerfLongVariant::sample() { > if (_sample_helper != NULL) { > *(jlong*)_valuep = _sample_helper->take_sample(); > } > } > > No need for further webrevs. > > Thanks, > David > > On 10/03/2017 9:13 PM, Shafi Ahmad wrote: >> Hi, >> >> Thank you Coleen, David and Serguei for the review. >> >> Please find updated webrev >> http://cr.openjdk.java.net/~shshahma/8167425/webrev.02/ >> >> Regards, >> Shafi >> >>> -----Original Message----- >>> From: Serguei Spitsyn >>> Sent: Friday, March 10, 2017 4:29 PM >>> To: Shafi Ahmad ; David Holmes >>> ; Coleen Phillimore >>> ; hotspot-dev at openjdk.java.net >>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>> PerfLongVariant::sample >>> >>> Hi Shafi, >>> >>> I agree the comment is not needed. >>> Otherwise, it looks good to me. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 3/8/17 21:30, Shafi Ahmad wrote: >>>> Hi David, >>>> >>>> Yes, you are right. >>>> >>>> The comments are not at all needed because of if (_sample_helper == >>>> NULL) return; >>>> >>>> Regards, >>>> Shafi >>>>> -----Original Message----- >>>>> From: David Holmes >>>>> Sent: Thursday, March 09, 2017 10:49 AM >>>>> To: Shafi Ahmad ; Coleen Phillimore >>>>> ; hotspot-dev at openjdk.java.net >>>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>>>> PerfLongVariant::sample >>>>> >>>>> Hi Shafi, >>>>> >>>>> On 9/03/2017 3:08 PM, Shafi Ahmad wrote: >>>>>> Hi, >>>>>> >>>>>> Please find updated webrev link. >>>>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.01/ >>>>> The more I read this code block, the comment and the commented-out >>>>> assertion: >>>>> >>>>> 215 void PerfLongVariant::sample() { >>>>> 216 >>>>> 217 // This should not happen. Maybe the first sample is taken >>>>> 218 // while the _sample_helper is being null'ed out. >>>>> 219 // assert(_sample_helper != NULL || _sampled != NULL, >>>>> "unexpected state"); >>>>> 220 if (_sample_helper == NULL) return; >>>>> 221 >>>>> 222 *(jlong*)_valuep = _sample_helper->take_sample(); >>>>> 223 } >>>>> >>>>> the less it makes sense to me. ??? >>>>> >>>>> David >>>>> >>>>> >>>>> >>>>> >>>>>> Regards, >>>>>> Shafi >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: David Holmes >>>>>>> Sent: Thursday, March 09, 2017 7:35 AM >>>>>>> To: Coleen Phillimore ; hotspot- >>>>>>> dev at openjdk.java.net >>>>>>> Subject: Re: JDK 10 RFR JDK-8167425: Redundant code in method >>>>>>> PerfLongVariant::sample >>>>>>> >>>>>>> Hi Coleen, >>>>>>> >>>>>>> On 9/03/2017 1:42 AM, coleen.phillimore at oracle.com wrote: >>>>>>>> This doesn't look right. >>>>>>>> >>>>>>>> Don't you need to still check for _sampled != NULL? >>>>>>> That use of _sampled was in dead code that is now removed. >>>>>>> >>>>>>> if (_sample_helper == NULL) return; >>>>>>> - if (_sample_helper != NULL) { >>>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>>>> - } >>>>>>> - else if (_sampled != NULL) { >>>>>>> - *(jlong*)_valuep = *_sampled; >>>>>>> - } >>>>>>> } >>>>>>> >>>>>>> This change looks fine to me. >>>>>>> >>>>>>>> Can you remove the JJJ in the comment? >>>>>>> I second that :) >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> I don't know what tests were run where Jon (aka JJJ in the code) >>>>>>>> found this, do you? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> >>>>>>>> On 3/8/17 10:21 AM, Shafi Ahmad wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> May I get the review done for this. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Shafi >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Shafi Ahmad >>>>>>>>>> Sent: Wednesday, March 01, 2017 4:27 PM >>>>>>>>>> To: hotspot-dev at openjdk.java.net >>>>>>>>>> Subject: FW: JDK 10 RFR JDK-8167425: Redundant code in method >>>>>>>>>> PerfLongVariant::sample >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Summary: >>>>>>>>>> It's a very small change to a single file. >>>>>>>>>> >>>>>>>>>> void PerfLongVariant::sample() { >>>>>>>>>> >>>>>>>>>> - assert(_sample_helper != NULL || _sampled != NULL, >>>>>>>>>> "unexpected state"); >>>>>>>>>> + // JJJ - This should not happen. Maybe the first sample is >>>>>>>>>> + taken // while the _sample_helper is being null'ed out. >>>>>>>>>> + // assert(_sample_helper != NULL || _sampled != NULL, >>>>>>>>>> + "unexpected state"); if (_sample_helper == NULL) return; >>>>>>>>>> >>>>>>>>>> if (_sample_helper != NULL) { >>>>>>>>>> *(jlong*)_valuep = _sample_helper->take_sample(); >>>>>>>>>> } >>>>>>>>>> else if (_sampled != NULL) { >>>>>>>>>> *(jlong*)_valuep = *_sampled; >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> Above five lines are modified in changeset >>>>>>>>>> >>>>>>> >>> http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/da91efe96a93#l802. >>>>>>> 1 >>>>>>>>>> 2 Due to the addition of NULL check and return >>>>>>>>>> if (_sample_helper == NULL) return; the else-if block >>>>>>>>>> becomes redundant and statement 'if (_sample_helper != NULL)' is >>>>>>>>>> not >>>>> needed. >>>>>>>>>> Webrev link: >>>>>>> http://cr.openjdk.java.net/~shshahma/8167425/webrev.00/ >>>>>>>>>> Jdk10 bug link: https://bugs.openjdk.java.net/browse/JDK-8167425 >>>>>>>>>> >>>>>>>>>> Testing: jprt. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Shafi >>> From sybersnake at gmail.com Fri Mar 10 12:45:51 2017 From: sybersnake at gmail.com (Jon V.) Date: Fri, 10 Mar 2017 07:45:51 -0500 Subject: How to pin/avoid gc for an oop? In-Reply-To: <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> Message-ID: My knowledge of the JVM is limited to the GC and memory allocation, but I think I can be helpful. You are correct, switching from InVM to InNative to prevent the GC from running would be a bad idea if you plan on spending a long time inside your atomic block. I don't remember how Handles work. OOPs are managed in a ptr table. When you get a ptr to an OOP it is actually a pointer to the ptr table. Dereferencing it will get you the actual OOP. This is done that way to keep the references valid across GC intervals when the actual object may be moved in memory. I'm not aware of any way to create an OOP where you own the lifecycle. Creating OOPs outside of the JVM lifecycle causes all sorts of errors when the JVM tries to count references. For the reasons above the undo log is going to be expensive in terms of garbage creation. The atomic scope heap undo log might be the only way to do it without requiring extensive modifications to the JVM. On Fri, Mar 10, 2017 at 3:56 AM, Christian wrote: > Hi Jon, > > The STM is to be integrated into the JVM, but the transactional > execution only affects modifications to the managed memory done by > bytecode instructions. JVM internal structures are unaffected. > > Here an example: > > class X { > int f; > > public static update(X x, int _new) { > atomic { // tx start > // (1) update field f of x: > > // undo log maintains a reference to x and old value of f > // note: x contains a direct reference to the lock > > x.f = _new; > > // (2) assume GC is run > // Note: The atomic section can be large and blocking, > // thus, inhibiting a GC run is not an option. > > // (3) End of the atomic section (tx end): > // 1. In case of abort: restore old value > // 2. Release lock (obtained via reference to x) > } > } > } > > How can I assure that the reference to x in the undo log that I made at > (1) remains valid at (3), even though a GC run may occur at (2)? I > thought that Handles (share/vm/runtime/handles.hpp) may be an option, > but I might have misunderstood what they are used for. I can of course > make the undo log in the managed heap as well, but I hoped that this can > be done directly in the JVM. > Am 9. M?rz 2017, um 21:12, "Jon V." schrieb: >> >> Are you trying to build a transactional memory system for the JVM or your >> own Java code? >> >> On Thu, Mar 9, 2017 at 2:33 PM Christian < ch-hagedorn at hispeed.ch> >> wrote: >> >>> Hi Jon, >>> >>> I need this for an undo log of a transactional memory system with >>> pessimistic locking/eager version management. >>> Am 9. M?rz 2017, um 19:00, "Jon V." < sybersnake at gmail.com> schrieb: >>> >>> Can you clarify what you mean by Undo and why you think this should be >>> done at the VM level so we can better understand the request. >>> >>> If you don't want something to be garbage collected then simply don't >>> dereference it. >>> >>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn < >>> ch-hagedorn at hispeed.ch> wrote: >>> >>> Hi, >>> >>> >>> >>> I want to implement an undo functionality and thus need to access an oop >>> even when it is out of scope in the Java code. How can I pin such an oop >>> to >>> avoid garbage collection of its used memory? >>> >>> >>> >>> Best regards, >>> >>> Christian >>> >>> From aph at redhat.com Fri Mar 10 14:55:30 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 10 Mar 2017 14:55:30 +0000 Subject: JDK 9 rampdown and a plea for mercy Message-ID: <9f0423b6-82b1-3bbc-680d-96688cb13b05@redhat.com> As you may remember, I did some work on ByteBuffers a couple of years ago. At the time the generated code was sparklingly good, and I was happy the job was done well. Yesterday I looked at the code we are generating in JDK9, and was horrified to see that it now is a steaming pile of ordure. It might be, of course, that I am mistaken about it being better before, but I really don't think so: I wouldn't have let it go out like that. I suppose it was just good luck (or a C2 bug) that great code was generated. The full gory details are in https://bugs.openjdk.java.net/browse/JDK-8176513, but the bottom line is that unless this regression is fixed all my ByteBuffer work will have been for naught. Roland has had a look at what is happening, and he thinks that this can be fixed fairly quickly: he already has a working patch. Is there any way that we can get it in? One thing for the future: we need to keep an eye on code quality regressions. I'm not sure how, exactly. And perhaps I need to remember that just because C2 generates good code today, it might not generate good code tomorrow. Thanks, Andrew. From george.triantafillou at oracle.com Fri Mar 10 16:53:14 2017 From: george.triantafillou at oracle.com (George Triantafillou) Date: Fri, 10 Mar 2017 11:53:14 -0500 Subject: RFR(M): 8175300 - Enable artifact resolution for jtreg tests In-Reply-To: <428501d298f4$2ec302a0$8c4907e0$@oracle.com> References: <37ec01d2977d$cfb1fc20$6f15f460$@oracle.com> <428501d298f4$2ec302a0$8c4907e0$@oracle.com> Message-ID: <96fb8cb2-8591-c112-33ce-72d15e3d96f7@oracle.com> Hi Christian, Looks good. -George On 3/9/2017 11:42 AM, Christian Tornqvist wrote: > Updated webrev based on feedback from Magnus Ihse Bursie: > http://cr.openjdk.java.net/~ctornqvi/webrev/8175300/webrev.01/ > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of > Christian Tornqvist > Sent: Tuesday, March 7, 2017 3:03 PM > To: hotspot-dev at openjdk.java.net; 'build-dev' > Subject: RFR(M): 8175300 - Enable artifact resolution for jtreg tests > > Hi everyone, > > > > Please review this change that adds the ability to have external > dependencies in jtreg tests. As a proof of concept, I've added Scimark to > the Hotspot repository. The concept is this, annotate the test with one or > more @Artifact annotation describing the external dependencies, these can > then either be resolved manually (using -Djdk.test.lib.artifacts.) or > automatically using a custom ArtifactManager. > > > > The make changes were contributed by Erik Joelsson. > > These changes have been verified by running the Scimark test locally (using > make) and in JPRT. > > > > Bugs: > > https://bugs.openjdk.java.net/browse/JDK-8175300 > > Webrev: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8175300/webrev.00/ > > > > Thanks, > > Christian > > > > > > > > From volker.simonis at gmail.com Fri Mar 10 16:58:30 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 10 Mar 2017 17:58:30 +0100 Subject: JDK 9 rampdown and a plea for mercy In-Reply-To: <9f0423b6-82b1-3bbc-680d-96688cb13b05@redhat.com> References: <9f0423b6-82b1-3bbc-680d-96688cb13b05@redhat.com> Message-ID: On Fri, Mar 10, 2017 at 3:55 PM, Andrew Haley wrote: > As you may remember, I did some work on ByteBuffers a couple of years > ago. At the time the generated code was sparklingly good, and I was > happy the job was done well. Yesterday I looked at the code we are > generating in JDK9, and was horrified to see that it now is a steaming > pile of ordure. It might be, of course, that I am mistaken about it > being better before, but I really don't think so: I wouldn't have let > it go out like that. I suppose it was just good luck (or a C2 bug) > that great code was generated. > > The full gory details are in > https://bugs.openjdk.java.net/browse/JDK-8176513, but the bottom line > is that unless this regression is fixed all my ByteBuffer work will > have been for naught. > > Roland has had a look at what is happening, and he thinks that this > can be fixed fairly quickly: he already has a working patch. Is there > any way that we can get it in? > > One thing for the future: we need to keep an eye on code quality > regressions. I'm not sure how, exactly. And perhaps I need to > remember that just because C2 generates good code today, it might not > generate good code tomorrow. > I totally agree. One possibility would be to expose the OptoAssembly trough the WhiteBox API. That way we could write tests which ensure that certain patterns are really matched in the expected way. It will still be hard to write such tests because they will be mostly platform specific, but at least it could be a start. One problem with the OptoAssembly (i.e. the name of the corresponding nodes) is that they aren't available in the opt/product builds. I think making them available wouldn't be too much of an overhead and could be acceptable. I once had a corresponding change, but never made it available. "JDK-8046030: WhiteBox :: get native code disassembly or optoassembly" [1] tracked a simiklar issue, but was closed :( In our commercial VM we have some tests where we dump all the matched nodes just to ensure that every expected node is still matched. But that's of course just a start for checking that the generated code for a method is good (and remains good): Regards, Volker [1] https://bugs.openjdk.java.net/browse/JDK-8046030 > Thanks, > > Andrew. From ch-hagedorn at hispeed.ch Thu Mar 9 19:32:57 2017 From: ch-hagedorn at hispeed.ch (Christian) Date: Thu, 09 Mar 2017 20:32:57 +0100 Subject: How to pin/avoid gc for an oop? In-Reply-To: References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> Message-ID: <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> Hi Jon, I need this for an undo log of a transactional memory system with pessimistic locking/eager version management. Am 9. M?rz 2017, 19:00, um 19:00, "Jon V." schrieb: >Can you clarify what you mean by Undo and why you think this should be >done >at the VM level so we can better understand the request. > >If you don't want something to be garbage collected then simply don't >dereference it. > >On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn > >wrote: > >> Hi, >> >> >> >> I want to implement an undo functionality and thus need to access an >oop >> even when it is out of scope in the Java code. How can I pin such an >oop to >> avoid garbage collection of its used memory? >> >> >> >> Best regards, >> >> Christian >> >> From ch-hagedorn at hispeed.ch Fri Mar 10 08:56:44 2017 From: ch-hagedorn at hispeed.ch (Christian) Date: Fri, 10 Mar 2017 09:56:44 +0100 Subject: How to pin/avoid gc for an oop? In-Reply-To: References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> Message-ID: <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> Hi Jon, The STM is to be integrated into the JVM, but the transactional execution only affects modifications to the managed memory done by bytecode instructions. JVM internal structures are unaffected. Here an example: class X { ?? int f; ?? public static update(X x, int _new) { ????? atomic { // tx start ???????? // (1) update field f of x: ???????? // undo log maintains a reference to x and old value of f ???????? // note: x contains a direct reference to the lock ???????? x.f = _new; ???????? // (2) assume GC is run ???????? // Note: The atomic section can be large and blocking, ???????? // thus, inhibiting a GC run is not an option. ???????? // (3) End of the atomic section (tx end): ???????? // 1. In case of abort: restore old value ???????? // 2. Release lock (obtained via reference to x) ????? } ?? } } How can I assure that the reference to x in the undo log that I made at (1) remains valid at (3), even though a GC run may occur at (2)? I thought that Handles (share/vm/runtime/handles.hpp) may be an option, but I might have misunderstood what they are used for. I can of course make the undo log in the managed heap as well, but I hoped that this can be done directly in the JVM. Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." schrieb: >Are you trying to build a transactional memory system for the JVM or >your >own Java code? > >On Thu, Mar 9, 2017 at 2:33 PM Christian >wrote: > >> Hi Jon, >> >> I need this for an undo log of a transactional memory system with >> pessimistic locking/eager version management. >> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >> >> Can you clarify what you mean by Undo and why you think this should >be >> done at the VM level so we can better understand the request. >> >> If you don't want something to be garbage collected then simply don't >> dereference it. >> >> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn > >> wrote: >> >> Hi, >> >> >> >> I want to implement an undo functionality and thus need to access an >oop >> even when it is out of scope in the Java code. How can I pin such an >oop to >> avoid garbage collection of its used memory? >> >> >> >> Best regards, >> >> Christian >> >> From uschindler at apache.org Fri Mar 10 18:18:38 2017 From: uschindler at apache.org (Uwe Schindler) Date: Fri, 10 Mar 2017 19:18:38 +0100 Subject: ByteBuffer performance issue in Java 9? Message-ID: <010401d299ca$c198b620$44ca2260$@apache.org> Hi, we just noticed this issue: https://bugs.openjdk.java.net/browse/JDK-8176513 As Apache Lucene relies heavily on performance of ByteBuffers (especially MappedByteBuffer), this would be a desaster if it would get even slower than Java 8. We were so happy that there was much work going on to improve the performance of ByteBuffers so they were at some point almost as fast as byte[]. Because of that we are currently working on making Lucene work fine with Java 9, because once it is out, we would inform all users and recommend to use Java 9 because of these optimizations. The tests of Lucene are already running perfectly, no problems like data corrumption (remember Java 7 GA or 7u40). Also Apache Solr people try to fix the remaining Hadoop/Kerberos-Auth issues because of Jigsaw. We also have unmapping working using Unsafe::invokeCleaner, so we are prepared... I am not yet sure if this bug affects us, we have to run perf tests first. Nevertheless, this is our usage pattern: - we only read from ByteBuffers (MappedByteBuffer), never store anything (we use mmap to execute Lucene searches on the mapped index with often 10th sometimes also 100th of gigabytes of data files that were mmapped). - we do sequential reads (position() and then getByte(), getXxX) - we also do positional reads like this test (for the so called DocValues in Lucene, which is a column based store). This is mainly like sorting search results based on a compare function that accesses a ByteBuffer as random-access source for comparison values. - we don't use IntBuffer, we work directly on ByteBuffer I asked Mike McCandless to run the performance tests with a recent Java 9 version to see if it affects us. Is there any information, when this bug was introduced into Java 9? Nevertheless, please fix it for Java 9 GA, if it affects us it would be a major loss of search performance for us! It would be good to get some information about your plans :-) Thanks! Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ From uschindler at apache.org Fri Mar 10 18:22:55 2017 From: uschindler at apache.org (Uwe Schindler) Date: Fri, 10 Mar 2017 19:22:55 +0100 Subject: JDK 9 rampdown and a plea for mercy In-Reply-To: <9f0423b6-82b1-3bbc-680d-96688cb13b05@redhat.com> References: <9f0423b6-82b1-3bbc-680d-96688cb13b05@redhat.com> Message-ID: <017201d299cb$584148d0$08c3da70$@apache.org> Thanks Andrew, I already sent a similar e-mail about Lucene to the jdk9-dev and hotspot-dev lists... I hope we can fix this! PLEASE, PLEASE! :-) Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Andrew Haley > Sent: Friday, March 10, 2017 3:56 PM > To: John Rose ; Vladimir Kozlov > > Cc: hotspot-dev Source Developers > Subject: JDK 9 rampdown and a plea for mercy > > As you may remember, I did some work on ByteBuffers a couple of years > ago. At the time the generated code was sparklingly good, and I was > happy the job was done well. Yesterday I looked at the code we are > generating in JDK9, and was horrified to see that it now is a steaming > pile of ordure. It might be, of course, that I am mistaken about it > being better before, but I really don't think so: I wouldn't have let > it go out like that. I suppose it was just good luck (or a C2 bug) > that great code was generated. > > The full gory details are in > https://bugs.openjdk.java.net/browse/JDK-8176513, but the bottom line > is that unless this regression is fixed all my ByteBuffer work will > have been for naught. > > Roland has had a look at what is happening, and he thinks that this > can be fixed fairly quickly: he already has a working patch. Is there > any way that we can get it in? > > One thing for the future: we need to keep an eye on code quality > regressions. I'm not sure how, exactly. And perhaps I need to > remember that just because C2 generates good code today, it might not > generate good code tomorrow. > > Thanks, > > Andrew. From aph at redhat.com Fri Mar 10 18:27:03 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 10 Mar 2017 18:27:03 +0000 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: <010401d299ca$c198b620$44ca2260$@apache.org> References: <010401d299ca$c198b620$44ca2260$@apache.org> Message-ID: <80747a62-5483-3a16-17d4-e278d60cbf1e@redhat.com> On 10/03/17 18:18, Uwe Schindler wrote: > It would be good to get some information about your plans :-) Thanks! I believe you should not worry too much. The biggest performance drop seems to be heap-based ByteBuffers rather than off-heap ByteBuffers, which is what you use. The only problem you'll see is that they don't get vectorized, which you can probably live with. On the other hand, heap ByteBuffers are very badly affected. Sorry to cause a panic; I should have explained better. Andrew. From uwe at thetaphi.de Fri Mar 10 18:34:09 2017 From: uwe at thetaphi.de (Uwe Schindler) Date: Fri, 10 Mar 2017 19:34:09 +0100 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: <80747a62-5483-3a16-17d4-e278d60cbf1e@redhat.com> References: <010401d299ca$c198b620$44ca2260$@apache.org> <80747a62-5483-3a16-17d4-e278d60cbf1e@redhat.com> Message-ID: <017c01d299cc$e8862810$b9927830$@thetaphi.de> Hi Andrew, > > It would be good to get some information about your plans :-) Thanks! > > I believe you should not worry too much. The biggest performance drop > seems to be heap-based ByteBuffers rather than off-heap ByteBuffers, > which is what you use. The only problem you'll see is that they don't > get vectorized, which you can probably live with. On the other hand, > heap ByteBuffers are very badly affected. > > Sorry to cause a panic; I should have explained better. No problem, thanks for the explanation. We will of course do some performance tests, this is why I put Mike McCandless on the list of people on this mail. We will report back if we see any significant problems. It depends if vectorization affects us. We have vectorization at other places, but I don't know about our use of ByteBuffers (seems unlikely). Do you know in which build these problems were introduced, so we can have a comparison: - Java 8u121 - Java 9 build ??? - before the problem - Java 9 b159 Uwe From aph at redhat.com Fri Mar 10 18:55:20 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 10 Mar 2017 18:55:20 +0000 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: <017c01d299cc$e8862810$b9927830$@thetaphi.de> References: <010401d299ca$c198b620$44ca2260$@apache.org> <80747a62-5483-3a16-17d4-e278d60cbf1e@redhat.com> <017c01d299cc$e8862810$b9927830$@thetaphi.de> Message-ID: <65b5829c-b14f-a7df-01d3-f710b7919e67@redhat.com> On 10/03/17 18:34, Uwe Schindler wrote: > Do you know in which build these problems were introduced, so we can have a comparison: I have no idea. Andrew. From rwestrel at redhat.com Fri Mar 10 19:54:05 2017 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 10 Mar 2017 20:54:05 +0100 Subject: JDK 9 rampdown and a plea for mercy In-Reply-To: <9f0423b6-82b1-3bbc-680d-96688cb13b05@redhat.com> References: <9f0423b6-82b1-3bbc-680d-96688cb13b05@redhat.com> Message-ID: > Roland has had a look at what is happening, and he thinks that this > can be fixed fairly quickly: he already has a working patch. Is there > any way that we can get it in? The 3 changes I think are needed are: - we add MemBarCPUOrder around accesses when we don't know whether they are on heap or off heap (i.e. if base == null or not). That hurts optimizations a lot. I'd like to enable profiling of arguments at calls to unsafe methods so we can have speculative data on base and can then null check it if profiling tells us it's not null. - We add MemBarCPUOrder around all "mismatched" accesses (integer write to byte array). That seems too strong to me for arrays. I'd like to relax that. - There's an addressing shape that's not supported by superword that prevent vectorization from triggering with ByteBuffers. I think that can be fixed. Roland. From david.holmes at oracle.com Sat Mar 11 03:57:47 2017 From: david.holmes at oracle.com (David Holmes) Date: Sat, 11 Mar 2017 13:57:47 +1000 Subject: How to pin/avoid gc for an oop? In-Reply-To: <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> Message-ID: On 10/03/2017 6:56 PM, Christian wrote: > Hi Jon, > > The STM is to be integrated into the JVM, but the transactional > execution only affects modifications to the managed memory done by > bytecode instructions. JVM internal structures are unaffected. > > Here an example: > > class X { > int f; > > public static update(X x, int _new) { > atomic { // tx start > // (1) update field f of x: > > // undo log maintains a reference to x and old value of f > // note: x contains a direct reference to the lock > > x.f = _new; > > // (2) assume GC is run > // Note: The atomic section can be large and blocking, > // thus, inhibiting a GC run is not an option. > > // (3) End of the atomic section (tx end): > // 1. In case of abort: restore old value > // 2. Release lock (obtained via reference to x) > } > } > } > > How can I assure that the reference to x in the undo log that I made at > (1) remains valid at (3), even though a GC run may occur at (2)? I My making sure the GC sees everything in the undo log. The details will depend on how you propose to implement that log. > thought that Handles (share/vm/runtime/handles.hpp) may be an option, > but I might have misunderstood what they are used for. I can of course Handles are used inside the VM to protect oops when we need to use them across calls that might lead to safepoint checks and thus to GC occurring. > make the undo log in the managed heap as well, but I hoped that this can > be done directly in the JVM. There are a number of ways to do this. If your log is a native VM data structure then it might store Handles. Or you could make the log a GC root and make it known to the GC, and store oops directly. Or you could implement it all in Java - which is probably a lot simpler, albeit likely less performant. David > > Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." schrieb: >> Are you trying to build a transactional memory system for the JVM or >> your >> own Java code? >> >> On Thu, Mar 9, 2017 at 2:33 PM Christian >> wrote: >> >>> Hi Jon, >>> >>> I need this for an undo log of a transactional memory system with >>> pessimistic locking/eager version management. >>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>> >>> Can you clarify what you mean by Undo and why you think this should >> be >>> done at the VM level so we can better understand the request. >>> >>> If you don't want something to be garbage collected then simply don't >>> dereference it. >>> >>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >> >>> wrote: >>> >>> Hi, >>> >>> >>> >>> I want to implement an undo functionality and thus need to access an >> oop >>> even when it is out of scope in the Java code. How can I pin such an >> oop to >>> avoid garbage collection of its used memory? >>> >>> >>> >>> Best regards, >>> >>> Christian >>> >>> From thomas.stuefe at gmail.com Sat Mar 11 09:08:30 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 11 Mar 2017 10:08:30 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: References: Message-ID: Ping... Did I send this to the wrong mailing list? On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe wrote: > (Resent to hotspot-dev) > > On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe > wrote: > >> Hi all, >> >> I would like to get a review for this cleanup change for the >> metaspace.cpp. This has been a long time brewing in my backlog, but I had >> to wait until jdk10 is open. The cleanup is actually quite small, the >> largest part of the change is the added test coding. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >> >> The change implements a suggestion made by Mikael Gerdin when >> reviewing JDK-8170520 last year, who suggested that the ChunkManager class >> should handle returns of both non-humongous and humongous chunks, see >> original discussion here: >> >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >> 016-November/021949.html >> >> "It would be great if you could have a look at unifying the >> ChunkManager::return_chunks API for standard and humongous chunks so >> that we wouldn't need the special casing for this in ~SpaceManager >> That way we could also make humongous_dictionary() private to >> ChunkManager which would improve encapsulation." >> >> The cleanup does this and a bit more, all changes having to do with the >> fact that the ChunkManager class unnecessarily exposes internals to the >> other classes in metaspace.cpp and that with a little bit of reorganization >> this can be avoided. The benefit is simpler coding and way better >> encapsulation, which will help a lot with future changes (in preparation >> for https://bugs.openjdk.java.net/browse/JDK-8166690). >> >> -------- >> >> The changes in detail: >> >> The main issue was that ChunkManager::return_chunks() did not handle >> humongous chunks. To return humongous chunks, one had to access the >> humongous chunk dictionary inside the ChunkManager and add the chunk >> yourself, including maintaining all counters. This happened in >> ~SpaceManager and made this destructor very complicated. >> >> This is solved by moving the handling of humongous chunk returns to the >> ChunkManager class itself. For convenience, I split the old >> "ChunkManager::return_chunks" method into two new ones, >> "ChunkManager::return_single_chunk()" which returns a single chunk to >> the ChunkManager without walking the chunk chain, and >> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >> the ChunkManager. Both functions are now able to handle both non-humongous >> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >> simplified) and just calls "ChunkManager::return_chunk_list()" for all >> its chunk lists. >> >> So now we can remove the public access to the humongous chunk dictionary >> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >> function private. >> >> ---- >> >> Then there was the fact that the non-humongous chunk lists were also >> exposed to public via ChunkManager::free_chunks(). The only reason was that >> when a VirtualSpaceNode is retired, it accessed the ChunkManager free lists >> to find out the size of the items of the free lists. >> >> This was solved by adding a new function, ChunkManager::size_by_index(), >> which returns the size of the items of a chunk list given by its chunk >> index. >> >> So ChunkManager::free_chunks() could be made private too. >> >> --- >> >> The rest are minor changes: >> >> - made ChunkManager::find_free_chunks_list() private because it was not >> used from outside the class >> - Fixed an assert in dec_free_chunks_total() >> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >> to keep the existing logging intact or add useful logging where possible. >> >> ---- >> >> A large chunk of the changes is the added gtests. There is a new class >> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >> code. >> >> Note that I dislike the fact that this test is implemented inside >> metaspace.cpp itself. For now I kept my new tests like the existing tests >> inside this file, but I think these tests should be moved to >> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >> separate fix though, because it needs a bit of remodeling (the tests need >> to access internals in metaspace.cpp). >> >> ---- >> >> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >> AIX. No issues popped up which were associated with my change. >> >> Thanks for reviewing and Kind Regards, Thomas >> >> >> >> >> >> >> >> >> >> >> >> > From ch-hagedorn at hispeed.ch Sat Mar 11 11:08:43 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Sat, 11 Mar 2017 12:08:43 +0100 Subject: AW: How to pin/avoid gc for an oop? In-Reply-To: References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> Message-ID: <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> Thank you for your answers. If it is possible to do it in the JVM I would prefer to do that. I have tried to maintain an array of handles. However I think I have not understood its usage correctly. I only saw examples by allocating it on the stack with Handle h1(thread, oop). But how can I create a handle on the heap such that it is kept alive for an access later? It asserts not to call the global operator "new" such that "new Handle(..)" does not work. How could I make the log a GC root? Christian -----Urspr?ngliche Nachricht----- Von: David Holmes [mailto:david.holmes at oracle.com] Gesendet: Samstag, 11. M?rz 2017 04:58 An: Christian; Jon V. Cc: hotspot-dev at openjdk.java.net Betreff: Re: How to pin/avoid gc for an oop? On 10/03/2017 6:56 PM, Christian wrote: > Hi Jon, > > The STM is to be integrated into the JVM, but the transactional > execution only affects modifications to the managed memory done by > bytecode instructions. JVM internal structures are unaffected. > > Here an example: > > class X { > int f; > > public static update(X x, int _new) { > atomic { // tx start > // (1) update field f of x: > > // undo log maintains a reference to x and old value of f > // note: x contains a direct reference to the lock > > x.f = _new; > > // (2) assume GC is run > // Note: The atomic section can be large and blocking, > // thus, inhibiting a GC run is not an option. > > // (3) End of the atomic section (tx end): > // 1. In case of abort: restore old value > // 2. Release lock (obtained via reference to x) > } > } > } > > How can I assure that the reference to x in the undo log that I made > at > (1) remains valid at (3), even though a GC run may occur at (2)? I My making sure the GC sees everything in the undo log. The details will depend on how you propose to implement that log. > thought that Handles (share/vm/runtime/handles.hpp) may be an option, > but I might have misunderstood what they are used for. I can of course Handles are used inside the VM to protect oops when we need to use them across calls that might lead to safepoint checks and thus to GC occurring. > make the undo log in the managed heap as well, but I hoped that this > can be done directly in the JVM. There are a number of ways to do this. If your log is a native VM data structure then it might store Handles. Or you could make the log a GC root and make it known to the GC, and store oops directly. Or you could implement it all in Java - which is probably a lot simpler, albeit likely less performant. David > > Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." schrieb: >> Are you trying to build a transactional memory system for the JVM or >> your own Java code? >> >> On Thu, Mar 9, 2017 at 2:33 PM Christian >> wrote: >> >>> Hi Jon, >>> >>> I need this for an undo log of a transactional memory system with >>> pessimistic locking/eager version management. >>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>> >>> Can you clarify what you mean by Undo and why you think this should >> be >>> done at the VM level so we can better understand the request. >>> >>> If you don't want something to be garbage collected then simply >>> don't dereference it. >>> >>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >> >>> wrote: >>> >>> Hi, >>> >>> >>> >>> I want to implement an undo functionality and thus need to access an >> oop >>> even when it is out of scope in the Java code. How can I pin such an >> oop to >>> avoid garbage collection of its used memory? >>> >>> >>> >>> Best regards, >>> >>> Christian >>> >>> From david.holmes at oracle.com Sat Mar 11 23:38:54 2017 From: david.holmes at oracle.com (David Holmes) Date: Sun, 12 Mar 2017 09:38:54 +1000 Subject: AW: How to pin/avoid gc for an oop? In-Reply-To: <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> Message-ID: On 11/03/2017 9:08 PM, Christian Hagedorn wrote: > Thank you for your answers. If it is possible to do it in the JVM I would prefer to do that. > > I have tried to maintain an array of handles. However I think I have not understood its usage correctly. I only saw examples by allocating it on the stack with Handle h1(thread, oop). But how can I create a handle on the heap such that it is kept alive for an access later? It asserts not to call the global operator "new" such that "new Handle(..)" does not work. Sorry that was incorrect advice. Handles are per-thread so you can't have a data structure storing them across threads. > How could I make the log a GC root? GC folk should chime in on the details of that - I don't want to give more bad advice. David > Christian > > -----Urspr?ngliche Nachricht----- > Von: David Holmes [mailto:david.holmes at oracle.com] > Gesendet: Samstag, 11. M?rz 2017 04:58 > An: Christian; Jon V. > Cc: hotspot-dev at openjdk.java.net > Betreff: Re: How to pin/avoid gc for an oop? > > On 10/03/2017 6:56 PM, Christian wrote: >> Hi Jon, >> >> The STM is to be integrated into the JVM, but the transactional >> execution only affects modifications to the managed memory done by >> bytecode instructions. JVM internal structures are unaffected. >> >> Here an example: >> >> class X { >> int f; >> >> public static update(X x, int _new) { >> atomic { // tx start >> // (1) update field f of x: >> >> // undo log maintains a reference to x and old value of f >> // note: x contains a direct reference to the lock >> >> x.f = _new; >> >> // (2) assume GC is run >> // Note: The atomic section can be large and blocking, >> // thus, inhibiting a GC run is not an option. >> >> // (3) End of the atomic section (tx end): >> // 1. In case of abort: restore old value >> // 2. Release lock (obtained via reference to x) >> } >> } >> } >> >> How can I assure that the reference to x in the undo log that I made >> at >> (1) remains valid at (3), even though a GC run may occur at (2)? I > > My making sure the GC sees everything in the undo log. The details will depend on how you propose to implement that log. > >> thought that Handles (share/vm/runtime/handles.hpp) may be an option, >> but I might have misunderstood what they are used for. I can of course > > Handles are used inside the VM to protect oops when we need to use them across calls that might lead to safepoint checks and thus to GC occurring. > >> make the undo log in the managed heap as well, but I hoped that this >> can be done directly in the JVM. > > There are a number of ways to do this. If your log is a native VM data structure then it might store Handles. Or you could make the log a GC root and make it known to the GC, and store oops directly. > > Or you could implement it all in Java - which is probably a lot simpler, albeit likely less performant. > > David > >> >> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." schrieb: >>> Are you trying to build a transactional memory system for the JVM or >>> your own Java code? >>> >>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>> wrote: >>> >>>> Hi Jon, >>>> >>>> I need this for an undo log of a transactional memory system with >>>> pessimistic locking/eager version management. >>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>> >>>> Can you clarify what you mean by Undo and why you think this should >>> be >>>> done at the VM level so we can better understand the request. >>>> >>>> If you don't want something to be garbage collected then simply >>>> don't dereference it. >>>> >>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>> >>>> wrote: >>>> >>>> Hi, >>>> >>>> >>>> >>>> I want to implement an undo functionality and thus need to access an >>> oop >>>> even when it is out of scope in the Java code. How can I pin such an >>> oop to >>>> avoid garbage collection of its used memory? >>>> >>>> >>>> >>>> Best regards, >>>> >>>> Christian >>>> >>>> > From ch-hagedorn at hispeed.ch Sun Mar 12 00:52:57 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Sun, 12 Mar 2017 01:52:57 +0100 Subject: AW: AW: How to pin/avoid gc for an oop? In-Reply-To: References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> Message-ID: <000301d29aca$fe862080$fb926180$@hispeed.ch> The undo log would be stored as an array of handles inside a thread and does not refer to other threads. Would that work then? Christian -----Urspr?ngliche Nachricht----- Von: David Holmes [mailto:david.holmes at oracle.com] Gesendet: Sonntag, 12. M?rz 2017 00:39 An: Christian Hagedorn; hotspot-dev at openjdk.java.net Betreff: Re: AW: How to pin/avoid gc for an oop? On 11/03/2017 9:08 PM, Christian Hagedorn wrote: > Thank you for your answers. If it is possible to do it in the JVM I would prefer to do that. > > I have tried to maintain an array of handles. However I think I have not understood its usage correctly. I only saw examples by allocating it on the stack with Handle h1(thread, oop). But how can I create a handle on the heap such that it is kept alive for an access later? It asserts not to call the global operator "new" such that "new Handle(..)" does not work. Sorry that was incorrect advice. Handles are per-thread so you can't have a data structure storing them across threads. > How could I make the log a GC root? GC folk should chime in on the details of that - I don't want to give more bad advice. David > Christian > > -----Urspr?ngliche Nachricht----- > Von: David Holmes [mailto:david.holmes at oracle.com] > Gesendet: Samstag, 11. M?rz 2017 04:58 > An: Christian; Jon V. > Cc: hotspot-dev at openjdk.java.net > Betreff: Re: How to pin/avoid gc for an oop? > > On 10/03/2017 6:56 PM, Christian wrote: >> Hi Jon, >> >> The STM is to be integrated into the JVM, but the transactional >> execution only affects modifications to the managed memory done by >> bytecode instructions. JVM internal structures are unaffected. >> >> Here an example: >> >> class X { >> int f; >> >> public static update(X x, int _new) { >> atomic { // tx start >> // (1) update field f of x: >> >> // undo log maintains a reference to x and old value of f >> // note: x contains a direct reference to the lock >> >> x.f = _new; >> >> // (2) assume GC is run >> // Note: The atomic section can be large and blocking, >> // thus, inhibiting a GC run is not an option. >> >> // (3) End of the atomic section (tx end): >> // 1. In case of abort: restore old value >> // 2. Release lock (obtained via reference to x) >> } >> } >> } >> >> How can I assure that the reference to x in the undo log that I made >> at >> (1) remains valid at (3), even though a GC run may occur at (2)? I > > My making sure the GC sees everything in the undo log. The details will depend on how you propose to implement that log. > >> thought that Handles (share/vm/runtime/handles.hpp) may be an option, >> but I might have misunderstood what they are used for. I can of >> course > > Handles are used inside the VM to protect oops when we need to use them across calls that might lead to safepoint checks and thus to GC occurring. > >> make the undo log in the managed heap as well, but I hoped that this >> can be done directly in the JVM. > > There are a number of ways to do this. If your log is a native VM data structure then it might store Handles. Or you could make the log a GC root and make it known to the GC, and store oops directly. > > Or you could implement it all in Java - which is probably a lot simpler, albeit likely less performant. > > David > >> >> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." schrieb: >>> Are you trying to build a transactional memory system for the JVM or >>> your own Java code? >>> >>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>> wrote: >>> >>>> Hi Jon, >>>> >>>> I need this for an undo log of a transactional memory system with >>>> pessimistic locking/eager version management. >>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>> >>>> Can you clarify what you mean by Undo and why you think this should >>> be >>>> done at the VM level so we can better understand the request. >>>> >>>> If you don't want something to be garbage collected then simply >>>> don't dereference it. >>>> >>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>> >>>> wrote: >>>> >>>> Hi, >>>> >>>> >>>> >>>> I want to implement an undo functionality and thus need to access >>>> an >>> oop >>>> even when it is out of scope in the Java code. How can I pin such >>>> an >>> oop to >>>> avoid garbage collection of its used memory? >>>> >>>> >>>> >>>> Best regards, >>>> >>>> Christian >>>> >>>> > From sybersnake at gmail.com Sun Mar 12 00:59:41 2017 From: sybersnake at gmail.com (Jon V.) Date: Sun, 12 Mar 2017 00:59:41 +0000 Subject: AW: How to pin/avoid gc for an oop? In-Reply-To: <000301d29aca$fe862080$fb926180$@hispeed.ch> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> Message-ID: I don't believe that handles will keep the objects from being garbage collected. Hotspot does not support object pinning. Need input from a GC dev On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn wrote: > The undo log would be stored as an array of handles inside a thread and > does not refer to other threads. Would that work then? > > Christian > > -----Urspr?ngliche Nachricht----- > Von: David Holmes [mailto:david.holmes at oracle.com] > Gesendet: Sonntag, 12. M?rz 2017 00:39 > An: Christian Hagedorn; hotspot-dev at openjdk.java.net > Betreff: Re: AW: How to pin/avoid gc for an oop? > > On 11/03/2017 9:08 PM, Christian Hagedorn wrote: > > Thank you for your answers. If it is possible to do it in the JVM I > would prefer to do that. > > > > I have tried to maintain an array of handles. However I think I have not > understood its usage correctly. I only saw examples by allocating it on the > stack with Handle h1(thread, oop). But how can I create a handle on the > heap such that it is kept alive for an access later? It asserts not to call > the global operator "new" such that "new Handle(..)" does not work. > > Sorry that was incorrect advice. Handles are per-thread so you can't have > a data structure storing them across threads. > > > How could I make the log a GC root? > > GC folk should chime in on the details of that - I don't want to give more > bad advice. > > David > > > Christian > > > > -----Urspr?ngliche Nachricht----- > > Von: David Holmes [mailto:david.holmes at oracle.com] > > Gesendet: Samstag, 11. M?rz 2017 04:58 > > An: Christian; Jon V. > > Cc: hotspot-dev at openjdk.java.net > > Betreff: Re: How to pin/avoid gc for an oop? > > > > On 10/03/2017 6:56 PM, Christian wrote: > >> Hi Jon, > >> > >> The STM is to be integrated into the JVM, but the transactional > >> execution only affects modifications to the managed memory done by > >> bytecode instructions. JVM internal structures are unaffected. > >> > >> Here an example: > >> > >> class X { > >> int f; > >> > >> public static update(X x, int _new) { > >> atomic { // tx start > >> // (1) update field f of x: > >> > >> // undo log maintains a reference to x and old value of f > >> // note: x contains a direct reference to the lock > >> > >> x.f = _new; > >> > >> // (2) assume GC is run > >> // Note: The atomic section can be large and blocking, > >> // thus, inhibiting a GC run is not an option. > >> > >> // (3) End of the atomic section (tx end): > >> // 1. In case of abort: restore old value > >> // 2. Release lock (obtained via reference to x) > >> } > >> } > >> } > >> > >> How can I assure that the reference to x in the undo log that I made > >> at > >> (1) remains valid at (3), even though a GC run may occur at (2)? I > > > > My making sure the GC sees everything in the undo log. The details will > depend on how you propose to implement that log. > > > >> thought that Handles (share/vm/runtime/handles.hpp) may be an option, > >> but I might have misunderstood what they are used for. I can of > >> course > > > > Handles are used inside the VM to protect oops when we need to use them > across calls that might lead to safepoint checks and thus to GC occurring. > > > >> make the undo log in the managed heap as well, but I hoped that this > >> can be done directly in the JVM. > > > > There are a number of ways to do this. If your log is a native VM data > structure then it might store Handles. Or you could make the log a GC root > and make it known to the GC, and store oops directly. > > > > Or you could implement it all in Java - which is probably a lot simpler, > albeit likely less performant. > > > > David > > > >> > >> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." > schrieb: > >>> Are you trying to build a transactional memory system for the JVM or > >>> your own Java code? > >>> > >>> On Thu, Mar 9, 2017 at 2:33 PM Christian > >>> wrote: > >>> > >>>> Hi Jon, > >>>> > >>>> I need this for an undo log of a transactional memory system with > >>>> pessimistic locking/eager version management. > >>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: > >>>> > >>>> Can you clarify what you mean by Undo and why you think this should > >>> be > >>>> done at the VM level so we can better understand the request. > >>>> > >>>> If you don't want something to be garbage collected then simply > >>>> don't dereference it. > >>>> > >>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn > >>> > >>>> wrote: > >>>> > >>>> Hi, > >>>> > >>>> > >>>> > >>>> I want to implement an undo functionality and thus need to access > >>>> an > >>> oop > >>>> even when it is out of scope in the Java code. How can I pin such > >>>> an > >>> oop to > >>>> avoid garbage collection of its used memory? > >>>> > >>>> > >>>> > >>>> Best regards, > >>>> > >>>> Christian > >>>> > >>>> > > > > From david.holmes at oracle.com Sun Mar 12 05:04:30 2017 From: david.holmes at oracle.com (David Holmes) Date: Sun, 12 Mar 2017 15:04:30 +1000 Subject: AW: How to pin/avoid gc for an oop? In-Reply-To: References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> Message-ID: On 12/03/2017 10:59 AM, Jon V. wrote: > I don't believe that handles will keep the objects from being garbage > collected. Handles are GC roots. Or rather the per-thread handle area is a GC root. > Hotspot does not support object pinning. This isn't pinning - the object can still be moved. But the Handle ensures it remains reachable. > Need input from a GC dev > > On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn > wrote: > >> The undo log would be stored as an array of handles inside a thread and >> does not refer to other threads. Would that work then? Handles are intended to be scoped and reclaimed using HandleMarks. This seem to require a different kind of usage. David ----- >> >> Christian >> >> -----Urspr?ngliche Nachricht----- >> Von: David Holmes [mailto:david.holmes at oracle.com] >> Gesendet: Sonntag, 12. M?rz 2017 00:39 >> An: Christian Hagedorn; hotspot-dev at openjdk.java.net >> Betreff: Re: AW: How to pin/avoid gc for an oop? >> >> On 11/03/2017 9:08 PM, Christian Hagedorn wrote: >>> Thank you for your answers. If it is possible to do it in the JVM I >> would prefer to do that. >>> >>> I have tried to maintain an array of handles. However I think I have not >> understood its usage correctly. I only saw examples by allocating it on the >> stack with Handle h1(thread, oop). But how can I create a handle on the >> heap such that it is kept alive for an access later? It asserts not to call >> the global operator "new" such that "new Handle(..)" does not work. >> >> Sorry that was incorrect advice. Handles are per-thread so you can't have >> a data structure storing them across threads. >> >>> How could I make the log a GC root? >> >> GC folk should chime in on the details of that - I don't want to give more >> bad advice. >> >> David >> >>> Christian >>> >>> -----Urspr?ngliche Nachricht----- >>> Von: David Holmes [mailto:david.holmes at oracle.com] >>> Gesendet: Samstag, 11. M?rz 2017 04:58 >>> An: Christian; Jon V. >>> Cc: hotspot-dev at openjdk.java.net >>> Betreff: Re: How to pin/avoid gc for an oop? >>> >>> On 10/03/2017 6:56 PM, Christian wrote: >>>> Hi Jon, >>>> >>>> The STM is to be integrated into the JVM, but the transactional >>>> execution only affects modifications to the managed memory done by >>>> bytecode instructions. JVM internal structures are unaffected. >>>> >>>> Here an example: >>>> >>>> class X { >>>> int f; >>>> >>>> public static update(X x, int _new) { >>>> atomic { // tx start >>>> // (1) update field f of x: >>>> >>>> // undo log maintains a reference to x and old value of f >>>> // note: x contains a direct reference to the lock >>>> >>>> x.f = _new; >>>> >>>> // (2) assume GC is run >>>> // Note: The atomic section can be large and blocking, >>>> // thus, inhibiting a GC run is not an option. >>>> >>>> // (3) End of the atomic section (tx end): >>>> // 1. In case of abort: restore old value >>>> // 2. Release lock (obtained via reference to x) >>>> } >>>> } >>>> } >>>> >>>> How can I assure that the reference to x in the undo log that I made >>>> at >>>> (1) remains valid at (3), even though a GC run may occur at (2)? I >>> >>> My making sure the GC sees everything in the undo log. The details will >> depend on how you propose to implement that log. >>> >>>> thought that Handles (share/vm/runtime/handles.hpp) may be an option, >>>> but I might have misunderstood what they are used for. I can of >>>> course >>> >>> Handles are used inside the VM to protect oops when we need to use them >> across calls that might lead to safepoint checks and thus to GC occurring. >>> >>>> make the undo log in the managed heap as well, but I hoped that this >>>> can be done directly in the JVM. >>> >>> There are a number of ways to do this. If your log is a native VM data >> structure then it might store Handles. Or you could make the log a GC root >> and make it known to the GC, and store oops directly. >>> >>> Or you could implement it all in Java - which is probably a lot simpler, >> albeit likely less performant. >>> >>> David >>> >>>> >>>> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." >> schrieb: >>>>> Are you trying to build a transactional memory system for the JVM or >>>>> your own Java code? >>>>> >>>>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>>>> wrote: >>>>> >>>>>> Hi Jon, >>>>>> >>>>>> I need this for an undo log of a transactional memory system with >>>>>> pessimistic locking/eager version management. >>>>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>>>> >>>>>> Can you clarify what you mean by Undo and why you think this should >>>>> be >>>>>> done at the VM level so we can better understand the request. >>>>>> >>>>>> If you don't want something to be garbage collected then simply >>>>>> don't dereference it. >>>>>> >>>>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>>>> >>>>>> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> >>>>>> I want to implement an undo functionality and thus need to access >>>>>> an >>>>> oop >>>>>> even when it is out of scope in the Java code. How can I pin such >>>>>> an >>>>> oop to >>>>>> avoid garbage collection of its used memory? >>>>>> >>>>>> >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Christian >>>>>> >>>>>> >>> >> >> From coleen.phillimore at oracle.com Sun Mar 12 15:32:27 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Sun, 12 Mar 2017 11:32:27 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles Message-ID: Summary: Use unhandled pointers for Klass and InstanceKlass, remove handles with no implementation. These are fairly extensive changes. The KlassHandle types have been dummy types since permgen elimination and were thought to be useful for future features. They aren't, so can be removed (see bug for more details). This allows stricter typing because you can use the more specific type, and using const more. I didn't add const to these changes, because of the cascading effect. The change isn't hard to review, just tedious. The main bug that I had was redeclaring a type inside a scope, and InstanceKlass::cast(k) can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. It's so nice being able to single step on gdb without going into KlassHandle constructors! open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8155672 Tested with all hotspot jtreg tests, java/lang/invoke tests, java/lang/instrument tests, all closed tonga colocated tests, and JPRT. I can continue to hold this change for a convenient time for merging purposes with other people's JDK10 changes that they've been holding, or if there are other jdk9 changes that are likely to cause a problem for merging. I'll update the copyrights to 2017 on commit. Thanks, Coleen From ch-hagedorn at hispeed.ch Mon Mar 13 00:59:30 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Mon, 13 Mar 2017 01:59:30 +0100 Subject: AW: AW: How to pin/avoid gc for an oop? In-Reply-To: References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> Message-ID: <000001d29b95$1353d820$39fb8860$@hispeed.ch> So handles are not really an option. What else could I do to make my undo log a GC root to ensure reachability of the oops in it? Christian -----Urspr?ngliche Nachricht----- Von: David Holmes [mailto:david.holmes at oracle.com] Gesendet: Sonntag, 12. M?rz 2017 06:05 An: Jon V.; Christian Hagedorn; hotspot-dev at openjdk.java.net Betreff: Re: AW: How to pin/avoid gc for an oop? On 12/03/2017 10:59 AM, Jon V. wrote: > I don't believe that handles will keep the objects from being garbage > collected. Handles are GC roots. Or rather the per-thread handle area is a GC root. > Hotspot does not support object pinning. This isn't pinning - the object can still be moved. But the Handle ensures it remains reachable. > Need input from a GC dev > > On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn > > wrote: > >> The undo log would be stored as an array of handles inside a thread >> and does not refer to other threads. Would that work then? Handles are intended to be scoped and reclaimed using HandleMarks. This seem to require a different kind of usage. David ----- >> >> Christian >> >> -----Urspr?ngliche Nachricht----- >> Von: David Holmes [mailto:david.holmes at oracle.com] >> Gesendet: Sonntag, 12. M?rz 2017 00:39 >> An: Christian Hagedorn; hotspot-dev at openjdk.java.net >> Betreff: Re: AW: How to pin/avoid gc for an oop? >> >> On 11/03/2017 9:08 PM, Christian Hagedorn wrote: >>> Thank you for your answers. If it is possible to do it in the JVM I >> would prefer to do that. >>> >>> I have tried to maintain an array of handles. However I think I have >>> not >> understood its usage correctly. I only saw examples by allocating it >> on the stack with Handle h1(thread, oop). But how can I create a >> handle on the heap such that it is kept alive for an access later? It >> asserts not to call the global operator "new" such that "new Handle(..)" does not work. >> >> Sorry that was incorrect advice. Handles are per-thread so you can't >> have a data structure storing them across threads. >> >>> How could I make the log a GC root? >> >> GC folk should chime in on the details of that - I don't want to give >> more bad advice. >> >> David >> >>> Christian >>> >>> -----Urspr?ngliche Nachricht----- >>> Von: David Holmes [mailto:david.holmes at oracle.com] >>> Gesendet: Samstag, 11. M?rz 2017 04:58 >>> An: Christian; Jon V. >>> Cc: hotspot-dev at openjdk.java.net >>> Betreff: Re: How to pin/avoid gc for an oop? >>> >>> On 10/03/2017 6:56 PM, Christian wrote: >>>> Hi Jon, >>>> >>>> The STM is to be integrated into the JVM, but the transactional >>>> execution only affects modifications to the managed memory done by >>>> bytecode instructions. JVM internal structures are unaffected. >>>> >>>> Here an example: >>>> >>>> class X { >>>> int f; >>>> >>>> public static update(X x, int _new) { >>>> atomic { // tx start >>>> // (1) update field f of x: >>>> >>>> // undo log maintains a reference to x and old value of f >>>> // note: x contains a direct reference to the lock >>>> >>>> x.f = _new; >>>> >>>> // (2) assume GC is run >>>> // Note: The atomic section can be large and blocking, >>>> // thus, inhibiting a GC run is not an option. >>>> >>>> // (3) End of the atomic section (tx end): >>>> // 1. In case of abort: restore old value >>>> // 2. Release lock (obtained via reference to x) >>>> } >>>> } >>>> } >>>> >>>> How can I assure that the reference to x in the undo log that I >>>> made at >>>> (1) remains valid at (3), even though a GC run may occur at (2)? I >>> >>> My making sure the GC sees everything in the undo log. The details >>> will >> depend on how you propose to implement that log. >>> >>>> thought that Handles (share/vm/runtime/handles.hpp) may be an >>>> option, but I might have misunderstood what they are used for. I >>>> can of course >>> >>> Handles are used inside the VM to protect oops when we need to use >>> them >> across calls that might lead to safepoint checks and thus to GC occurring. >>> >>>> make the undo log in the managed heap as well, but I hoped that >>>> this can be done directly in the JVM. >>> >>> There are a number of ways to do this. If your log is a native VM >>> data >> structure then it might store Handles. Or you could make the log a GC >> root and make it known to the GC, and store oops directly. >>> >>> Or you could implement it all in Java - which is probably a lot >>> simpler, >> albeit likely less performant. >>> >>> David >>> >>>> >>>> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." >> schrieb: >>>>> Are you trying to build a transactional memory system for the JVM >>>>> or your own Java code? >>>>> >>>>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>>>> wrote: >>>>> >>>>>> Hi Jon, >>>>>> >>>>>> I need this for an undo log of a transactional memory system with >>>>>> pessimistic locking/eager version management. >>>>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>>>> >>>>>> Can you clarify what you mean by Undo and why you think this >>>>>> should >>>>> be >>>>>> done at the VM level so we can better understand the request. >>>>>> >>>>>> If you don't want something to be garbage collected then simply >>>>>> don't dereference it. >>>>>> >>>>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>>>> >>>>>> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> >>>>>> >>>>>> I want to implement an undo functionality and thus need to access >>>>>> an >>>>> oop >>>>>> even when it is out of scope in the Java code. How can I pin such >>>>>> an >>>>> oop to >>>>>> avoid garbage collection of its used memory? >>>>>> >>>>>> >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Christian >>>>>> >>>>>> >>> >> >> From shafi.s.ahmad at oracle.com Mon Mar 13 03:52:04 2017 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Sun, 12 Mar 2017 20:52:04 -0700 (PDT) Subject: [8u-dev] RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: References: <9d0fe9e5-21e4-b88d-6a9a-263c9eb26d1e@oracle.com> Message-ID: <5b0128b5-54e7-4042-943f-37f552ab8f88@default> Hi, May I get the second review for this simple backport. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Thursday, March 09, 2017 3:39 PM > To: David Holmes ; hotspot- > dev at openjdk.java.net > Subject: RE: [8u-dev] RFR JDK-8171194: Exception "Duplicate field > name&signature in class file" should report the name and signature of the > field > > Hi David, > > Thank you for the review and letting me know about the enhancement > approval process. > > Regards, > Shafi > > > -----Original Message----- > > From: David Holmes > > Sent: Thursday, March 09, 2017 11:46 AM > > To: Shafi Ahmad ; hotspot- > > dev at openjdk.java.net > > Subject: Re: [8u-dev] RFR JDK-8171194: Exception "Duplicate field > > name&signature in class file" should report the name and signature of > > the field > > > > Hi Shafi, > > > > On 9/03/2017 3:39 PM, Shafi Ahmad wrote: > > > Hi, > > > > > > Please review the backport of "JDK-8171194: Exception "Duplicate > > > field > > name&signature in class file" should report the name and signature of > > the field" to jdk8u-dev. > > > The backport is not clean even though the changes are very small. > > > > > > Webrev: http://cr.openjdk.java.net/~shshahma/8176150/webrev.00/ > > > > The code changes look fine to me. > > > > But note that as this is an enhancement you will need to get an > > enhancement approval for it to into 8u, as well as the actual push approval: > > > > http://openjdk.java.net/projects/jdk8u/enhancement-template.html > > > > Thanks, > > David > > > > > Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8171194 > > > Jdk10 review: > > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025 > > > 87 > > > 5.html > > > > > > Testing: jprt and jtreg test. > > > > > > > > > Regards, > > > Shafi > > > From coleen.phillimore at oracle.com Mon Mar 13 04:18:31 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 00:18:31 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: Message-ID: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> As requested by David on the closed part of this review, I fixed variable names of the form h_ik and klass_h where the 'h' indicated that the klass was in a handle. open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev Also, fixing InstanceKlass to not pass this_k can be done in a separate cleanup. Thanks, Coleen On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: > Summary: Use unhandled pointers for Klass and InstanceKlass, remove > handles with no implementation. > > These are fairly extensive changes. The KlassHandle types have been > dummy types since permgen elimination and were thought to be useful > for future features. They aren't, so can be removed (see bug for more > details). This allows stricter typing because you can use the more > specific type, and using const more. I didn't add const to these > changes, because of the cascading effect. > > The change isn't hard to review, just tedious. The main bug that I > had was redeclaring a type inside a scope, and InstanceKlass::cast(k) > can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. > > It's so nice being able to single step on gdb without going into > KlassHandle constructors! > > open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8155672 > > Tested with all hotspot jtreg tests, java/lang/invoke tests, > java/lang/instrument tests, all closed tonga colocated tests, and JPRT. > > I can continue to hold this change for a convenient time for merging > purposes with other people's JDK10 changes that they've been holding, > or if there are other jdk9 changes that are likely to cause a problem > for merging. I'll update the copyrights to 2017 on commit. > > Thanks, > Coleen From david.holmes at oracle.com Mon Mar 13 06:23:32 2017 From: david.holmes at oracle.com (David Holmes) Date: Mon, 13 Mar 2017 16:23:32 +1000 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> Message-ID: <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> Hi Coleen, Wow that was a hard slog! :) On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: > > As requested by David on the closed part of this review, I fixed > variable names of the form h_ik and klass_h where the 'h' indicated that > the klass was in a handle. Thanks for that. But I'm afraid there are quite a number of other patterns including the "h" still present: src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved (though that code uses h_ when there doesn't appear to be a handle in use) src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, InstanceKlass* h_cp_entry_f1 src/share/vm/prims/jni.cpp: Klass* h_k src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* _ikh, InstanceKlass* ikh() src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* instanceK_h src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class src/share/vm/prims/nativeLookup.cpp: Klass* kh src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh > open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev There are numerous code indentation issues with some of the changed code (mainly around parameter/argument lists) - but they all seem to be pre-existing so I decided not to try and call them out. You did fix quite a few, and I don't think you introduced any new ones. :) The main thing to look for with use of handles internal to methods is whether once changed to a direct Klass/InstanceKlass*, the variable becomes redundant. You picked up on a few of these but I spotted a couple of others (reading the patch file the context isn't always sufficient to spot these): share/vm/c1/c1_Runtime1.cpp { Klass* klass = resolve_field_return_klass(caller_method, bci, CHECK); - init_klass = KlassHandle(THREAD, klass); + init_klass = klass; You don't need the klass local but can assign direct to init_klass. // convert to handle - load_klass = KlassHandle(THREAD, k); + load_klass = k; Comment should be deleted, plus it seems you should be able to assign to load_klass directly earlier through the switch instead of introducing 'k'. --- src/share/vm/jvmci/jvmciEnv.cpp 117 Klass* kls; This local is not needed as you can assign directly to found_klass now. --- Finally a meta-question re the changes in: src/share/vm/classfile/dictionary.hpp and the flow out from that. I'm unclear about all the changes from Klass to InstanceKlass. Are all dictionary/hashtable entries guaranteed to be instance classes? --- > Also, fixing InstanceKlass to not pass this_k can be done in a separate > cleanup. Ok. Thanks, David > Thanks, > Coleen > > On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >> handles with no implementation. >> >> These are fairly extensive changes. The KlassHandle types have been >> dummy types since permgen elimination and were thought to be useful >> for future features. They aren't, so can be removed (see bug for more >> details). This allows stricter typing because you can use the more >> specific type, and using const more. I didn't add const to these >> changes, because of the cascading effect. >> >> The change isn't hard to review, just tedious. The main bug that I >> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >> >> It's so nice being able to single step on gdb without going into >> KlassHandle constructors! >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >> >> Tested with all hotspot jtreg tests, java/lang/invoke tests, >> java/lang/instrument tests, all closed tonga colocated tests, and JPRT. >> >> I can continue to hold this change for a convenient time for merging >> purposes with other people's JDK10 changes that they've been holding, >> or if there are other jdk9 changes that are likely to cause a problem >> for merging. I'll update the copyrights to 2017 on commit. >> >> Thanks, >> Coleen > From mikael.gerdin at oracle.com Mon Mar 13 08:25:45 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 13 Mar 2017 09:25:45 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: References: Message-ID: <2f66f12b-52ea-45c3-c51a-26dcbd6e299e@oracle.com> Hi Thomas, This is the correct list but I suspect nobody has yet carved out the time to review this. I'll look at it this week! /Mikael On 2017-03-11 10:08, Thomas St?fe wrote: > Ping... Did I send this to the wrong mailing list? > > On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe > wrote: > >> (Resent to hotspot-dev) >> >> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >> wrote: >> >>> Hi all, >>> >>> I would like to get a review for this cleanup change for the >>> metaspace.cpp. This has been a long time brewing in my backlog, but I had >>> to wait until jdk10 is open. The cleanup is actually quite small, the >>> largest part of the change is the added test coding. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>> >>> The change implements a suggestion made by Mikael Gerdin when >>> reviewing JDK-8170520 last year, who suggested that the ChunkManager class >>> should handle returns of both non-humongous and humongous chunks, see >>> original discussion here: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>> 016-November/021949.html >>> >>> "It would be great if you could have a look at unifying the >>> ChunkManager::return_chunks API for standard and humongous chunks so >>> that we wouldn't need the special casing for this in ~SpaceManager >>> That way we could also make humongous_dictionary() private to >>> ChunkManager which would improve encapsulation." >>> >>> The cleanup does this and a bit more, all changes having to do with the >>> fact that the ChunkManager class unnecessarily exposes internals to the >>> other classes in metaspace.cpp and that with a little bit of reorganization >>> this can be avoided. The benefit is simpler coding and way better >>> encapsulation, which will help a lot with future changes (in preparation >>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>> >>> -------- >>> >>> The changes in detail: >>> >>> The main issue was that ChunkManager::return_chunks() did not handle >>> humongous chunks. To return humongous chunks, one had to access the >>> humongous chunk dictionary inside the ChunkManager and add the chunk >>> yourself, including maintaining all counters. This happened in >>> ~SpaceManager and made this destructor very complicated. >>> >>> This is solved by moving the handling of humongous chunk returns to the >>> ChunkManager class itself. For convenience, I split the old >>> "ChunkManager::return_chunks" method into two new ones, >>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>> the ChunkManager without walking the chunk chain, and >>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>> the ChunkManager. Both functions are now able to handle both non-humongous >>> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >>> simplified) and just calls "ChunkManager::return_chunk_list()" for all >>> its chunk lists. >>> >>> So now we can remove the public access to the humongous chunk dictionary >>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>> function private. >>> >>> ---- >>> >>> Then there was the fact that the non-humongous chunk lists were also >>> exposed to public via ChunkManager::free_chunks(). The only reason was that >>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free lists >>> to find out the size of the items of the free lists. >>> >>> This was solved by adding a new function, ChunkManager::size_by_index(), >>> which returns the size of the items of a chunk list given by its chunk >>> index. >>> >>> So ChunkManager::free_chunks() could be made private too. >>> >>> --- >>> >>> The rest are minor changes: >>> >>> - made ChunkManager::find_free_chunks_list() private because it was not >>> used from outside the class >>> - Fixed an assert in dec_free_chunks_total() >>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >>> to keep the existing logging intact or add useful logging where possible. >>> >>> ---- >>> >>> A large chunk of the changes is the added gtests. There is a new class >>> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >>> code. >>> >>> Note that I dislike the fact that this test is implemented inside >>> metaspace.cpp itself. For now I kept my new tests like the existing tests >>> inside this file, but I think these tests should be moved to >>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>> separate fix though, because it needs a bit of remodeling (the tests need >>> to access internals in metaspace.cpp). >>> >>> ---- >>> >>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >>> AIX. No issues popped up which were associated with my change. >>> >>> Thanks for reviewing and Kind Regards, Thomas >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> From thomas.stuefe at gmail.com Mon Mar 13 08:27:20 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 13 Mar 2017 09:27:20 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: <2f66f12b-52ea-45c3-c51a-26dcbd6e299e@oracle.com> References: <2f66f12b-52ea-45c3-c51a-26dcbd6e299e@oracle.com> Message-ID: Thank you Mikael! On Mon, Mar 13, 2017 at 9:25 AM, Mikael Gerdin wrote: > Hi Thomas, > > This is the correct list but I suspect nobody has yet carved out the time > to review this. > I'll look at it this week! > > /Mikael > > > On 2017-03-11 10:08, Thomas St?fe wrote: > >> Ping... Did I send this to the wrong mailing list? >> >> On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe >> wrote: >> >> (Resent to hotspot-dev) >>> >>> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >>> wrote: >>> >>> Hi all, >>>> >>>> I would like to get a review for this cleanup change for the >>>> metaspace.cpp. This has been a long time brewing in my backlog, but I >>>> had >>>> to wait until jdk10 is open. The cleanup is actually quite small, the >>>> largest part of the change is the added test coding. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>>> >>>> The change implements a suggestion made by Mikael Gerdin when >>>> reviewing JDK-8170520 last year, who suggested that the ChunkManager >>>> class >>>> should handle returns of both non-humongous and humongous chunks, see >>>> original discussion here: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>>> 016-November/021949.html >>>> >>>> "It would be great if you could have a look at unifying the >>>> ChunkManager::return_chunks API for standard and humongous chunks so >>>> that we wouldn't need the special casing for this in ~SpaceManager >>>> That way we could also make humongous_dictionary() private to >>>> ChunkManager which would improve encapsulation." >>>> >>>> The cleanup does this and a bit more, all changes having to do with the >>>> fact that the ChunkManager class unnecessarily exposes internals to the >>>> other classes in metaspace.cpp and that with a little bit of >>>> reorganization >>>> this can be avoided. The benefit is simpler coding and way better >>>> encapsulation, which will help a lot with future changes (in preparation >>>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>>> >>>> -------- >>>> >>>> The changes in detail: >>>> >>>> The main issue was that ChunkManager::return_chunks() did not handle >>>> humongous chunks. To return humongous chunks, one had to access the >>>> humongous chunk dictionary inside the ChunkManager and add the chunk >>>> yourself, including maintaining all counters. This happened in >>>> ~SpaceManager and made this destructor very complicated. >>>> >>>> This is solved by moving the handling of humongous chunk returns to the >>>> ChunkManager class itself. For convenience, I split the old >>>> "ChunkManager::return_chunks" method into two new ones, >>>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>>> the ChunkManager without walking the chunk chain, and >>>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>>> the ChunkManager. Both functions are now able to handle both >>>> non-humongous >>>> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >>>> simplified) and just calls "ChunkManager::return_chunk_list()" for all >>>> its chunk lists. >>>> >>>> So now we can remove the public access to the humongous chunk dictionary >>>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>>> function private. >>>> >>>> ---- >>>> >>>> Then there was the fact that the non-humongous chunk lists were also >>>> exposed to public via ChunkManager::free_chunks(). The only reason was >>>> that >>>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free >>>> lists >>>> to find out the size of the items of the free lists. >>>> >>>> This was solved by adding a new function, ChunkManager::size_by_index(), >>>> which returns the size of the items of a chunk list given by its chunk >>>> index. >>>> >>>> So ChunkManager::free_chunks() could be made private too. >>>> >>>> --- >>>> >>>> The rest are minor changes: >>>> >>>> - made ChunkManager::find_free_chunks_list() private because it was not >>>> used from outside the class >>>> - Fixed an assert in dec_free_chunks_total() >>>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >>>> to keep the existing logging intact or add useful logging where >>>> possible. >>>> >>>> ---- >>>> >>>> A large chunk of the changes is the added gtests. There is a new class >>>> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >>>> code. >>>> >>>> Note that I dislike the fact that this test is implemented inside >>>> metaspace.cpp itself. For now I kept my new tests like the existing >>>> tests >>>> inside this file, but I think these tests should be moved to >>>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>>> separate fix though, because it needs a bit of remodeling (the tests >>>> need >>>> to access internals in metaspace.cpp). >>>> >>>> ---- >>>> >>>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >>>> AIX. No issues popped up which were associated with my change. >>>> >>>> Thanks for reviewing and Kind Regards, Thomas >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> From aph at redhat.com Mon Mar 13 08:37:02 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 13 Mar 2017 08:37:02 +0000 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: References: <010401d299ca$c198b620$44ca2260$@apache.org> Message-ID: On 12/03/17 21:15, Michael McCandless wrote: > I compared Java 1.8.0_121 (base, below) and Java 9 ea 160 (comp), indexing > and searching all Wikipedia English content, using MMapDirectory, and > net/net there is some smallish slowdown, but it's not catastrophic. Right, but ByteBuffers were significantly rewritten for a significant performance *increase*. Any slowdown shows that something has gone very wrong indeed. Andrew. From aph at redhat.com Mon Mar 13 09:50:16 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 13 Mar 2017 09:50:16 +0000 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: References: <010401d299ca$c198b620$44ca2260$@apache.org> Message-ID: On 13/03/17 08:37, Andrew Haley wrote: > On 12/03/17 21:15, Michael McCandless wrote: >> I compared Java 1.8.0_121 (base, below) and Java 9 ea 160 (comp), indexing >> and searching all Wikipedia English content, using MMapDirectory, and >> net/net there is some smallish slowdown, but it's not catastrophic. > > Right, but ByteBuffers were significantly rewritten for a significant > performance *increase*. Just for clarification: heap-based ByteBuffers are most significantly affected. Mapped ByteBuffers see a much smaller, if any, effect, as I said in my earlier post. Andrew. From uschindler at apache.org Mon Mar 13 10:35:14 2017 From: uschindler at apache.org (Uwe Schindler) Date: Mon, 13 Mar 2017 11:35:14 +0100 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: References: <010401d299ca$c198b620$44ca2260$@apache.org> Message-ID: <011201d29be5$8250dfc0$86f29f40$@apache.org> Hi Andrew, yes that was my impression, too. Just for cross-checking: Mike, is it possible to also add a perf comparison between Java 8 and Java 9 when using SimpleFSDirectory or NIOFSDirectory (which are both FileChannel based since Java 7, the name is just backwards-compatibility)? If we see a slowdown there (maybe even larger) than it is not related to ByteBuffer positional/byte-wise reads and there is a general performance issue somewhere else. > Right, but ByteBuffers were significantly rewritten for a significant > performance *increase*. Any slowdown shows that something has gone > very wrong indeed. That would be a pity, because of that we should check the above case with non-mmap based, conventional index access. As far as I remember: at the time when you announced the bytebuffer improvements we did some performance measurements and were impressed by the speedup. I think Robert Muir did something. Mike, do you know? Maybe we should check with a Java 9 preview build from that time. I know that you can download older builds by changing the build number in the download URL. Uwe From thomas.schatzl at oracle.com Mon Mar 13 11:34:21 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 13 Mar 2017 12:34:21 +0100 Subject: AW: AW: How to pin/avoid gc for an oop? In-Reply-To: <000001d29b95$1353d820$39fb8860$@hispeed.ch> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> <000001d29b95$1353d820$39fb8860$@hispeed.ch> Message-ID: <1489404861.3420.30.camel@oracle.com> Hi, On Mon, 2017-03-13 at 01:59 +0100, Christian Hagedorn wrote: > So handles are not really an option. What else could I do to make my > undo log a GC root to ensure reachability of the oops in it? > ? roughly these steps: 1) register (add) the oop you want to keep live during gc to some internal data structure. Depending on requirements like available operations on it, the amount of oops you store there, scalability and other factors, this may be different for you. 2) make sure the GC algorithm of your choice iterates over them during GC and evacuates/fixes up these oops. One example you may look at is how the GC's handle the "Universe's" direct oops. E.g. search for calls to Universe::oops_do() how this works. Since Universe::oops_do() is already called in the right places, you could start experimenting by adding your undo log to the Universe and make sure that the closure passed to Universe::oops_do() is called for every oop in that data structure. That should properly keep the required oops alive across GCs for all but G1. For that, when you modify objects in your undo log you will need to apply its pre-barrier on the original value (All other gcs do not require a pre-barrier, so nothing to do). This boils down to calling G1SDATBCardTableModRefBS::enqueue(old_obj_value) for all modifications (unless you can guarantee that old_obj_value has been NULL). Thanks, ? Thomas From ioi.lam at oracle.com Mon Mar 13 12:23:14 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 13 Mar 2017 20:23:14 +0800 Subject: AW: AW: How to pin/avoid gc for an oop? In-Reply-To: <000001d29b95$1353d820$39fb8860$@hispeed.ch> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> <000001d29b95$1353d820$39fb8860$@hispeed.ch> Message-ID: <58C68F32.7070108@oracle.com> How about using JNI NewGlobalRef/DeleteGlobalRef? Unlike the C++ "Handle" class (which is scoped and automatically reclaimed), a reference created by NewGlobalRef remains valid until it's explicitly deleted using DeleteGlobalRef. - Ioi On 3/13/17 8:59 AM, Christian Hagedorn wrote: > So handles are not really an option. What else could I do to make my undo log a GC root to ensure reachability of the oops in it? > > Christian > > -----Urspr?ngliche Nachricht----- > Von: David Holmes [mailto:david.holmes at oracle.com] > Gesendet: Sonntag, 12. M?rz 2017 06:05 > An: Jon V.; Christian Hagedorn; hotspot-dev at openjdk.java.net > Betreff: Re: AW: How to pin/avoid gc for an oop? > > On 12/03/2017 10:59 AM, Jon V. wrote: >> I don't believe that handles will keep the objects from being garbage >> collected. > Handles are GC roots. Or rather the per-thread handle area is a GC root. > >> Hotspot does not support object pinning. > This isn't pinning - the object can still be moved. But the Handle ensures it remains reachable. > >> Need input from a GC dev >> >> On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn >> >> wrote: >> >>> The undo log would be stored as an array of handles inside a thread >>> and does not refer to other threads. Would that work then? > Handles are intended to be scoped and reclaimed using HandleMarks. This seem to require a different kind of usage. > > David > ----- > >>> Christian >>> >>> -----Urspr?ngliche Nachricht----- >>> Von: David Holmes [mailto:david.holmes at oracle.com] >>> Gesendet: Sonntag, 12. M?rz 2017 00:39 >>> An: Christian Hagedorn; hotspot-dev at openjdk.java.net >>> Betreff: Re: AW: How to pin/avoid gc for an oop? >>> >>> On 11/03/2017 9:08 PM, Christian Hagedorn wrote: >>>> Thank you for your answers. If it is possible to do it in the JVM I >>> would prefer to do that. >>>> I have tried to maintain an array of handles. However I think I have >>>> not >>> understood its usage correctly. I only saw examples by allocating it >>> on the stack with Handle h1(thread, oop). But how can I create a >>> handle on the heap such that it is kept alive for an access later? It >>> asserts not to call the global operator "new" such that "new Handle(..)" does not work. >>> >>> Sorry that was incorrect advice. Handles are per-thread so you can't >>> have a data structure storing them across threads. >>> >>>> How could I make the log a GC root? >>> GC folk should chime in on the details of that - I don't want to give >>> more bad advice. >>> >>> David >>> >>>> Christian >>>> >>>> -----Urspr?ngliche Nachricht----- >>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>> Gesendet: Samstag, 11. M?rz 2017 04:58 >>>> An: Christian; Jon V. >>>> Cc: hotspot-dev at openjdk.java.net >>>> Betreff: Re: How to pin/avoid gc for an oop? >>>> >>>> On 10/03/2017 6:56 PM, Christian wrote: >>>>> Hi Jon, >>>>> >>>>> The STM is to be integrated into the JVM, but the transactional >>>>> execution only affects modifications to the managed memory done by >>>>> bytecode instructions. JVM internal structures are unaffected. >>>>> >>>>> Here an example: >>>>> >>>>> class X { >>>>> int f; >>>>> >>>>> public static update(X x, int _new) { >>>>> atomic { // tx start >>>>> // (1) update field f of x: >>>>> >>>>> // undo log maintains a reference to x and old value of f >>>>> // note: x contains a direct reference to the lock >>>>> >>>>> x.f = _new; >>>>> >>>>> // (2) assume GC is run >>>>> // Note: The atomic section can be large and blocking, >>>>> // thus, inhibiting a GC run is not an option. >>>>> >>>>> // (3) End of the atomic section (tx end): >>>>> // 1. In case of abort: restore old value >>>>> // 2. Release lock (obtained via reference to x) >>>>> } >>>>> } >>>>> } >>>>> >>>>> How can I assure that the reference to x in the undo log that I >>>>> made at >>>>> (1) remains valid at (3), even though a GC run may occur at (2)? I >>>> My making sure the GC sees everything in the undo log. The details >>>> will >>> depend on how you propose to implement that log. >>>>> thought that Handles (share/vm/runtime/handles.hpp) may be an >>>>> option, but I might have misunderstood what they are used for. I >>>>> can of course >>>> Handles are used inside the VM to protect oops when we need to use >>>> them >>> across calls that might lead to safepoint checks and thus to GC occurring. >>>>> make the undo log in the managed heap as well, but I hoped that >>>>> this can be done directly in the JVM. >>>> There are a number of ways to do this. If your log is a native VM >>>> data >>> structure then it might store Handles. Or you could make the log a GC >>> root and make it known to the GC, and store oops directly. >>>> Or you could implement it all in Java - which is probably a lot >>>> simpler, >>> albeit likely less performant. >>>> David >>>> >>>>> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." >>> schrieb: >>>>>> Are you trying to build a transactional memory system for the JVM >>>>>> or your own Java code? >>>>>> >>>>>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>>>>> wrote: >>>>>> >>>>>>> Hi Jon, >>>>>>> >>>>>>> I need this for an undo log of a transactional memory system with >>>>>>> pessimistic locking/eager version management. >>>>>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>>>>> >>>>>>> Can you clarify what you mean by Undo and why you think this >>>>>>> should >>>>>> be >>>>>>> done at the VM level so we can better understand the request. >>>>>>> >>>>>>> If you don't want something to be garbage collected then simply >>>>>>> don't dereference it. >>>>>>> >>>>>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> >>>>>>> I want to implement an undo functionality and thus need to access >>>>>>> an >>>>>> oop >>>>>>> even when it is out of scope in the Java code. How can I pin such >>>>>>> an >>>>>> oop to >>>>>>> avoid garbage collection of its used memory? >>>>>>> >>>>>>> >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> Christian >>>>>>> >>>>>>> >>> From shade at redhat.com Mon Mar 13 12:36:29 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 13 Mar 2017 13:36:29 +0100 Subject: AW: AW: How to pin/avoid gc for an oop? In-Reply-To: <58C68F32.7070108@oracle.com> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> <000001d29b95$1353d820$39fb8860$@hispeed.ch> <58C68F32.7070108@oracle.com> Message-ID: <05517514-3125-c81a-58f7-9d2d12a7f16c@redhat.com> Handle-izing / JNIRef-ing the oop != pinning. a) If you want pinning, you have to cooperate with GC somehow. JNI has "critical" operations for this, so it would be instructive to see jni_GetPrimitiveArrayCritical does this with GCLocker. b) If you just want the reachability, and are in native code, then JNI GlobalRef would probably be the cleanest choice. c) If you just want the reachability, and are in Java code, then calling java.lang.ref.Reference.reachabilityFence would retain the local var in the thread roots. -Aleksey On 03/13/2017 01:23 PM, Ioi Lam wrote: > How about using JNI NewGlobalRef/DeleteGlobalRef? Unlike the C++ "Handle" class > (which is scoped and automatically reclaimed), a reference created by > NewGlobalRef remains valid until it's explicitly deleted using DeleteGlobalRef. > > - Ioi > > On 3/13/17 8:59 AM, Christian Hagedorn wrote: >> So handles are not really an option. What else could I do to make my undo log >> a GC root to ensure reachability of the oops in it? >> >> Christian >> >> -----Urspr?ngliche Nachricht----- >> Von: David Holmes [mailto:david.holmes at oracle.com] >> Gesendet: Sonntag, 12. M?rz 2017 06:05 >> An: Jon V.; Christian Hagedorn; hotspot-dev at openjdk.java.net >> Betreff: Re: AW: How to pin/avoid gc for an oop? >> >> On 12/03/2017 10:59 AM, Jon V. wrote: >>> I don't believe that handles will keep the objects from being garbage >>> collected. >> Handles are GC roots. Or rather the per-thread handle area is a GC root. >> >>> Hotspot does not support object pinning. >> This isn't pinning - the object can still be moved. But the Handle ensures it >> remains reachable. >> >>> Need input from a GC dev >>> >>> On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn >>> >>> wrote: >>> >>>> The undo log would be stored as an array of handles inside a thread >>>> and does not refer to other threads. Would that work then? >> Handles are intended to be scoped and reclaimed using HandleMarks. This seem >> to require a different kind of usage. >> >> David >> ----- >> >>>> Christian >>>> >>>> -----Urspr?ngliche Nachricht----- >>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>> Gesendet: Sonntag, 12. M?rz 2017 00:39 >>>> An: Christian Hagedorn; hotspot-dev at openjdk.java.net >>>> Betreff: Re: AW: How to pin/avoid gc for an oop? >>>> >>>> On 11/03/2017 9:08 PM, Christian Hagedorn wrote: >>>>> Thank you for your answers. If it is possible to do it in the JVM I >>>> would prefer to do that. >>>>> I have tried to maintain an array of handles. However I think I have >>>>> not >>>> understood its usage correctly. I only saw examples by allocating it >>>> on the stack with Handle h1(thread, oop). But how can I create a >>>> handle on the heap such that it is kept alive for an access later? It >>>> asserts not to call the global operator "new" such that "new Handle(..)" >>>> does not work. >>>> >>>> Sorry that was incorrect advice. Handles are per-thread so you can't >>>> have a data structure storing them across threads. >>>> >>>>> How could I make the log a GC root? >>>> GC folk should chime in on the details of that - I don't want to give >>>> more bad advice. >>>> >>>> David >>>> >>>>> Christian >>>>> >>>>> -----Urspr?ngliche Nachricht----- >>>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>>> Gesendet: Samstag, 11. M?rz 2017 04:58 >>>>> An: Christian; Jon V. >>>>> Cc: hotspot-dev at openjdk.java.net >>>>> Betreff: Re: How to pin/avoid gc for an oop? >>>>> >>>>> On 10/03/2017 6:56 PM, Christian wrote: >>>>>> Hi Jon, >>>>>> >>>>>> The STM is to be integrated into the JVM, but the transactional >>>>>> execution only affects modifications to the managed memory done by >>>>>> bytecode instructions. JVM internal structures are unaffected. >>>>>> >>>>>> Here an example: >>>>>> >>>>>> class X { >>>>>> int f; >>>>>> >>>>>> public static update(X x, int _new) { >>>>>> atomic { // tx start >>>>>> // (1) update field f of x: >>>>>> >>>>>> // undo log maintains a reference to x and old value of f >>>>>> // note: x contains a direct reference to the lock >>>>>> >>>>>> x.f = _new; >>>>>> >>>>>> // (2) assume GC is run >>>>>> // Note: The atomic section can be large and blocking, >>>>>> // thus, inhibiting a GC run is not an option. >>>>>> >>>>>> // (3) End of the atomic section (tx end): >>>>>> // 1. In case of abort: restore old value >>>>>> // 2. Release lock (obtained via reference to x) >>>>>> } >>>>>> } >>>>>> } >>>>>> >>>>>> How can I assure that the reference to x in the undo log that I >>>>>> made at >>>>>> (1) remains valid at (3), even though a GC run may occur at (2)? I >>>>> My making sure the GC sees everything in the undo log. The details >>>>> will >>>> depend on how you propose to implement that log. >>>>>> thought that Handles (share/vm/runtime/handles.hpp) may be an >>>>>> option, but I might have misunderstood what they are used for. I >>>>>> can of course >>>>> Handles are used inside the VM to protect oops when we need to use >>>>> them >>>> across calls that might lead to safepoint checks and thus to GC occurring. >>>>>> make the undo log in the managed heap as well, but I hoped that >>>>>> this can be done directly in the JVM. >>>>> There are a number of ways to do this. If your log is a native VM >>>>> data >>>> structure then it might store Handles. Or you could make the log a GC >>>> root and make it known to the GC, and store oops directly. >>>>> Or you could implement it all in Java - which is probably a lot >>>>> simpler, >>>> albeit likely less performant. >>>>> David >>>>> >>>>>> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." >>>> schrieb: >>>>>>> Are you trying to build a transactional memory system for the JVM >>>>>>> or your own Java code? >>>>>>> >>>>>>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Jon, >>>>>>>> >>>>>>>> I need this for an undo log of a transactional memory system with >>>>>>>> pessimistic locking/eager version management. >>>>>>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>>>>>> >>>>>>>> Can you clarify what you mean by Undo and why you think this >>>>>>>> should >>>>>>> be >>>>>>>> done at the VM level so we can better understand the request. >>>>>>>> >>>>>>>> If you don't want something to be garbage collected then simply >>>>>>>> don't dereference it. >>>>>>>> >>>>>>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I want to implement an undo functionality and thus need to access >>>>>>>> an >>>>>>> oop >>>>>>>> even when it is out of scope in the Java code. How can I pin such >>>>>>>> an >>>>>>> oop to >>>>>>>> avoid garbage collection of its used memory? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> Christian >>>>>>>> >>>>>>>> >>>> > From coleen.phillimore at oracle.com Mon Mar 13 12:37:37 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 08:37:37 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> Message-ID: David, I'm so sorry, I should have pointed to open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev On 3/13/17 2:23 AM, David Holmes wrote: > Hi Coleen, > > Wow that was a hard slog! :) > > On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >> >> As requested by David on the closed part of this review, I fixed >> variable names of the form h_ik and klass_h where the 'h' indicated that >> the klass was in a handle. > > Thanks for that. But I'm afraid there are quite a number of other > patterns including the "h" still present: > > src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh > src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h > src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k > src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k > src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k > src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved (though > that code uses h_ when there doesn't appear to be a handle in use) > src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k > src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k > src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, > InstanceKlass* h_cp_entry_f1 > src/share/vm/prims/jni.cpp: Klass* h_k > src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h > src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh > src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* > _ikh, InstanceKlass* ikh() > src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* > instanceK_h > src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh > src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined > src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh > src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh > src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh > src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class > src/share/vm/prims/nativeLookup.cpp: Klass* kh > src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh > src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass > src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh > src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh > These were the ones I fixed and retested. > >> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev > > There are numerous code indentation issues with some of the changed > code (mainly around parameter/argument lists) - but they all seem to > be pre-existing so I decided not to try and call them out. You did fix > quite a few, and I don't think you introduced any new ones. :) > > The main thing to look for with use of handles internal to methods is > whether once changed to a direct Klass/InstanceKlass*, the variable > becomes redundant. You picked up on a few of these but I spotted a > couple of others (reading the patch file the context isn't always > sufficient to spot these): > > share/vm/c1/c1_Runtime1.cpp > > { Klass* klass = resolve_field_return_klass(caller_method, > bci, CHECK); > - init_klass = KlassHandle(THREAD, klass); > + init_klass = klass; > > You don't need the klass local but can assign direct to init_klass. > > // convert to handle > - load_klass = KlassHandle(THREAD, k); > + load_klass = k; > > Comment should be deleted, plus it seems you should be able to assign > to load_klass directly earlier through the switch instead of > introducing 'k'. This logic is not obvious and I'd rather not change it. init_klass and load_klass are used further down for different purposes. I removed the comment though. > > --- > > src/share/vm/jvmci/jvmciEnv.cpp > > 117 Klass* kls; > > This local is not needed as you can assign directly to found_klass now. Yes, that's better. > > --- > > Finally a meta-question re the changes in: > > src/share/vm/classfile/dictionary.hpp > > and the flow out from that. I'm unclear about all the changes from > Klass to InstanceKlass. Are all dictionary/hashtable entries > guaranteed to be instance classes? Yes, they are. This avoided multiple InstanceKlass::cast() calls. > > --- > > >> Also, fixing InstanceKlass to not pass this_k can be done in a separate >> cleanup. > > Ok. Thank you for slogging through all of this. Coleen > > Thanks, > David > >> Thanks, >> Coleen >> >> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>> handles with no implementation. >>> >>> These are fairly extensive changes. The KlassHandle types have been >>> dummy types since permgen elimination and were thought to be useful >>> for future features. They aren't, so can be removed (see bug for more >>> details). This allows stricter typing because you can use the more >>> specific type, and using const more. I didn't add const to these >>> changes, because of the cascading effect. >>> >>> The change isn't hard to review, just tedious. The main bug that I >>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>> >>> It's so nice being able to single step on gdb without going into >>> KlassHandle constructors! >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>> >>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>> java/lang/instrument tests, all closed tonga colocated tests, and JPRT. >>> >>> I can continue to hold this change for a convenient time for merging >>> purposes with other people's JDK10 changes that they've been holding, >>> or if there are other jdk9 changes that are likely to cause a problem >>> for merging. I'll update the copyrights to 2017 on commit. >>> >>> Thanks, >>> Coleen >> From ioi.lam at oracle.com Mon Mar 13 12:43:09 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 13 Mar 2017 20:43:09 +0800 Subject: AW: AW: How to pin/avoid gc for an oop? In-Reply-To: <05517514-3125-c81a-58f7-9d2d12a7f16c@redhat.com> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> <000001d29b95$1353d820$39fb8860$@hispeed.ch> <58C68F32.7070108@oracle.com> <05517514-3125-c81a-58f7-9d2d12a7f16c@redhat.com> Message-ID: <58C693DD.40604@oracle.com> I don't think Christian actually needs object pinning (as in avoiding relocation of an object due to GC). - Ioi On 3/13/17 8:36 PM, Aleksey Shipilev wrote: > Handle-izing / JNIRef-ing the oop != pinning. > > a) If you want pinning, you have to cooperate with GC somehow. JNI has > "critical" operations for this, so it would be instructive to see > jni_GetPrimitiveArrayCritical does this with GCLocker. > > b) If you just want the reachability, and are in native code, then JNI GlobalRef > would probably be the cleanest choice. > > c) If you just want the reachability, and are in Java code, then calling > java.lang.ref.Reference.reachabilityFence would retain the local var in the > thread roots. > > -Aleksey > > On 03/13/2017 01:23 PM, Ioi Lam wrote: >> How about using JNI NewGlobalRef/DeleteGlobalRef? Unlike the C++ "Handle" class >> (which is scoped and automatically reclaimed), a reference created by >> NewGlobalRef remains valid until it's explicitly deleted using DeleteGlobalRef. >> >> - Ioi >> >> On 3/13/17 8:59 AM, Christian Hagedorn wrote: >>> So handles are not really an option. What else could I do to make my undo log >>> a GC root to ensure reachability of the oops in it? >>> >>> Christian >>> >>> -----Urspr?ngliche Nachricht----- >>> Von: David Holmes [mailto:david.holmes at oracle.com] >>> Gesendet: Sonntag, 12. M?rz 2017 06:05 >>> An: Jon V.; Christian Hagedorn; hotspot-dev at openjdk.java.net >>> Betreff: Re: AW: How to pin/avoid gc for an oop? >>> >>> On 12/03/2017 10:59 AM, Jon V. wrote: >>>> I don't believe that handles will keep the objects from being garbage >>>> collected. >>> Handles are GC roots. Or rather the per-thread handle area is a GC root. >>> >>>> Hotspot does not support object pinning. >>> This isn't pinning - the object can still be moved. But the Handle ensures it >>> remains reachable. >>> >>>> Need input from a GC dev >>>> >>>> On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn >>>> >>>> wrote: >>>> >>>>> The undo log would be stored as an array of handles inside a thread >>>>> and does not refer to other threads. Would that work then? >>> Handles are intended to be scoped and reclaimed using HandleMarks. This seem >>> to require a different kind of usage. >>> >>> David >>> ----- >>> >>>>> Christian >>>>> >>>>> -----Urspr?ngliche Nachricht----- >>>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>>> Gesendet: Sonntag, 12. M?rz 2017 00:39 >>>>> An: Christian Hagedorn; hotspot-dev at openjdk.java.net >>>>> Betreff: Re: AW: How to pin/avoid gc for an oop? >>>>> >>>>> On 11/03/2017 9:08 PM, Christian Hagedorn wrote: >>>>>> Thank you for your answers. If it is possible to do it in the JVM I >>>>> would prefer to do that. >>>>>> I have tried to maintain an array of handles. However I think I have >>>>>> not >>>>> understood its usage correctly. I only saw examples by allocating it >>>>> on the stack with Handle h1(thread, oop). But how can I create a >>>>> handle on the heap such that it is kept alive for an access later? It >>>>> asserts not to call the global operator "new" such that "new Handle(..)" >>>>> does not work. >>>>> >>>>> Sorry that was incorrect advice. Handles are per-thread so you can't >>>>> have a data structure storing them across threads. >>>>> >>>>>> How could I make the log a GC root? >>>>> GC folk should chime in on the details of that - I don't want to give >>>>> more bad advice. >>>>> >>>>> David >>>>> >>>>>> Christian >>>>>> >>>>>> -----Urspr?ngliche Nachricht----- >>>>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>>>> Gesendet: Samstag, 11. M?rz 2017 04:58 >>>>>> An: Christian; Jon V. >>>>>> Cc: hotspot-dev at openjdk.java.net >>>>>> Betreff: Re: How to pin/avoid gc for an oop? >>>>>> >>>>>> On 10/03/2017 6:56 PM, Christian wrote: >>>>>>> Hi Jon, >>>>>>> >>>>>>> The STM is to be integrated into the JVM, but the transactional >>>>>>> execution only affects modifications to the managed memory done by >>>>>>> bytecode instructions. JVM internal structures are unaffected. >>>>>>> >>>>>>> Here an example: >>>>>>> >>>>>>> class X { >>>>>>> int f; >>>>>>> >>>>>>> public static update(X x, int _new) { >>>>>>> atomic { // tx start >>>>>>> // (1) update field f of x: >>>>>>> >>>>>>> // undo log maintains a reference to x and old value of f >>>>>>> // note: x contains a direct reference to the lock >>>>>>> >>>>>>> x.f = _new; >>>>>>> >>>>>>> // (2) assume GC is run >>>>>>> // Note: The atomic section can be large and blocking, >>>>>>> // thus, inhibiting a GC run is not an option. >>>>>>> >>>>>>> // (3) End of the atomic section (tx end): >>>>>>> // 1. In case of abort: restore old value >>>>>>> // 2. Release lock (obtained via reference to x) >>>>>>> } >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> How can I assure that the reference to x in the undo log that I >>>>>>> made at >>>>>>> (1) remains valid at (3), even though a GC run may occur at (2)? I >>>>>> My making sure the GC sees everything in the undo log. The details >>>>>> will >>>>> depend on how you propose to implement that log. >>>>>>> thought that Handles (share/vm/runtime/handles.hpp) may be an >>>>>>> option, but I might have misunderstood what they are used for. I >>>>>>> can of course >>>>>> Handles are used inside the VM to protect oops when we need to use >>>>>> them >>>>> across calls that might lead to safepoint checks and thus to GC occurring. >>>>>>> make the undo log in the managed heap as well, but I hoped that >>>>>>> this can be done directly in the JVM. >>>>>> There are a number of ways to do this. If your log is a native VM >>>>>> data >>>>> structure then it might store Handles. Or you could make the log a GC >>>>> root and make it known to the GC, and store oops directly. >>>>>> Or you could implement it all in Java - which is probably a lot >>>>>> simpler, >>>>> albeit likely less performant. >>>>>> David >>>>>> >>>>>>> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." >>>>> schrieb: >>>>>>>> Are you trying to build a transactional memory system for the JVM >>>>>>>> or your own Java code? >>>>>>>> >>>>>>>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Jon, >>>>>>>>> >>>>>>>>> I need this for an undo log of a transactional memory system with >>>>>>>>> pessimistic locking/eager version management. >>>>>>>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>>>>>>> >>>>>>>>> Can you clarify what you mean by Undo and why you think this >>>>>>>>> should >>>>>>>> be >>>>>>>>> done at the VM level so we can better understand the request. >>>>>>>>> >>>>>>>>> If you don't want something to be garbage collected then simply >>>>>>>>> don't dereference it. >>>>>>>>> >>>>>>>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I want to implement an undo functionality and thus need to access >>>>>>>>> an >>>>>>>> oop >>>>>>>>> even when it is out of scope in the Java code. How can I pin such >>>>>>>>> an >>>>>>>> oop to >>>>>>>>> avoid garbage collection of its used memory? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> >>>>>>>>> Christian >>>>>>>>> >>>>>>>>> From lois.foltan at oracle.com Mon Mar 13 14:38:13 2017 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 13 Mar 2017 10:38:13 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> Message-ID: On 3/13/2017 8:37 AM, coleen.phillimore at oracle.com wrote: > > David, I'm so sorry, I should have pointed to > > open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev Coleen, Looks good. A couple of instances of 'h' that I spotted during the review: src/share/vm/aot - a couple of files still have 'h' in use src/share/vm/classfile/systemDictionary.hpp - handle_resolution_exception still has a parameter name of "klass_h" src/share/vm/jvmci/jvmciRuntime.cpp - "InstanceKlass* h =" src/share/vm/services/classLoadingService.hpp - is that really a _klass_handle_array or now a _klass_array? Lois > > On 3/13/17 2:23 AM, David Holmes wrote: >> Hi Coleen, >> >> Wow that was a hard slog! :) >> >> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>> >>> As requested by David on the closed part of this review, I fixed >>> variable names of the form h_ik and klass_h where the 'h' indicated >>> that >>> the klass was in a handle. >> >> Thanks for that. But I'm afraid there are quite a number of other >> patterns including the "h" still present: >> >> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved >> (though that code uses h_ when there doesn't appear to be a handle in >> use) >> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >> InstanceKlass* h_cp_entry_f1 >> src/share/vm/prims/jni.cpp: Klass* h_k >> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh >> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >> _ikh, InstanceKlass* ikh() >> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >> instanceK_h >> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >> src/share/vm/prims/nativeLookup.cpp: Klass* kh >> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >> > > These were the ones I fixed and retested. >> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >> >> There are numerous code indentation issues with some of the changed >> code (mainly around parameter/argument lists) - but they all seem to >> be pre-existing so I decided not to try and call them out. You did >> fix quite a few, and I don't think you introduced any new ones. :) >> >> The main thing to look for with use of handles internal to methods is >> whether once changed to a direct Klass/InstanceKlass*, the variable >> becomes redundant. You picked up on a few of these but I spotted a >> couple of others (reading the patch file the context isn't always >> sufficient to spot these): >> >> share/vm/c1/c1_Runtime1.cpp >> >> { Klass* klass = resolve_field_return_klass(caller_method, >> bci, CHECK); >> - init_klass = KlassHandle(THREAD, klass); >> + init_klass = klass; >> >> You don't need the klass local but can assign direct to init_klass. >> >> // convert to handle >> - load_klass = KlassHandle(THREAD, k); >> + load_klass = k; >> >> Comment should be deleted, plus it seems you should be able to assign >> to load_klass directly earlier through the switch instead of >> introducing 'k'. > > This logic is not obvious and I'd rather not change it. init_klass and > load_klass are used further down for different purposes. I removed > the comment though. >> >> --- >> >> src/share/vm/jvmci/jvmciEnv.cpp >> >> 117 Klass* kls; >> >> This local is not needed as you can assign directly to found_klass now. > > Yes, that's better. >> >> --- >> >> Finally a meta-question re the changes in: >> >> src/share/vm/classfile/dictionary.hpp >> >> and the flow out from that. I'm unclear about all the changes from >> Klass to InstanceKlass. Are all dictionary/hashtable entries >> guaranteed to be instance classes? > > Yes, they are. This avoided multiple InstanceKlass::cast() calls. >> >> --- >> >> >>> Also, fixing InstanceKlass to not pass this_k can be done in a separate >>> cleanup. >> >> Ok. > > Thank you for slogging through all of this. > > Coleen >> >> Thanks, >> David >> >>> Thanks, >>> Coleen >>> >>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>> handles with no implementation. >>>> >>>> These are fairly extensive changes. The KlassHandle types have been >>>> dummy types since permgen elimination and were thought to be useful >>>> for future features. They aren't, so can be removed (see bug for more >>>> details). This allows stricter typing because you can use the more >>>> specific type, and using const more. I didn't add const to these >>>> changes, because of the cascading effect. >>>> >>>> The change isn't hard to review, just tedious. The main bug that I >>>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>>> >>>> It's so nice being able to single step on gdb without going into >>>> KlassHandle constructors! >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>> >>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>> JPRT. >>>> >>>> I can continue to hold this change for a convenient time for merging >>>> purposes with other people's JDK10 changes that they've been holding, >>>> or if there are other jdk9 changes that are likely to cause a problem >>>> for merging. I'll update the copyrights to 2017 on commit. >>>> >>>> Thanks, >>>> Coleen >>> > From mikael.gerdin at oracle.com Mon Mar 13 15:21:04 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 13 Mar 2017 16:21:04 +0100 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> Message-ID: <2355d4cc-d383-fc71-bfa6-4a811219a53a@oracle.com> Hi Coleen, On 2017-03-13 13:37, coleen.phillimore at oracle.com wrote: > > David, I'm so sorry, I should have pointed to > > open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev There are still a bunch of *Klass methods which are static and have an InstanceKlass* (which used to be a handle) as their first parameter. Do you want to leave that to a future cleanup or fold it into this change? Example: 1309 // Static methods that are used to implement member methods where an exposed this pointer 1310 // is needed due to possible GCs 1311 static bool link_class_impl (instanceKlassHandle this_k, bool throw_verifyerror, TRAPS); 1312 static bool verify_code (instanceKlassHandle this_k, bool throw_verifyerror, TRAPS); 1313 static void initialize_impl (instanceKlassHandle this_k, TRAPS); 1314 static void initialize_super_interfaces (instanceKlassHandle this_k, TRAPS); 1315 static void eager_initialize_impl (instanceKlassHandle this_k); 1316 static void set_initialization_state_and_notify_impl (instanceKlassHandle this_k, ClassState state, TRAPS); 1317 static void call_class_initializer_impl (instanceKlassHandle this_k, TRAPS); 1318 static Klass* array_klass_impl (instanceKlassHandle this_k, bool or_null, int n, TRAPS); 1319 static void do_local_static_fields_impl (instanceKlassHandle this_k, void f(fieldDescriptor* fd, Handle, TRAPS), Handle, TRAPS); 1320 /* jni_id_for_impl for jfieldID only */ 1321 static JNIid* jni_id_for_impl (instanceKlassHandle this_k, int offset); 1322 All the above should probably be nonstatic member functions on InstanceKlass instead but had to be made static since a permgen gc could have moved "this" out from under the execution. /Mikael > > On 3/13/17 2:23 AM, David Holmes wrote: >> Hi Coleen, >> >> Wow that was a hard slog! :) >> >> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>> >>> As requested by David on the closed part of this review, I fixed >>> variable names of the form h_ik and klass_h where the 'h' indicated that >>> the klass was in a handle. >> >> Thanks for that. But I'm afraid there are quite a number of other >> patterns including the "h" still present: >> >> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved (though >> that code uses h_ when there doesn't appear to be a handle in use) >> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >> InstanceKlass* h_cp_entry_f1 >> src/share/vm/prims/jni.cpp: Klass* h_k >> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh >> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >> _ikh, InstanceKlass* ikh() >> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >> instanceK_h >> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >> src/share/vm/prims/nativeLookup.cpp: Klass* kh >> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >> > > These were the ones I fixed and retested. >> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >> >> There are numerous code indentation issues with some of the changed >> code (mainly around parameter/argument lists) - but they all seem to >> be pre-existing so I decided not to try and call them out. You did fix >> quite a few, and I don't think you introduced any new ones. :) >> >> The main thing to look for with use of handles internal to methods is >> whether once changed to a direct Klass/InstanceKlass*, the variable >> becomes redundant. You picked up on a few of these but I spotted a >> couple of others (reading the patch file the context isn't always >> sufficient to spot these): >> >> share/vm/c1/c1_Runtime1.cpp >> >> { Klass* klass = resolve_field_return_klass(caller_method, >> bci, CHECK); >> - init_klass = KlassHandle(THREAD, klass); >> + init_klass = klass; >> >> You don't need the klass local but can assign direct to init_klass. >> >> // convert to handle >> - load_klass = KlassHandle(THREAD, k); >> + load_klass = k; >> >> Comment should be deleted, plus it seems you should be able to assign >> to load_klass directly earlier through the switch instead of >> introducing 'k'. > > This logic is not obvious and I'd rather not change it. init_klass and > load_klass are used further down for different purposes. I removed the > comment though. >> >> --- >> >> src/share/vm/jvmci/jvmciEnv.cpp >> >> 117 Klass* kls; >> >> This local is not needed as you can assign directly to found_klass now. > > Yes, that's better. >> >> --- >> >> Finally a meta-question re the changes in: >> >> src/share/vm/classfile/dictionary.hpp >> >> and the flow out from that. I'm unclear about all the changes from >> Klass to InstanceKlass. Are all dictionary/hashtable entries >> guaranteed to be instance classes? > > Yes, they are. This avoided multiple InstanceKlass::cast() calls. >> >> --- >> >> >>> Also, fixing InstanceKlass to not pass this_k can be done in a separate >>> cleanup. >> >> Ok. > > Thank you for slogging through all of this. > > Coleen >> >> Thanks, >> David >> >>> Thanks, >>> Coleen >>> >>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>> handles with no implementation. >>>> >>>> These are fairly extensive changes. The KlassHandle types have been >>>> dummy types since permgen elimination and were thought to be useful >>>> for future features. They aren't, so can be removed (see bug for more >>>> details). This allows stricter typing because you can use the more >>>> specific type, and using const more. I didn't add const to these >>>> changes, because of the cascading effect. >>>> >>>> The change isn't hard to review, just tedious. The main bug that I >>>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>>> >>>> It's so nice being able to single step on gdb without going into >>>> KlassHandle constructors! >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>> >>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>> java/lang/instrument tests, all closed tonga colocated tests, and JPRT. >>>> >>>> I can continue to hold this change for a convenient time for merging >>>> purposes with other people's JDK10 changes that they've been holding, >>>> or if there are other jdk9 changes that are likely to cause a problem >>>> for merging. I'll update the copyrights to 2017 on commit. >>>> >>>> Thanks, >>>> Coleen >>> > From coleen.phillimore at oracle.com Mon Mar 13 15:55:02 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 11:55:02 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> Message-ID: On 3/13/17 10:38 AM, Lois Foltan wrote: > > > On 3/13/2017 8:37 AM, coleen.phillimore at oracle.com wrote: >> >> David, I'm so sorry, I should have pointed to >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev > > Coleen, > Looks good. A couple of instances of 'h' that I spotted during the > review: > > src/share/vm/aot - a couple of files still have 'h' in use > src/share/vm/classfile/systemDictionary.hpp - > handle_resolution_exception still has a parameter name of "klass_h" > src/share/vm/jvmci/jvmciRuntime.cpp - "InstanceKlass* h =" > src/share/vm/services/classLoadingService.hpp - is that really a > _klass_handle_array or now a _klass_array? I fixed all of these to be ik, and classLoadingService now has a _klass_array. Thanks for reviewing this! Coleen > > Lois > >> >> On 3/13/17 2:23 AM, David Holmes wrote: >>> Hi Coleen, >>> >>> Wow that was a hard slog! :) >>> >>> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> As requested by David on the closed part of this review, I fixed >>>> variable names of the form h_ik and klass_h where the 'h' indicated >>>> that >>>> the klass was in a handle. >>> >>> Thanks for that. But I'm afraid there are quite a number of other >>> patterns including the "h" still present: >>> >>> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >>> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >>> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >>> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved >>> (though that code uses h_ when there doesn't appear to be a handle >>> in use) >>> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >>> InstanceKlass* h_cp_entry_f1 >>> src/share/vm/prims/jni.cpp: Klass* h_k >>> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >>> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh >>> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >>> _ikh, InstanceKlass* ikh() >>> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >>> instanceK_h >>> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >>> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >>> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >>> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >>> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >>> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >>> src/share/vm/prims/nativeLookup.cpp: Klass* kh >>> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >>> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >>> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >>> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >>> >> >> These were the ones I fixed and retested. >>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>> >>> There are numerous code indentation issues with some of the changed >>> code (mainly around parameter/argument lists) - but they all seem to >>> be pre-existing so I decided not to try and call them out. You did >>> fix quite a few, and I don't think you introduced any new ones. :) >>> >>> The main thing to look for with use of handles internal to methods >>> is whether once changed to a direct Klass/InstanceKlass*, the >>> variable becomes redundant. You picked up on a few of these but I >>> spotted a couple of others (reading the patch file the context isn't >>> always sufficient to spot these): >>> >>> share/vm/c1/c1_Runtime1.cpp >>> >>> { Klass* klass = resolve_field_return_klass(caller_method, >>> bci, CHECK); >>> - init_klass = KlassHandle(THREAD, klass); >>> + init_klass = klass; >>> >>> You don't need the klass local but can assign direct to init_klass. >>> >>> // convert to handle >>> - load_klass = KlassHandle(THREAD, k); >>> + load_klass = k; >>> >>> Comment should be deleted, plus it seems you should be able to >>> assign to load_klass directly earlier through the switch instead of >>> introducing 'k'. >> >> This logic is not obvious and I'd rather not change it. init_klass >> and load_klass are used further down for different purposes. I >> removed the comment though. >>> >>> --- >>> >>> src/share/vm/jvmci/jvmciEnv.cpp >>> >>> 117 Klass* kls; >>> >>> This local is not needed as you can assign directly to found_klass now. >> >> Yes, that's better. >>> >>> --- >>> >>> Finally a meta-question re the changes in: >>> >>> src/share/vm/classfile/dictionary.hpp >>> >>> and the flow out from that. I'm unclear about all the changes from >>> Klass to InstanceKlass. Are all dictionary/hashtable entries >>> guaranteed to be instance classes? >> >> Yes, they are. This avoided multiple InstanceKlass::cast() calls. >>> >>> --- >>> >>> >>>> Also, fixing InstanceKlass to not pass this_k can be done in a >>>> separate >>>> cleanup. >>> >>> Ok. >> >> Thank you for slogging through all of this. >> >> Coleen >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Coleen >>>> >>>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>>> handles with no implementation. >>>>> >>>>> These are fairly extensive changes. The KlassHandle types have been >>>>> dummy types since permgen elimination and were thought to be useful >>>>> for future features. They aren't, so can be removed (see bug for >>>>> more >>>>> details). This allows stricter typing because you can use the more >>>>> specific type, and using const more. I didn't add const to these >>>>> changes, because of the cascading effect. >>>>> >>>>> The change isn't hard to review, just tedious. The main bug that I >>>>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>>>> >>>>> It's so nice being able to single step on gdb without going into >>>>> KlassHandle constructors! >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>>> >>>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>>> JPRT. >>>>> >>>>> I can continue to hold this change for a convenient time for merging >>>>> purposes with other people's JDK10 changes that they've been holding, >>>>> or if there are other jdk9 changes that are likely to cause a problem >>>>> for merging. I'll update the copyrights to 2017 on commit. >>>>> >>>>> Thanks, >>>>> Coleen >>>> >> > From aph at redhat.com Mon Mar 13 15:58:56 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 13 Mar 2017 15:58:56 +0000 Subject: JDK 9 rampdown and a plea for mercy In-Reply-To: References: <9f0423b6-82b1-3bbc-680d-96688cb13b05@redhat.com> Message-ID: <74ea188b-64ef-666c-8077-9affcdf42b21@redhat.com> On 10/03/17 19:54, Roland Westrelin wrote: > >> Roland has had a look at what is happening, and he thinks that this >> can be fixed fairly quickly: he already has a working patch. Is there >> any way that we can get it in? > > The 3 changes I think are needed are: > > - we add MemBarCPUOrder around accesses when we don't know whether they > are on heap or off heap (i.e. if base == null or not). That hurts > optimizations a lot. I'd like to enable profiling of arguments at > calls to unsafe methods so we can have speculative data on base and > can then null check it if profiling tells us it's not null. > > - We add MemBarCPUOrder around all "mismatched" accesses (integer write > to byte array). That seems too strong to me for arrays. I'd like to > relax that. > > - There's an addressing shape that's not supported by superword that > prevent vectorization from triggering with ByteBuffers. I think that > can be fixed. I think we should concentrate on whatever is the simplest and least-potentially-breaking change. MemBarCPUOrders wreck the performance of most HeapByteBuffer writes so if we can get rid of them we should, IMO. I've certainly heard people say that ByteBuffers are slow, and I really want that problem to go away. Vectorization is the cherry on top of the cake, and I can live without it. Andrew. From coleen.phillimore at oracle.com Mon Mar 13 16:09:24 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 12:09:24 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: <2355d4cc-d383-fc71-bfa6-4a811219a53a@oracle.com> References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> <2355d4cc-d383-fc71-bfa6-4a811219a53a@oracle.com> Message-ID: Hi Mikael, On 3/13/17 11:21 AM, Mikael Gerdin wrote: > Hi Coleen, > > On 2017-03-13 13:37, coleen.phillimore at oracle.com wrote: >> >> David, I'm so sorry, I should have pointed to >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev > > There are still a bunch of *Klass methods which are static and have an > InstanceKlass* (which used to be a handle) as their first parameter. > Do you want to leave that to a future cleanup or fold it into this > change? > > Example: > 1309 // Static methods that are used to implement member methods > where an exposed this pointer > 1310 // is needed due to possible GCs > 1311 static bool link_class_impl (instanceKlassHandle this_k, bool > throw_verifyerror, TRAPS); > 1312 static bool verify_code (instanceKlassHandle this_k, bool > throw_verifyerror, TRAPS); > 1313 static void initialize_impl (instanceKlassHandle this_k, TRAPS); > 1314 static void initialize_super_interfaces (instanceKlassHandle > this_k, TRAPS); > 1315 static void eager_initialize_impl (instanceKlassHandle this_k); > 1316 static void set_initialization_state_and_notify_impl > (instanceKlassHandle this_k, ClassState state, TRAPS); > 1317 static void call_class_initializer_impl (instanceKlassHandle > this_k, TRAPS); > 1318 static Klass* array_klass_impl (instanceKlassHandle this_k, > bool or_null, int n, TRAPS); > 1319 static void do_local_static_fields_impl (instanceKlassHandle > this_k, void f(fieldDescriptor* fd, Handle, TRAPS), Handle, TRAPS); > 1320 /* jni_id_for_impl for jfieldID only */ > 1321 static JNIid* jni_id_for_impl (instanceKlassHandle this_k, int > offset); > 1322 > > All the above should probably be nonstatic member functions on > InstanceKlass instead but had to be made static since a permgen gc > could have moved "this" out from under the execution. I wanted to leave this as a future cleanup, because I think the changes here would less cosmetic and probably would lead to more opportunity to clean things up. Thanks - can I make you reviewer? Coleen > > /Mikael > >> >> On 3/13/17 2:23 AM, David Holmes wrote: >>> Hi Coleen, >>> >>> Wow that was a hard slog! :) >>> >>> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> As requested by David on the closed part of this review, I fixed >>>> variable names of the form h_ik and klass_h where the 'h' indicated >>>> that >>>> the klass was in a handle. >>> >>> Thanks for that. But I'm afraid there are quite a number of other >>> patterns including the "h" still present: >>> >>> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >>> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >>> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >>> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved (though >>> that code uses h_ when there doesn't appear to be a handle in use) >>> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >>> InstanceKlass* h_cp_entry_f1 >>> src/share/vm/prims/jni.cpp: Klass* h_k >>> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >>> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh >>> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >>> _ikh, InstanceKlass* ikh() >>> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >>> instanceK_h >>> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >>> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >>> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >>> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >>> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >>> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >>> src/share/vm/prims/nativeLookup.cpp: Klass* kh >>> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >>> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >>> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >>> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >>> >> >> These were the ones I fixed and retested. >>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>> >>> There are numerous code indentation issues with some of the changed >>> code (mainly around parameter/argument lists) - but they all seem to >>> be pre-existing so I decided not to try and call them out. You did fix >>> quite a few, and I don't think you introduced any new ones. :) >>> >>> The main thing to look for with use of handles internal to methods is >>> whether once changed to a direct Klass/InstanceKlass*, the variable >>> becomes redundant. You picked up on a few of these but I spotted a >>> couple of others (reading the patch file the context isn't always >>> sufficient to spot these): >>> >>> share/vm/c1/c1_Runtime1.cpp >>> >>> { Klass* klass = resolve_field_return_klass(caller_method, >>> bci, CHECK); >>> - init_klass = KlassHandle(THREAD, klass); >>> + init_klass = klass; >>> >>> You don't need the klass local but can assign direct to init_klass. >>> >>> // convert to handle >>> - load_klass = KlassHandle(THREAD, k); >>> + load_klass = k; >>> >>> Comment should be deleted, plus it seems you should be able to assign >>> to load_klass directly earlier through the switch instead of >>> introducing 'k'. >> >> This logic is not obvious and I'd rather not change it. init_klass and >> load_klass are used further down for different purposes. I removed the >> comment though. >>> >>> --- >>> >>> src/share/vm/jvmci/jvmciEnv.cpp >>> >>> 117 Klass* kls; >>> >>> This local is not needed as you can assign directly to found_klass now. >> >> Yes, that's better. >>> >>> --- >>> >>> Finally a meta-question re the changes in: >>> >>> src/share/vm/classfile/dictionary.hpp >>> >>> and the flow out from that. I'm unclear about all the changes from >>> Klass to InstanceKlass. Are all dictionary/hashtable entries >>> guaranteed to be instance classes? >> >> Yes, they are. This avoided multiple InstanceKlass::cast() calls. >>> >>> --- >>> >>> >>>> Also, fixing InstanceKlass to not pass this_k can be done in a >>>> separate >>>> cleanup. >>> >>> Ok. >> >> Thank you for slogging through all of this. >> >> Coleen >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Coleen >>>> >>>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>>> handles with no implementation. >>>>> >>>>> These are fairly extensive changes. The KlassHandle types have been >>>>> dummy types since permgen elimination and were thought to be useful >>>>> for future features. They aren't, so can be removed (see bug for >>>>> more >>>>> details). This allows stricter typing because you can use the more >>>>> specific type, and using const more. I didn't add const to these >>>>> changes, because of the cascading effect. >>>>> >>>>> The change isn't hard to review, just tedious. The main bug that I >>>>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>>>> >>>>> It's so nice being able to single step on gdb without going into >>>>> KlassHandle constructors! >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>>> >>>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>>> JPRT. >>>>> >>>>> I can continue to hold this change for a convenient time for merging >>>>> purposes with other people's JDK10 changes that they've been holding, >>>>> or if there are other jdk9 changes that are likely to cause a problem >>>>> for merging. I'll update the copyrights to 2017 on commit. >>>>> >>>>> Thanks, >>>>> Coleen >>>> >> From mikael.gerdin at oracle.com Mon Mar 13 16:11:36 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 13 Mar 2017 17:11:36 +0100 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> <2355d4cc-d383-fc71-bfa6-4a811219a53a@oracle.com> Message-ID: Hi Coleen, On 2017-03-13 17:09, coleen.phillimore at oracle.com wrote: > Hi Mikael, > > On 3/13/17 11:21 AM, Mikael Gerdin wrote: >> Hi Coleen, >> >> On 2017-03-13 13:37, coleen.phillimore at oracle.com wrote: >>> >>> David, I'm so sorry, I should have pointed to >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev >> >> There are still a bunch of *Klass methods which are static and have an >> InstanceKlass* (which used to be a handle) as their first parameter. >> Do you want to leave that to a future cleanup or fold it into this >> change? >> >> Example: >> 1309 // Static methods that are used to implement member methods >> where an exposed this pointer >> 1310 // is needed due to possible GCs >> 1311 static bool link_class_impl (instanceKlassHandle this_k, bool >> throw_verifyerror, TRAPS); >> 1312 static bool verify_code (instanceKlassHandle this_k, bool >> throw_verifyerror, TRAPS); >> 1313 static void initialize_impl (instanceKlassHandle this_k, TRAPS); >> 1314 static void initialize_super_interfaces (instanceKlassHandle >> this_k, TRAPS); >> 1315 static void eager_initialize_impl (instanceKlassHandle this_k); >> 1316 static void set_initialization_state_and_notify_impl >> (instanceKlassHandle this_k, ClassState state, TRAPS); >> 1317 static void call_class_initializer_impl (instanceKlassHandle >> this_k, TRAPS); >> 1318 static Klass* array_klass_impl (instanceKlassHandle this_k, >> bool or_null, int n, TRAPS); >> 1319 static void do_local_static_fields_impl (instanceKlassHandle >> this_k, void f(fieldDescriptor* fd, Handle, TRAPS), Handle, TRAPS); >> 1320 /* jni_id_for_impl for jfieldID only */ >> 1321 static JNIid* jni_id_for_impl (instanceKlassHandle this_k, int >> offset); >> 1322 >> >> All the above should probably be nonstatic member functions on >> InstanceKlass instead but had to be made static since a permgen gc >> could have moved "this" out from under the execution. > > I wanted to leave this as a future cleanup, because I think the changes > here would less cosmetic and probably would lead to more opportunity to > clean things up. Ok, fine by me. > > Thanks - can I make you reviewer? I only looked at instanceKlass.hpp/cpp to see if you had done the change I asked about so I think not :) Thanks for doing this cleanup! /Mikael > > Coleen > >> >> /Mikael >> >>> >>> On 3/13/17 2:23 AM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> Wow that was a hard slog! :) >>>> >>>> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> As requested by David on the closed part of this review, I fixed >>>>> variable names of the form h_ik and klass_h where the 'h' indicated >>>>> that >>>>> the klass was in a handle. >>>> >>>> Thanks for that. But I'm afraid there are quite a number of other >>>> patterns including the "h" still present: >>>> >>>> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >>>> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >>>> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >>>> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >>>> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >>>> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved (though >>>> that code uses h_ when there doesn't appear to be a handle in use) >>>> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >>>> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >>>> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >>>> InstanceKlass* h_cp_entry_f1 >>>> src/share/vm/prims/jni.cpp: Klass* h_k >>>> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >>>> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh >>>> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >>>> _ikh, InstanceKlass* ikh() >>>> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >>>> instanceK_h >>>> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >>>> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >>>> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >>>> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >>>> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >>>> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >>>> src/share/vm/prims/nativeLookup.cpp: Klass* kh >>>> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >>>> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >>>> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >>>> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >>>> >>> >>> These were the ones I fixed and retested. >>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>> >>>> There are numerous code indentation issues with some of the changed >>>> code (mainly around parameter/argument lists) - but they all seem to >>>> be pre-existing so I decided not to try and call them out. You did fix >>>> quite a few, and I don't think you introduced any new ones. :) >>>> >>>> The main thing to look for with use of handles internal to methods is >>>> whether once changed to a direct Klass/InstanceKlass*, the variable >>>> becomes redundant. You picked up on a few of these but I spotted a >>>> couple of others (reading the patch file the context isn't always >>>> sufficient to spot these): >>>> >>>> share/vm/c1/c1_Runtime1.cpp >>>> >>>> { Klass* klass = resolve_field_return_klass(caller_method, >>>> bci, CHECK); >>>> - init_klass = KlassHandle(THREAD, klass); >>>> + init_klass = klass; >>>> >>>> You don't need the klass local but can assign direct to init_klass. >>>> >>>> // convert to handle >>>> - load_klass = KlassHandle(THREAD, k); >>>> + load_klass = k; >>>> >>>> Comment should be deleted, plus it seems you should be able to assign >>>> to load_klass directly earlier through the switch instead of >>>> introducing 'k'. >>> >>> This logic is not obvious and I'd rather not change it. init_klass and >>> load_klass are used further down for different purposes. I removed the >>> comment though. >>>> >>>> --- >>>> >>>> src/share/vm/jvmci/jvmciEnv.cpp >>>> >>>> 117 Klass* kls; >>>> >>>> This local is not needed as you can assign directly to found_klass now. >>> >>> Yes, that's better. >>>> >>>> --- >>>> >>>> Finally a meta-question re the changes in: >>>> >>>> src/share/vm/classfile/dictionary.hpp >>>> >>>> and the flow out from that. I'm unclear about all the changes from >>>> Klass to InstanceKlass. Are all dictionary/hashtable entries >>>> guaranteed to be instance classes? >>> >>> Yes, they are. This avoided multiple InstanceKlass::cast() calls. >>>> >>>> --- >>>> >>>> >>>>> Also, fixing InstanceKlass to not pass this_k can be done in a >>>>> separate >>>>> cleanup. >>>> >>>> Ok. >>> >>> Thank you for slogging through all of this. >>> >>> Coleen >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>>>> handles with no implementation. >>>>>> >>>>>> These are fairly extensive changes. The KlassHandle types have been >>>>>> dummy types since permgen elimination and were thought to be useful >>>>>> for future features. They aren't, so can be removed (see bug for >>>>>> more >>>>>> details). This allows stricter typing because you can use the more >>>>>> specific type, and using const more. I didn't add const to these >>>>>> changes, because of the cascading effect. >>>>>> >>>>>> The change isn't hard to review, just tedious. The main bug that I >>>>>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>>>>> >>>>>> It's so nice being able to single step on gdb without going into >>>>>> KlassHandle constructors! >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>>>> >>>>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>>>> JPRT. >>>>>> >>>>>> I can continue to hold this change for a convenient time for merging >>>>>> purposes with other people's JDK10 changes that they've been holding, >>>>>> or if there are other jdk9 changes that are likely to cause a problem >>>>>> for merging. I'll update the copyrights to 2017 on commit. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>> >>> > From dl at cs.oswego.edu Mon Mar 13 16:19:34 2017 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 13 Mar 2017 12:19:34 -0400 Subject: JDK 9 VarHandle memory mode usage guide Message-ID: <4d391897-a774-d45b-36bb-85e824502536@cs.oswego.edu> [Reposting from concurrency-interest list] As may recall, JDK 9 VarHandles with new memory modes were incorporated without a full overhaul of the Java Memory Model that would be needed to formally specify them. For the past year or so I've been sporadically working on an expert-programmer guide that conveys as much information as possible while skirting the issues (like out-of-thin-air) that led to stalls in revising the JMM. Considering that there's now no chance for a JLS JMM update to occur before JDK 9 release, this might be the we can do. The current draft is available at http://gee.cs.oswego.edu/dl/html/j9mm.html If you are on this list, you are probably in the main target audience for this guide. Someday, a less imposing overview might be nice. Plus more examples. -Doug From ch-hagedorn at hispeed.ch Mon Mar 13 16:27:03 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Mon, 13 Mar 2017 17:27:03 +0100 Subject: AW: AW: AW: How to pin/avoid gc for an oop? In-Reply-To: <58C693DD.40604@oracle.com> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> <000001d29b95$1353d820$39fb8860$@hispeed.ch> <58C68F32.7070108@oracle.com> <05517514-3125-c81a-58f7-9d2d12a7f16c@redhat.com> <58C693DD.40604@oracle.com> Message-ID: <000001d29c16$a6b85ec0$f4291c40$@hispeed.ch> Thank you all for your help and your suggestions! :) That is right I only need a way to still reach it but not actually pin it. It seems to be working by creating a global ref with JNIHandles::make_global(handle), which is exactly what the jni_NewGlobalRef function does pointed out by Ioi. The GC did not remove the oops. Christian -----Urspr?ngliche Nachricht----- Von: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] Im Auftrag von Ioi Lam Gesendet: Montag, 13. M?rz 2017 13:43 An: Aleksey Shipilev; hotspot-dev at openjdk.java.net Betreff: Re: AW: AW: How to pin/avoid gc for an oop? I don't think Christian actually needs object pinning (as in avoiding relocation of an object due to GC). - Ioi On 3/13/17 8:36 PM, Aleksey Shipilev wrote: > Handle-izing / JNIRef-ing the oop != pinning. > > a) If you want pinning, you have to cooperate with GC somehow. JNI has > "critical" operations for this, so it would be instructive to see > jni_GetPrimitiveArrayCritical does this with GCLocker. > > b) If you just want the reachability, and are in native code, then JNI > GlobalRef would probably be the cleanest choice. > > c) If you just want the reachability, and are in Java code, then > calling java.lang.ref.Reference.reachabilityFence would retain the > local var in the thread roots. > > -Aleksey > > On 03/13/2017 01:23 PM, Ioi Lam wrote: >> How about using JNI NewGlobalRef/DeleteGlobalRef? Unlike the C++ >> "Handle" class (which is scoped and automatically reclaimed), a >> reference created by NewGlobalRef remains valid until it's explicitly deleted using DeleteGlobalRef. >> >> - Ioi >> >> On 3/13/17 8:59 AM, Christian Hagedorn wrote: >>> So handles are not really an option. What else could I do to make my >>> undo log a GC root to ensure reachability of the oops in it? >>> >>> Christian >>> >>> -----Urspr?ngliche Nachricht----- >>> Von: David Holmes [mailto:david.holmes at oracle.com] >>> Gesendet: Sonntag, 12. M?rz 2017 06:05 >>> An: Jon V.; Christian Hagedorn; hotspot-dev at openjdk.java.net >>> Betreff: Re: AW: How to pin/avoid gc for an oop? >>> >>> On 12/03/2017 10:59 AM, Jon V. wrote: >>>> I don't believe that handles will keep the objects from being >>>> garbage collected. >>> Handles are GC roots. Or rather the per-thread handle area is a GC root. >>> >>>> Hotspot does not support object pinning. >>> This isn't pinning - the object can still be moved. But the Handle >>> ensures it remains reachable. >>> >>>> Need input from a GC dev >>>> >>>> On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn >>>> >>>> wrote: >>>> >>>>> The undo log would be stored as an array of handles inside a >>>>> thread and does not refer to other threads. Would that work then? >>> Handles are intended to be scoped and reclaimed using HandleMarks. >>> This seem to require a different kind of usage. >>> >>> David >>> ----- >>> >>>>> Christian >>>>> >>>>> -----Urspr?ngliche Nachricht----- >>>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>>> Gesendet: Sonntag, 12. M?rz 2017 00:39 >>>>> An: Christian Hagedorn; hotspot-dev at openjdk.java.net >>>>> Betreff: Re: AW: How to pin/avoid gc for an oop? >>>>> >>>>> On 11/03/2017 9:08 PM, Christian Hagedorn wrote: >>>>>> Thank you for your answers. If it is possible to do it in the JVM >>>>>> I >>>>> would prefer to do that. >>>>>> I have tried to maintain an array of handles. However I think I >>>>>> have not >>>>> understood its usage correctly. I only saw examples by allocating >>>>> it on the stack with Handle h1(thread, oop). But how can I create >>>>> a handle on the heap such that it is kept alive for an access >>>>> later? It asserts not to call the global operator "new" such that "new Handle(..)" >>>>> does not work. >>>>> >>>>> Sorry that was incorrect advice. Handles are per-thread so you >>>>> can't have a data structure storing them across threads. >>>>> >>>>>> How could I make the log a GC root? >>>>> GC folk should chime in on the details of that - I don't want to >>>>> give more bad advice. >>>>> >>>>> David >>>>> >>>>>> Christian >>>>>> >>>>>> -----Urspr?ngliche Nachricht----- >>>>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>>>> Gesendet: Samstag, 11. M?rz 2017 04:58 >>>>>> An: Christian; Jon V. >>>>>> Cc: hotspot-dev at openjdk.java.net >>>>>> Betreff: Re: How to pin/avoid gc for an oop? >>>>>> >>>>>> On 10/03/2017 6:56 PM, Christian wrote: >>>>>>> Hi Jon, >>>>>>> >>>>>>> The STM is to be integrated into the JVM, but the transactional >>>>>>> execution only affects modifications to the managed memory done >>>>>>> by bytecode instructions. JVM internal structures are unaffected. >>>>>>> >>>>>>> Here an example: >>>>>>> >>>>>>> class X { >>>>>>> int f; >>>>>>> >>>>>>> public static update(X x, int _new) { >>>>>>> atomic { // tx start >>>>>>> // (1) update field f of x: >>>>>>> >>>>>>> // undo log maintains a reference to x and old value of f >>>>>>> // note: x contains a direct reference to the lock >>>>>>> >>>>>>> x.f = _new; >>>>>>> >>>>>>> // (2) assume GC is run >>>>>>> // Note: The atomic section can be large and blocking, >>>>>>> // thus, inhibiting a GC run is not an option. >>>>>>> >>>>>>> // (3) End of the atomic section (tx end): >>>>>>> // 1. In case of abort: restore old value >>>>>>> // 2. Release lock (obtained via reference to x) >>>>>>> } >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> How can I assure that the reference to x in the undo log that I >>>>>>> made at >>>>>>> (1) remains valid at (3), even though a GC run may occur at (2)? >>>>>>> I >>>>>> My making sure the GC sees everything in the undo log. The >>>>>> details will >>>>> depend on how you propose to implement that log. >>>>>>> thought that Handles (share/vm/runtime/handles.hpp) may be an >>>>>>> option, but I might have misunderstood what they are used for. I >>>>>>> can of course >>>>>> Handles are used inside the VM to protect oops when we need to >>>>>> use them >>>>> across calls that might lead to safepoint checks and thus to GC occurring. >>>>>>> make the undo log in the managed heap as well, but I hoped that >>>>>>> this can be done directly in the JVM. >>>>>> There are a number of ways to do this. If your log is a native VM >>>>>> data >>>>> structure then it might store Handles. Or you could make the log a >>>>> GC root and make it known to the GC, and store oops directly. >>>>>> Or you could implement it all in Java - which is probably a lot >>>>>> simpler, >>>>> albeit likely less performant. >>>>>> David >>>>>> >>>>>>> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." >>>>>>> >>>>> schrieb: >>>>>>>> Are you trying to build a transactional memory system for the >>>>>>>> JVM or your own Java code? >>>>>>>> >>>>>>>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Jon, >>>>>>>>> >>>>>>>>> I need this for an undo log of a transactional memory system >>>>>>>>> with pessimistic locking/eager version management. >>>>>>>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>>>>>>> >>>>>>>>> Can you clarify what you mean by Undo and why you think this >>>>>>>>> should >>>>>>>> be >>>>>>>>> done at the VM level so we can better understand the request. >>>>>>>>> >>>>>>>>> If you don't want something to be garbage collected then >>>>>>>>> simply don't dereference it. >>>>>>>>> >>>>>>>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I want to implement an undo functionality and thus need to >>>>>>>>> access an >>>>>>>> oop >>>>>>>>> even when it is out of scope in the Java code. How can I pin >>>>>>>>> such an >>>>>>>> oop to >>>>>>>>> avoid garbage collection of its used memory? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> >>>>>>>>> Christian >>>>>>>>> >>>>>>>>> From vladimir.x.ivanov at oracle.com Mon Mar 13 16:37:23 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 13 Mar 2017 19:37:23 +0300 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> Message-ID: <32433ac7-4946-2873-4ec9-6b680c697a4b@oracle.com> Coleen, Glad to see klass handles finally being removed! Thanks for taking care of that. >>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev I don't think you need the changes in doCall.cpp: src/share/vm/opto/doCall.cpp: - ciInstanceKlass* h_klass = h->is_catch_all() ? env()->Throwable_klass() : h->catch_klass(); + ciInstanceKlass* klass = h->is_catch_all() ? env()->Throwable_klass() : h->catch_klass(); > I fixed all of these to be ik, and classLoadingService now has a > _klass_array. Have you updated the webrev? Still see the following: src/share/vm/aot/aotCodeHeap.hpp + bool load_klass_data(InstanceKlass* kh, Thread* thread); src/share/vm/aot/aotCodeHeap.cpp: +bool AOTCodeHeap::load_klass_data(InstanceKlass* kh, Thread* thread) { src/share/vm/aot/aotLoader.cpp: +void AOTLoader::load_for_klass(InstanceKlass* kh, Thread* thread) { src/share/vm/c1/c1_Runtime1.cpp: + InstanceKlass* h = InstanceKlass::cast(klass); src/share/vm/ci/ciKlass.hpp: + ciKlass(Klass* k_h, ciSymbol* name); + ciKlass(Klass* k_h); src/share/vm/interpreter/interpreterRuntime.cpp: + InstanceKlass* h_cp_entry_f1 = InstanceKlass::cast(cp_entry->f1_as_klass()); src/share/vm/jvmci/jvmciCompilerToVM.cpp: + Klass* h_resolved = method->method_holder(); src/share/vm/jvmci/jvmciRuntime.cpp: + InstanceKlass* h = InstanceKlass::cast(klass); src/share/vm/oops/instanceKlass.cpp: + InstanceKlass* ih = InstanceKlass::cast(interfaces->at(index)); src/share/vm/oops/objArrayKlass.cpp: +ObjArrayKlass* ObjArrayKlass::allocate(ClassLoaderData* loader_data, int n, Klass* klass_handle, Symbol* name, TRAPS) { src/share/vm/prims/jvm.cpp: + InstanceKlass* k_h = InstanceKlass::cast(k); src/share/vm/services/classLoadingService.hpp: + GrowableArray* _klass_handle_array; src/share/vm/services/management.cpp: + Klass* kh = lce.get_klass(i); Best regards, Vladimir Ivanov >>> On 3/13/17 2:23 AM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> Wow that was a hard slog! :) >>>> >>>> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> As requested by David on the closed part of this review, I fixed >>>>> variable names of the form h_ik and klass_h where the 'h' indicated >>>>> that >>>>> the klass was in a handle. >>>> >>>> Thanks for that. But I'm afraid there are quite a number of other >>>> patterns including the "h" still present: >>>> >>>> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >>>> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >>>> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >>>> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >>>> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >>>> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved >>>> (though that code uses h_ when there doesn't appear to be a handle >>>> in use) >>>> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >>>> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >>>> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >>>> InstanceKlass* h_cp_entry_f1 >>>> src/share/vm/prims/jni.cpp: Klass* h_k >>>> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >>>> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh >>>> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >>>> _ikh, InstanceKlass* ikh() >>>> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >>>> instanceK_h >>>> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >>>> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >>>> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >>>> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >>>> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >>>> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >>>> src/share/vm/prims/nativeLookup.cpp: Klass* kh >>>> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >>>> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >>>> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >>>> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >>>> >>> >>> These were the ones I fixed and retested. >>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>> >>>> There are numerous code indentation issues with some of the changed >>>> code (mainly around parameter/argument lists) - but they all seem to >>>> be pre-existing so I decided not to try and call them out. You did >>>> fix quite a few, and I don't think you introduced any new ones. :) >>>> >>>> The main thing to look for with use of handles internal to methods >>>> is whether once changed to a direct Klass/InstanceKlass*, the >>>> variable becomes redundant. You picked up on a few of these but I >>>> spotted a couple of others (reading the patch file the context isn't >>>> always sufficient to spot these): >>>> >>>> share/vm/c1/c1_Runtime1.cpp >>>> >>>> { Klass* klass = resolve_field_return_klass(caller_method, >>>> bci, CHECK); >>>> - init_klass = KlassHandle(THREAD, klass); >>>> + init_klass = klass; >>>> >>>> You don't need the klass local but can assign direct to init_klass. >>>> >>>> // convert to handle >>>> - load_klass = KlassHandle(THREAD, k); >>>> + load_klass = k; >>>> >>>> Comment should be deleted, plus it seems you should be able to >>>> assign to load_klass directly earlier through the switch instead of >>>> introducing 'k'. >>> >>> This logic is not obvious and I'd rather not change it. init_klass >>> and load_klass are used further down for different purposes. I >>> removed the comment though. >>>> >>>> --- >>>> >>>> src/share/vm/jvmci/jvmciEnv.cpp >>>> >>>> 117 Klass* kls; >>>> >>>> This local is not needed as you can assign directly to found_klass now. >>> >>> Yes, that's better. >>>> >>>> --- >>>> >>>> Finally a meta-question re the changes in: >>>> >>>> src/share/vm/classfile/dictionary.hpp >>>> >>>> and the flow out from that. I'm unclear about all the changes from >>>> Klass to InstanceKlass. Are all dictionary/hashtable entries >>>> guaranteed to be instance classes? >>> >>> Yes, they are. This avoided multiple InstanceKlass::cast() calls. >>>> >>>> --- >>>> >>>> >>>>> Also, fixing InstanceKlass to not pass this_k can be done in a >>>>> separate >>>>> cleanup. >>>> >>>> Ok. >>> >>> Thank you for slogging through all of this. >>> >>> Coleen >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>>>> handles with no implementation. >>>>>> >>>>>> These are fairly extensive changes. The KlassHandle types have been >>>>>> dummy types since permgen elimination and were thought to be useful >>>>>> for future features. They aren't, so can be removed (see bug for >>>>>> more >>>>>> details). This allows stricter typing because you can use the more >>>>>> specific type, and using const more. I didn't add const to these >>>>>> changes, because of the cascading effect. >>>>>> >>>>>> The change isn't hard to review, just tedious. The main bug that I >>>>>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>>>>> >>>>>> It's so nice being able to single step on gdb without going into >>>>>> KlassHandle constructors! >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>>>> >>>>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>>>> JPRT. >>>>>> >>>>>> I can continue to hold this change for a convenient time for merging >>>>>> purposes with other people's JDK10 changes that they've been holding, >>>>>> or if there are other jdk9 changes that are likely to cause a problem >>>>>> for merging. I'll update the copyrights to 2017 on commit. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>> >>> >> > From mikael.gerdin at oracle.com Mon Mar 13 16:59:32 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Mon, 13 Mar 2017 17:59:32 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: References: Message-ID: <8291c6ec-3619-6fa3-c462-bd5b5cc15741@oracle.com> Hi Thomas, I've had a quick look at the patch, overall it looks good but I haven't looked at the tests yet. On 2017-03-11 10:08, Thomas St?fe wrote: > Ping... Did I send this to the wrong mailing list? > > On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe > wrote: > >> (Resent to hotspot-dev) >> >> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >> wrote: >> >>> Hi all, >>> >>> I would like to get a review for this cleanup change for the >>> metaspace.cpp. This has been a long time brewing in my backlog, but I had >>> to wait until jdk10 is open. The cleanup is actually quite small, the >>> largest part of the change is the added test coding. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ While you're at it, would you mind breaking assert(_free_chunks_count > 0 && _free_chunks_total >= v, "About to go negative"); into two separate asserts and have them print the values? In ChunkManager::list_index you can now use size_by_index (and maybe make size_by_index inline in the ChunkManager class declaration? In ChunkManager::return_chunk_list you might as well use LogTarget(Trace, gc, metaspace, freelist) log; since you only log to the trace level anyway. I'll get back to you with feedback on the tests once I have had a good look at them. Thanks for picking up this fix! /Mikael >>> >>> The change implements a suggestion made by Mikael Gerdin when >>> reviewing JDK-8170520 last year, who suggested that the ChunkManager class >>> should handle returns of both non-humongous and humongous chunks, see >>> original discussion here: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>> 016-November/021949.html >>> >>> "It would be great if you could have a look at unifying the >>> ChunkManager::return_chunks API for standard and humongous chunks so >>> that we wouldn't need the special casing for this in ~SpaceManager >>> That way we could also make humongous_dictionary() private to >>> ChunkManager which would improve encapsulation." >>> >>> The cleanup does this and a bit more, all changes having to do with the >>> fact that the ChunkManager class unnecessarily exposes internals to the >>> other classes in metaspace.cpp and that with a little bit of reorganization >>> this can be avoided. The benefit is simpler coding and way better >>> encapsulation, which will help a lot with future changes (in preparation >>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>> >>> -------- >>> >>> The changes in detail: >>> >>> The main issue was that ChunkManager::return_chunks() did not handle >>> humongous chunks. To return humongous chunks, one had to access the >>> humongous chunk dictionary inside the ChunkManager and add the chunk >>> yourself, including maintaining all counters. This happened in >>> ~SpaceManager and made this destructor very complicated. >>> >>> This is solved by moving the handling of humongous chunk returns to the >>> ChunkManager class itself. For convenience, I split the old >>> "ChunkManager::return_chunks" method into two new ones, >>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>> the ChunkManager without walking the chunk chain, and >>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>> the ChunkManager. Both functions are now able to handle both non-humongous >>> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >>> simplified) and just calls "ChunkManager::return_chunk_list()" for all >>> its chunk lists. >>> >>> So now we can remove the public access to the humongous chunk dictionary >>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>> function private. >>> >>> ---- >>> >>> Then there was the fact that the non-humongous chunk lists were also >>> exposed to public via ChunkManager::free_chunks(). The only reason was that >>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free lists >>> to find out the size of the items of the free lists. >>> >>> This was solved by adding a new function, ChunkManager::size_by_index(), >>> which returns the size of the items of a chunk list given by its chunk >>> index. >>> >>> So ChunkManager::free_chunks() could be made private too. >>> >>> --- >>> >>> The rest are minor changes: >>> >>> - made ChunkManager::find_free_chunks_list() private because it was not >>> used from outside the class >>> - Fixed an assert in dec_free_chunks_total() >>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >>> to keep the existing logging intact or add useful logging where possible. >>> >>> ---- >>> >>> A large chunk of the changes is the added gtests. There is a new class >>> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >>> code. >>> >>> Note that I dislike the fact that this test is implemented inside >>> metaspace.cpp itself. For now I kept my new tests like the existing tests >>> inside this file, but I think these tests should be moved to >>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>> separate fix though, because it needs a bit of remodeling (the tests need >>> to access internals in metaspace.cpp). >>> >>> ---- >>> >>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >>> AIX. No issues popped up which were associated with my change. >>> >>> Thanks for reviewing and Kind Regards, Thomas >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> From thomas.stuefe at gmail.com Mon Mar 13 17:08:19 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 13 Mar 2017 18:08:19 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: <8291c6ec-3619-6fa3-c462-bd5b5cc15741@oracle.com> References: <8291c6ec-3619-6fa3-c462-bd5b5cc15741@oracle.com> Message-ID: Hi Mikael, thanks a lot for reviewing! The changes you request make sense, but I'll wait until you reviewed the rest. Kind Regards, Thomas On Mon, Mar 13, 2017 at 5:59 PM, Mikael Gerdin wrote: > Hi Thomas, > > I've had a quick look at the patch, overall it looks good but I haven't > looked at the tests yet. > > On 2017-03-11 10:08, Thomas St?fe wrote: > >> Ping... Did I send this to the wrong mailing list? >> >> On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe >> wrote: >> >> (Resent to hotspot-dev) >>> >>> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >>> wrote: >>> >>> Hi all, >>>> >>>> I would like to get a review for this cleanup change for the >>>> metaspace.cpp. This has been a long time brewing in my backlog, but I >>>> had >>>> to wait until jdk10 is open. The cleanup is actually quite small, the >>>> largest part of the change is the added test coding. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>>> >>> > While you're at it, would you mind breaking > > assert(_free_chunks_count > 0 && > _free_chunks_total >= v, > "About to go negative"); > > into two separate asserts and have them print the values? > > In ChunkManager::list_index you can now use size_by_index (and maybe make > size_by_index inline in the ChunkManager class declaration? > > In ChunkManager::return_chunk_list you might as well use > LogTarget(Trace, gc, metaspace, freelist) log; > since you only log to the trace level anyway. > > I'll get back to you with feedback on the tests once I have had a good > look at them. > > Thanks for picking up this fix! > /Mikael > > > >>>> The change implements a suggestion made by Mikael Gerdin when >>>> reviewing JDK-8170520 last year, who suggested that the ChunkManager >>>> class >>>> should handle returns of both non-humongous and humongous chunks, see >>>> original discussion here: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>>> 016-November/021949.html >>>> >>>> "It would be great if you could have a look at unifying the >>>> ChunkManager::return_chunks API for standard and humongous chunks so >>>> that we wouldn't need the special casing for this in ~SpaceManager >>>> That way we could also make humongous_dictionary() private to >>>> ChunkManager which would improve encapsulation." >>>> >>>> The cleanup does this and a bit more, all changes having to do with the >>>> fact that the ChunkManager class unnecessarily exposes internals to the >>>> other classes in metaspace.cpp and that with a little bit of >>>> reorganization >>>> this can be avoided. The benefit is simpler coding and way better >>>> encapsulation, which will help a lot with future changes (in preparation >>>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>>> >>>> -------- >>>> >>>> The changes in detail: >>>> >>>> The main issue was that ChunkManager::return_chunks() did not handle >>>> humongous chunks. To return humongous chunks, one had to access the >>>> humongous chunk dictionary inside the ChunkManager and add the chunk >>>> yourself, including maintaining all counters. This happened in >>>> ~SpaceManager and made this destructor very complicated. >>>> >>>> This is solved by moving the handling of humongous chunk returns to the >>>> ChunkManager class itself. For convenience, I split the old >>>> "ChunkManager::return_chunks" method into two new ones, >>>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>>> the ChunkManager without walking the chunk chain, and >>>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>>> the ChunkManager. Both functions are now able to handle both >>>> non-humongous >>>> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >>>> simplified) and just calls "ChunkManager::return_chunk_list()" for all >>>> its chunk lists. >>>> >>>> So now we can remove the public access to the humongous chunk dictionary >>>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>>> function private. >>>> >>>> ---- >>>> >>>> Then there was the fact that the non-humongous chunk lists were also >>>> exposed to public via ChunkManager::free_chunks(). The only reason was >>>> that >>>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free >>>> lists >>>> to find out the size of the items of the free lists. >>>> >>>> This was solved by adding a new function, ChunkManager::size_by_index(), >>>> which returns the size of the items of a chunk list given by its chunk >>>> index. >>>> >>>> So ChunkManager::free_chunks() could be made private too. >>>> >>>> --- >>>> >>>> The rest are minor changes: >>>> >>>> - made ChunkManager::find_free_chunks_list() private because it was not >>>> used from outside the class >>>> - Fixed an assert in dec_free_chunks_total() >>>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >>>> to keep the existing logging intact or add useful logging where >>>> possible. >>>> >>>> ---- >>>> >>>> A large chunk of the changes is the added gtests. There is a new class >>>> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >>>> code. >>>> >>>> Note that I dislike the fact that this test is implemented inside >>>> metaspace.cpp itself. For now I kept my new tests like the existing >>>> tests >>>> inside this file, but I think these tests should be moved to >>>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>>> separate fix though, because it needs a bit of remodeling (the tests >>>> need >>>> to access internals in metaspace.cpp). >>>> >>>> ---- >>>> >>>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >>>> AIX. No issues popped up which were associated with my change. >>>> >>>> Thanks for reviewing and Kind Regards, Thomas >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> From lucene at mikemccandless.com Sun Mar 12 21:15:25 2017 From: lucene at mikemccandless.com (Michael McCandless) Date: Sun, 12 Mar 2017 17:15:25 -0400 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: <010401d299ca$c198b620$44ca2260$@apache.org> References: <010401d299ca$c198b620$44ca2260$@apache.org> Message-ID: Hi Uwe, I ran the standard luceneutil Wikipedia benchmark (same as what's run nightly at https://home.apache.org/~mikemccand/lucenebench). I compared Java 1.8.0_121 (base, below) and Java 9 ea 160 (comp), indexing and searching all Wikipedia English content, using MMapDirectory, and net/net there is some smallish slowdown, but it's not catastrophic. I'm not sure why sloppy phrase queries are particularly hit. This was after 20 JVM iterations: Task QPS base StdDev QPS comp StdDev Pct diff LowSloppyPhrase 11.02 (6.0%) 9.17 (3.1%) -16.8% ( -24% - -8%) HighSloppyPhrase 7.40 (4.5%) 6.36 (2.9%) -14.1% ( -20% - -6%) MedSloppyPhrase 5.13 (4.3%) 4.48 (2.4%) -12.7% ( -18% - -6%) HighTerm 44.54 (5.6%) 39.01 (3.6%) -12.4% ( -20% - -3%) OrHighLow 39.53 (7.1%) 34.70 (4.9%) -12.2% ( -22% - 0%) MedTerm 86.34 (5.2%) 75.88 (3.3%) -12.1% ( -19% - -3%) OrHighNotLow 52.00 (7.1%) 46.10 (6.0%) -11.3% ( -22% - 1%) BrowseDayOfYearSSDVFacets 5.33 (3.3%) 4.76 (5.8%) -10.6% ( -19% - -1%) HighTermDayOfYearSort 20.49 (5.5%) 18.36 (5.8%) -10.4% ( -20% - 1%) BrowseMonthTaxoFacets 1.47 (2.2%) 1.35 (6.8%) -8.6% ( -17% - 0%) OrHighNotMed 41.62 (3.5%) 38.18 (4.7%) -8.3% ( -15% - 0%) OrHighNotHigh 16.08 (3.3%) 14.84 (4.4%) -7.7% ( -14% - 0%) BrowseDayOfYearTaxoFacets 1.31 (2.1%) 1.21 (6.4%) -7.1% ( -15% - 1%) BrowseDateTaxoFacets 1.31 (2.1%) 1.22 (6.5%) -7.0% ( -15% - 1%) Prefix3 20.13 (7.2%) 18.74 (5.9%) -6.9% ( -18% - 6%) OrNotHighHigh 23.22 (3.1%) 21.63 (3.9%) -6.9% ( -13% - 0%) IntNRQ 7.33 (9.3%) 6.83 (11.2%) -6.8% ( -24% - 15%) Fuzzy2 105.61 (1.7%) 98.65 (1.3%) -6.6% ( -9% - -3%) Wildcard 56.59 (5.4%) 52.86 (4.3%) -6.6% ( -15% - 3%) MedSpanNear 36.06 (4.0%) 33.88 (3.6%) -6.0% ( -13% - 1%) HighSpanNear 35.40 (4.0%) 33.39 (3.8%) -5.7% ( -12% - 2%) Fuzzy1 89.36 (1.6%) 84.28 (1.5%) -5.7% ( -8% - -2%) OrHighMed 28.17 (9.8%) 26.66 (3.6%) -5.4% ( -17% - 8%) OrHighHigh 17.94 (10.5%) 17.01 (3.7%) -5.2% ( -17% - 10%) LowTerm 397.33 (2.4%) 377.12 (1.4%) -5.1% ( -8% - -1%) OrNotHighMed 86.69 (2.1%) 82.75 (3.1%) -4.5% ( -9% - 0%) HighPhrase 8.58 (7.2%) 8.19 (6.6%) -4.5% ( -17% - 10%) LowSpanNear 42.81 (1.5%) 41.17 (1.6%) -3.8% ( -6% - 0%) Respell 71.63 (1.3%) 69.17 (1.4%) -3.4% ( -6% - 0%) BrowseMonthSSDVFacets 5.74 (11.1%) 5.55 (10.8%) -3.4% ( -22% - 20%) AndHighHigh 33.22 (1.6%) 32.15 (1.5%) -3.2% ( -6% - 0%) LowPhrase 43.46 (1.6%) 42.39 (1.4%) -2.5% ( -5% - 0%) PKLookup 360.20 (2.1%) 352.10 (1.3%) -2.3% ( -5% - 1%) MedPhrase 33.01 (3.9%) 32.56 (3.7%) -1.4% ( -8% - 6%) AndHighLow 706.78 (2.4%) 706.29 (1.5%) -0.1% ( -3% - 3%) HighTermMonthSort 40.02 (4.4%) 40.36 (4.9%) 0.9% ( -8% - 10%) OrNotHighLow 585.09 (2.8%) 593.40 (1.8%) 1.4% ( -3% - 6%) AndHighMed 111.84 (2.6%) 113.65 (1.5%) 1.6% ( -2% - 5%) Mike McCandless http://blog.mikemccandless.com On Fri, Mar 10, 2017 at 1:18 PM, Uwe Schindler wrote: > Hi, > > we just noticed this issue: https://bugs.openjdk.java.net/ > browse/JDK-8176513 > > As Apache Lucene relies heavily on performance of ByteBuffers (especially > MappedByteBuffer), this would be a desaster if it would get even slower > than Java 8. We were so happy that there was much work going on to improve > the performance of ByteBuffers so they were at some point almost as fast as > byte[]. Because of that we are currently working on making Lucene work fine > with Java 9, because once it is out, we would inform all users and > recommend to use Java 9 because of these optimizations. The tests of Lucene > are already running perfectly, no problems like data corrumption (remember > Java 7 GA or 7u40). Also Apache Solr people try to fix the remaining > Hadoop/Kerberos-Auth issues because of Jigsaw. We also have unmapping > working using Unsafe::invokeCleaner, so we are prepared... > > I am not yet sure if this bug affects us, we have to run perf tests first. > Nevertheless, this is our usage pattern: > - we only read from ByteBuffers (MappedByteBuffer), never store anything > (we use mmap to execute Lucene searches on the mapped index with often 10th > sometimes also 100th of gigabytes of data files that were mmapped). > - we do sequential reads (position() and then getByte(), getXxX) > - we also do positional reads like this test (for the so called DocValues > in Lucene, which is a column based store). This is mainly like sorting > search results based on a compare function that accesses a ByteBuffer as > random-access source for comparison values. > - we don't use IntBuffer, we work directly on ByteBuffer > > I asked Mike McCandless to run the performance tests with a recent Java 9 > version to see if it affects us. Is there any information, when this bug > was introduced into Java 9? Nevertheless, please fix it for Java 9 GA, if > it affects us it would be a major loss of search performance for us! > > It would be good to get some information about your plans :-) Thanks! > > Uwe > > ----- > Uwe Schindler > uschindler at apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscribe at lucene.apache.org > For additional commands, e-mail: dev-help at lucene.apache.org > > From lucene at mikemccandless.com Mon Mar 13 14:16:47 2017 From: lucene at mikemccandless.com (Michael McCandless) Date: Mon, 13 Mar 2017 10:16:47 -0400 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: <011201d29be5$8250dfc0$86f29f40$@apache.org> References: <010401d299ca$c198b620$44ca2260$@apache.org> <011201d29be5$8250dfc0$86f29f40$@apache.org> Message-ID: Hi Uwe, OK, I'll test with NIOFSDirectory as well ... that's a good idea. I do remember testing earlier Java 9 builds long ago, but I can't remember what the outcome was. Mike McCandless http://blog.mikemccandless.com On Mon, Mar 13, 2017 at 6:35 AM, Uwe Schindler wrote: > Hi Andrew, > > yes that was my impression, too. > > Just for cross-checking: Mike, is it possible to also add a perf > comparison between Java 8 and Java 9 when using SimpleFSDirectory or > NIOFSDirectory (which are both FileChannel based since Java 7, the name is > just backwards-compatibility)? If we see a slowdown there (maybe even > larger) than it is not related to ByteBuffer positional/byte-wise reads and > there is a general performance issue somewhere else. > > > Right, but ByteBuffers were significantly rewritten for a significant > > performance *increase*. Any slowdown shows that something has gone > > very wrong indeed. > > That would be a pity, because of that we should check the above case with > non-mmap based, conventional index access. As far as I remember: at the > time when you announced the bytebuffer improvements we did some performance > measurements and were impressed by the speedup. I think Robert Muir did > something. Mike, do you know? > > Maybe we should check with a Java 9 preview build from that time. I know > that you can download older builds by changing the build number in the > download URL. > > Uwe > > From lucene at mikemccandless.com Mon Mar 13 15:43:34 2017 From: lucene at mikemccandless.com (Michael McCandless) Date: Mon, 13 Mar 2017 11:43:34 -0400 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: References: <010401d299ca$c198b620$44ca2260$@apache.org> <011201d29be5$8250dfc0$86f29f40$@apache.org> Message-ID: I reran the same test as before, this time using Lucene's NIOFSDirectory (java.nio's FileChannel for positional reads). There is still some slowdown, though a bit less than with MMapDirectory: Task QPS base StdDev QPS comp StdDev Pct diff LowSloppyPhrase 9.12 (5.2%) 7.89 (3.6%) -13.5% ( -21% - -4%) HighSloppyPhrase 6.25 (4.1%) 5.57 (3.3%) -10.8% ( -17% - -3%) MedSloppyPhrase 4.29 (3.8%) 3.85 (3.0%) -10.1% ( -16% - -3%) BrowseDayOfYearSSDVFacets 4.72 (8.8%) 4.32 (5.4%) -8.4% ( -20% - 6%) BrowseMonthTaxoFacets 1.31 (2.9%) 1.20 (4.7%) -8.3% ( -15% - 0%) BrowseDayOfYearTaxoFacets 1.17 (4.0%) 1.08 (4.1%) -7.7% ( -15% - 0%) BrowseDateTaxoFacets 1.18 (4.0%) 1.09 (4.1%) -7.6% ( -15% - 0%) BrowseMonthSSDVFacets 5.39 (4.3%) 4.98 (10.7%) -7.6% ( -21% - 7%) HighTerm 29.47 (5.3%) 27.45 (4.2%) -6.9% ( -15% - 2%) HighTermDayOfYearSort 14.24 (4.6%) 13.35 (5.5%) -6.3% ( -15% - 3%) MedTerm 44.47 (4.0%) 42.02 (3.4%) -5.5% ( -12% - 1%) OrHighNotLow 33.13 (5.0%) 31.39 (4.3%) -5.2% ( -13% - 4%) OrHighLow 26.84 (4.2%) 25.49 (4.0%) -5.0% ( -12% - 3%) HighPhrase 7.51 (5.4%) 7.17 (4.0%) -4.5% ( -13% - 5%) Fuzzy2 51.32 (0.9%) 49.03 (1.2%) -4.5% ( -6% - -2%) OrHighNotHigh 13.18 (3.5%) 12.64 (3.7%) -4.2% ( -10% - 3%) IntNRQ 6.28 (6.3%) 6.03 (9.9%) -4.0% ( -18% - 12%) OrHighNotMed 27.69 (2.9%) 26.65 (3.3%) -3.8% ( -9% - 2%) Fuzzy1 42.32 (0.8%) 40.88 (1.1%) -3.4% ( -5% - -1%) MedSpanNear 27.44 (2.4%) 26.57 (2.8%) -3.2% ( -8% - 2%) OrNotHighHigh 17.58 (2.8%) 17.04 (3.3%) -3.1% ( -8% - 3%) HighSpanNear 26.83 (2.2%) 26.03 (2.5%) -3.0% ( -7% - 1%) Respell 49.07 (1.2%) 47.62 (0.8%) -3.0% ( -4% - 0%) LowPhrase 31.19 (1.5%) 30.34 (1.2%) -2.7% ( -5% - 0%) Wildcard 51.30 (4.8%) 49.93 (4.3%) -2.7% ( -11% - 6%) LowSpanNear 31.05 (1.2%) 30.40 (1.5%) -2.1% ( -4% - 0%) LowTerm 105.83 (1.1%) 103.78 (1.4%) -1.9% ( -4% - 0%) OrNotHighMed 49.08 (1.8%) 48.16 (2.1%) -1.9% ( -5% - 2%) OrHighMed 18.44 (5.8%) 18.09 (5.1%) -1.9% ( -12% - 9%) AndHighHigh 21.22 (1.0%) 20.82 (1.3%) -1.9% ( -4% - 0%) HighTermMonthSort 32.01 (3.9%) 31.43 (5.3%) -1.8% ( -10% - 7%) OrHighHigh 12.89 (7.0%) 12.66 (6.0%) -1.8% ( -13% - 12%) MedPhrase 24.09 (2.3%) 23.80 (2.2%) -1.2% ( -5% - 3%) AndHighLow 447.74 (1.5%) 443.85 (1.7%) -0.9% ( -3% - 2%) Prefix3 18.41 (6.8%) 18.28 (5.6%) -0.7% ( -12% - 12%) OrNotHighLow 254.77 (1.4%) 254.59 (1.2%) -0.1% ( -2% - 2%) AndHighMed 63.15 (1.5%) 63.47 (0.9%) 0.5% ( -1% - 3%) PKLookup 347.80 (2.3%) 349.93 (2.1%) 0.6% ( -3% - 5%) Mike McCandless http://blog.mikemccandless.com On Mon, Mar 13, 2017 at 10:16 AM, Michael McCandless < lucene at mikemccandless.com> wrote: > Hi Uwe, > > OK, I'll test with NIOFSDirectory as well ... that's a good idea. > > I do remember testing earlier Java 9 builds long ago, but I can't remember > what the outcome was. > > Mike McCandless > > http://blog.mikemccandless.com > > On Mon, Mar 13, 2017 at 6:35 AM, Uwe Schindler > wrote: > >> Hi Andrew, >> >> yes that was my impression, too. >> >> Just for cross-checking: Mike, is it possible to also add a perf >> comparison between Java 8 and Java 9 when using SimpleFSDirectory or >> NIOFSDirectory (which are both FileChannel based since Java 7, the name is >> just backwards-compatibility)? If we see a slowdown there (maybe even >> larger) than it is not related to ByteBuffer positional/byte-wise reads and >> there is a general performance issue somewhere else. >> >> > Right, but ByteBuffers were significantly rewritten for a significant >> > performance *increase*. Any slowdown shows that something has gone >> > very wrong indeed. >> >> That would be a pity, because of that we should check the above case with >> non-mmap based, conventional index access. As far as I remember: at the >> time when you announced the bytebuffer improvements we did some performance >> measurements and were impressed by the speedup. I think Robert Muir did >> something. Mike, do you know? >> >> Maybe we should check with a Java 9 preview build from that time. I know >> that you can download older builds by changing the build number in the >> download URL. >> >> Uwe >> >> > From coleen.phillimore at oracle.com Mon Mar 13 17:38:27 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 13:38:27 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: <32433ac7-4946-2873-4ec9-6b680c697a4b@oracle.com> References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> <32433ac7-4946-2873-4ec9-6b680c697a4b@oracle.com> Message-ID: <32bd6d67-7af7-da0c-c933-54fd7ece48a4@oracle.com> Vladimir, thank you for reviewing this. I'm glad someone from the compiler group has looked at these changes. On 3/13/17 12:37 PM, Vladimir Ivanov wrote: > Coleen, > > Glad to see klass handles finally being removed! Thanks for taking > care of that. :) > >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev > > I don't think you need the changes in doCall.cpp: > > src/share/vm/opto/doCall.cpp: > > - ciInstanceKlass* h_klass = h->is_catch_all() ? > env()->Throwable_klass() : h->catch_klass(); > + ciInstanceKlass* klass = h->is_catch_all() ? > env()->Throwable_klass() : h->catch_klass(); > I found those instances with the pattern matching. I reverted doCall.cpp. > >> I fixed all of these to be ik, and classLoadingService now has a >> _klass_array. > > Have you updated the webrev? Still see the following: I haven't updated the webrev (it takes a while!) I'll update after this round of changes to .03. Phew, you guys have good eyes. See open webrev at http://cr.openjdk.java.net/~coleenp/8155672.03/webrev that includes the changes below (in about an hour). > > src/share/vm/aot/aotCodeHeap.hpp > > + bool load_klass_data(InstanceKlass* kh, Thread* thread); > > src/share/vm/aot/aotCodeHeap.cpp: > +bool AOTCodeHeap::load_klass_data(InstanceKlass* kh, Thread* thread) { > > src/share/vm/aot/aotLoader.cpp: > > +void AOTLoader::load_for_klass(InstanceKlass* kh, Thread* thread) { > > src/share/vm/c1/c1_Runtime1.cpp: > + InstanceKlass* h = InstanceKlass::cast(klass); > > src/share/vm/ci/ciKlass.hpp: > + ciKlass(Klass* k_h, ciSymbol* name); > > + ciKlass(Klass* k_h); I missed this one. > > src/share/vm/interpreter/interpreterRuntime.cpp: > > + InstanceKlass* h_cp_entry_f1 = > InstanceKlass::cast(cp_entry->f1_as_klass()); Missed this one in my pattern matching. Wasn't looking for h_ patterns, only h_k, k_h, kh and h_klass. fixed. > > src/share/vm/jvmci/jvmciCompilerToVM.cpp: > > + Klass* h_resolved = method->method_holder(); fixed. > > src/share/vm/jvmci/jvmciRuntime.cpp: > > + InstanceKlass* h = InstanceKlass::cast(klass); got this. > > src/share/vm/oops/instanceKlass.cpp: > > + InstanceKlass* ih = InstanceKlass::cast(interfaces->at(index)); ih is not an easy thing to search for. Changed to interk. > > src/share/vm/oops/objArrayKlass.cpp: > > +ObjArrayKlass* ObjArrayKlass::allocate(ClassLoaderData* loader_data, > int n, Klass* klass_handle, Symbol* name, TRAPS) { fixed. > > src/share/vm/prims/jvm.cpp: > > + InstanceKlass* k_h = InstanceKlass::cast(k); > > src/share/vm/services/classLoadingService.hpp: > > + GrowableArray* _klass_handle_array; > > src/share/vm/services/management.cpp: > > + Klass* kh = lce.get_klass(i); fixed. Thanks! Coleen > > Best regards, > Vladimir Ivanov > >>>> On 3/13/17 2:23 AM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> Wow that was a hard slog! :) >>>>> >>>>> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> As requested by David on the closed part of this review, I fixed >>>>>> variable names of the form h_ik and klass_h where the 'h' indicated >>>>>> that >>>>>> the klass was in a handle. >>>>> >>>>> Thanks for that. But I'm afraid there are quite a number of other >>>>> patterns including the "h" still present: >>>>> >>>>> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >>>>> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >>>>> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >>>>> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >>>>> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >>>>> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved >>>>> (though that code uses h_ when there doesn't appear to be a handle >>>>> in use) >>>>> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >>>>> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >>>>> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >>>>> InstanceKlass* h_cp_entry_f1 >>>>> src/share/vm/prims/jni.cpp: Klass* h_k >>>>> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >>>>> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* >>>>> iikh >>>>> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >>>>> _ikh, InstanceKlass* ikh() >>>>> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >>>>> instanceK_h >>>>> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >>>>> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >>>>> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >>>>> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >>>>> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >>>>> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >>>>> src/share/vm/prims/nativeLookup.cpp: Klass* kh >>>>> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >>>>> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >>>>> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >>>>> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >>>>> >>>> >>>> These were the ones I fixed and retested. >>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>> >>>>> There are numerous code indentation issues with some of the changed >>>>> code (mainly around parameter/argument lists) - but they all seem to >>>>> be pre-existing so I decided not to try and call them out. You did >>>>> fix quite a few, and I don't think you introduced any new ones. :) >>>>> >>>>> The main thing to look for with use of handles internal to methods >>>>> is whether once changed to a direct Klass/InstanceKlass*, the >>>>> variable becomes redundant. You picked up on a few of these but I >>>>> spotted a couple of others (reading the patch file the context isn't >>>>> always sufficient to spot these): >>>>> >>>>> share/vm/c1/c1_Runtime1.cpp >>>>> >>>>> { Klass* klass = resolve_field_return_klass(caller_method, >>>>> bci, CHECK); >>>>> - init_klass = KlassHandle(THREAD, klass); >>>>> + init_klass = klass; >>>>> >>>>> You don't need the klass local but can assign direct to init_klass. >>>>> >>>>> // convert to handle >>>>> - load_klass = KlassHandle(THREAD, k); >>>>> + load_klass = k; >>>>> >>>>> Comment should be deleted, plus it seems you should be able to >>>>> assign to load_klass directly earlier through the switch instead of >>>>> introducing 'k'. >>>> >>>> This logic is not obvious and I'd rather not change it. init_klass >>>> and load_klass are used further down for different purposes. I >>>> removed the comment though. >>>>> >>>>> --- >>>>> >>>>> src/share/vm/jvmci/jvmciEnv.cpp >>>>> >>>>> 117 Klass* kls; >>>>> >>>>> This local is not needed as you can assign directly to found_klass >>>>> now. >>>> >>>> Yes, that's better. >>>>> >>>>> --- >>>>> >>>>> Finally a meta-question re the changes in: >>>>> >>>>> src/share/vm/classfile/dictionary.hpp >>>>> >>>>> and the flow out from that. I'm unclear about all the changes from >>>>> Klass to InstanceKlass. Are all dictionary/hashtable entries >>>>> guaranteed to be instance classes? >>>> >>>> Yes, they are. This avoided multiple InstanceKlass::cast() calls. >>>>> >>>>> --- >>>>> >>>>> >>>>>> Also, fixing InstanceKlass to not pass this_k can be done in a >>>>>> separate >>>>>> cleanup. >>>>> >>>>> Ok. >>>> >>>> Thank you for slogging through all of this. >>>> >>>> Coleen >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>>>>> handles with no implementation. >>>>>>> >>>>>>> These are fairly extensive changes. The KlassHandle types have >>>>>>> been >>>>>>> dummy types since permgen elimination and were thought to be useful >>>>>>> for future features. They aren't, so can be removed (see bug for >>>>>>> more >>>>>>> details). This allows stricter typing because you can use the >>>>>>> more >>>>>>> specific type, and using const more. I didn't add const to these >>>>>>> changes, because of the cascading effect. >>>>>>> >>>>>>> The change isn't hard to review, just tedious. The main bug that I >>>>>>> had was redeclaring a type inside a scope, and >>>>>>> InstanceKlass::cast(k) >>>>>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) >>>>>>> could. >>>>>>> >>>>>>> It's so nice being able to single step on gdb without going into >>>>>>> KlassHandle constructors! >>>>>>> >>>>>>> open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>>>>> >>>>>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>>>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>>>>> JPRT. >>>>>>> >>>>>>> I can continue to hold this change for a convenient time for >>>>>>> merging >>>>>>> purposes with other people's JDK10 changes that they've been >>>>>>> holding, >>>>>>> or if there are other jdk9 changes that are likely to cause a >>>>>>> problem >>>>>>> for merging. I'll update the copyrights to 2017 on commit. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>> >>>> >>> >> From serguei.spitsyn at oracle.com Mon Mar 13 18:31:12 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 13 Mar 2017 11:31:12 -0700 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> Message-ID: Hi Coleen, It looks good. Nice cleanup! http://oklahoma.us.oracle.com/~cphillim/webrev/8155627.src.closed.02/webrev/share/vm/jfr/jfrClassAdapter.cpp.frames.html The HandleMark's at 1743 & 1784 can be not needed anymore. Thanks, Serguei On 3/12/17 21:18, coleen.phillimore at oracle.com wrote: > > As requested by David on the closed part of this review, I fixed > variable names of the form h_ik and klass_h where the 'h' indicated > that the klass was in a handle. > > open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev > > Also, fixing InstanceKlass to not pass this_k can be done in a > separate cleanup. > > Thanks, > Coleen > > On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >> handles with no implementation. >> >> These are fairly extensive changes. The KlassHandle types have been >> dummy types since permgen elimination and were thought to be useful >> for future features. They aren't, so can be removed (see bug for >> more details). This allows stricter typing because you can use the >> more specific type, and using const more. I didn't add const to >> these changes, because of the cascading effect. >> >> The change isn't hard to review, just tedious. The main bug that I >> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >> >> It's so nice being able to single step on gdb without going into >> KlassHandle constructors! >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >> >> Tested with all hotspot jtreg tests, java/lang/invoke tests, >> java/lang/instrument tests, all closed tonga colocated tests, and JPRT. >> >> I can continue to hold this change for a convenient time for merging >> purposes with other people's JDK10 changes that they've been holding, >> or if there are other jdk9 changes that are likely to cause a problem >> for merging. I'll update the copyrights to 2017 on commit. >> >> Thanks, >> Coleen > From coleen.phillimore at oracle.com Mon Mar 13 19:34:07 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 15:34:07 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> Message-ID: On 3/13/17 2:31 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > It looks good. > Nice cleanup! > > http://oklahoma.us.oracle.com/~cphillim/webrev/8155627.src.closed.02/webrev/share/vm/jfr/jfrClassAdapter.cpp.frames.html > > > The HandleMark's at 1743 & 1784 can be not needed anymore. Thank you for reviewing! I suspect many HandleMarks lying around the code aren't needed anymore, because KlassHandles (and even methodHandles) aren't cleaned up using HandleMark. I've removed those two (and ran some retests because this is getting dangerous). Are you going to review the open too? thank you! Coleen > > Thanks, > Serguei > > > On 3/12/17 21:18, coleen.phillimore at oracle.com wrote: >> >> As requested by David on the closed part of this review, I fixed >> variable names of the form h_ik and klass_h where the 'h' indicated >> that the klass was in a handle. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >> >> Also, fixing InstanceKlass to not pass this_k can be done in a >> separate cleanup. >> >> Thanks, >> Coleen >> >> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>> handles with no implementation. >>> >>> These are fairly extensive changes. The KlassHandle types have been >>> dummy types since permgen elimination and were thought to be useful >>> for future features. They aren't, so can be removed (see bug for >>> more details). This allows stricter typing because you can use the >>> more specific type, and using const more. I didn't add const to >>> these changes, because of the cascading effect. >>> >>> The change isn't hard to review, just tedious. The main bug that I >>> had was redeclaring a type inside a scope, and >>> InstanceKlass::cast(k) can't take a NULL k, whereas >>> instanceKlassHandle ik(THREAD, k) could. >>> >>> It's so nice being able to single step on gdb without going into >>> KlassHandle constructors! >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>> >>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>> java/lang/instrument tests, all closed tonga colocated tests, and JPRT. >>> >>> I can continue to hold this change for a convenient time for merging >>> purposes with other people's JDK10 changes that they've been >>> holding, or if there are other jdk9 changes that are likely to cause >>> a problem for merging. I'll update the copyrights to 2017 on commit. >>> >>> Thanks, >>> Coleen >> > From serguei.spitsyn at oracle.com Mon Mar 13 19:42:10 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 13 Mar 2017 12:42:10 -0700 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> Message-ID: On 3/13/17 12:34, coleen.phillimore at oracle.com wrote: > > > On 3/13/17 2:31 PM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> It looks good. >> Nice cleanup! >> >> http://oklahoma.us.oracle.com/~cphillim/webrev/8155627.src.closed.02/webrev/share/vm/jfr/jfrClassAdapter.cpp.frames.html >> >> >> The HandleMark's at 1743 & 1784 can be not needed anymore. > > Thank you for reviewing! I suspect many HandleMarks lying around the > code aren't needed anymore, because KlassHandles (and even > methodHandles) aren't cleaned up using HandleMark. I've removed > those two (and ran some retests because this is getting dangerous). > > Are you going to review the open too? Yes, but now I'm waiting for webrev version 03. This link is not resolved for me: http://cr.openjdk.java.net/~coleenp/8155672.03/webrev Thanks, Serguei > > thank you! > Coleen >> >> Thanks, >> Serguei >> >> >> On 3/12/17 21:18, coleen.phillimore at oracle.com wrote: >>> >>> As requested by David on the closed part of this review, I fixed >>> variable names of the form h_ik and klass_h where the 'h' indicated >>> that the klass was in a handle. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>> >>> Also, fixing InstanceKlass to not pass this_k can be done in a >>> separate cleanup. >>> >>> Thanks, >>> Coleen >>> >>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>> handles with no implementation. >>>> >>>> These are fairly extensive changes. The KlassHandle types have >>>> been dummy types since permgen elimination and were thought to be >>>> useful for future features. They aren't, so can be removed (see >>>> bug for more details). This allows stricter typing because you >>>> can use the more specific type, and using const more. I didn't add >>>> const to these changes, because of the cascading effect. >>>> >>>> The change isn't hard to review, just tedious. The main bug that I >>>> had was redeclaring a type inside a scope, and >>>> InstanceKlass::cast(k) can't take a NULL k, whereas >>>> instanceKlassHandle ik(THREAD, k) could. >>>> >>>> It's so nice being able to single step on gdb without going into >>>> KlassHandle constructors! >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>> >>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>> JPRT. >>>> >>>> I can continue to hold this change for a convenient time for >>>> merging purposes with other people's JDK10 changes that they've >>>> been holding, or if there are other jdk9 changes that are likely to >>>> cause a problem for merging. I'll update the copyrights to 2017 on >>>> commit. >>>> >>>> Thanks, >>>> Coleen >>> >> > From coleen.phillimore at oracle.com Mon Mar 13 19:54:56 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 15:54:56 -0400 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: References: Message-ID: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> Hi Thomas, Thank you for cleaning up the metaspace code. I think the change looks good. I have only one comment and that is: can you put ChunkManager::size_by_index() function definition up in the file with the other ChunkManager functions? If Mikael reviews the test, I'd be happy to sponsor this for you. Sorry about the inattention. thanks! Coleen On 3/11/17 4:08 AM, Thomas St?fe wrote: > Ping... Did I send this to the wrong mailing list? > > On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe > wrote: > >> (Resent to hotspot-dev) >> >> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >> wrote: >> >>> Hi all, >>> >>> I would like to get a review for this cleanup change for the >>> metaspace.cpp. This has been a long time brewing in my backlog, but I had >>> to wait until jdk10 is open. The cleanup is actually quite small, the >>> largest part of the change is the added test coding. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>> >>> The change implements a suggestion made by Mikael Gerdin when >>> reviewing JDK-8170520 last year, who suggested that the ChunkManager class >>> should handle returns of both non-humongous and humongous chunks, see >>> original discussion here: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>> 016-November/021949.html >>> >>> "It would be great if you could have a look at unifying the >>> ChunkManager::return_chunks API for standard and humongous chunks so >>> that we wouldn't need the special casing for this in ~SpaceManager >>> That way we could also make humongous_dictionary() private to >>> ChunkManager which would improve encapsulation." >>> >>> The cleanup does this and a bit more, all changes having to do with the >>> fact that the ChunkManager class unnecessarily exposes internals to the >>> other classes in metaspace.cpp and that with a little bit of reorganization >>> this can be avoided. The benefit is simpler coding and way better >>> encapsulation, which will help a lot with future changes (in preparation >>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>> >>> -------- >>> >>> The changes in detail: >>> >>> The main issue was that ChunkManager::return_chunks() did not handle >>> humongous chunks. To return humongous chunks, one had to access the >>> humongous chunk dictionary inside the ChunkManager and add the chunk >>> yourself, including maintaining all counters. This happened in >>> ~SpaceManager and made this destructor very complicated. >>> >>> This is solved by moving the handling of humongous chunk returns to the >>> ChunkManager class itself. For convenience, I split the old >>> "ChunkManager::return_chunks" method into two new ones, >>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>> the ChunkManager without walking the chunk chain, and >>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>> the ChunkManager. Both functions are now able to handle both non-humongous >>> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >>> simplified) and just calls "ChunkManager::return_chunk_list()" for all >>> its chunk lists. >>> >>> So now we can remove the public access to the humongous chunk dictionary >>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>> function private. >>> >>> ---- >>> >>> Then there was the fact that the non-humongous chunk lists were also >>> exposed to public via ChunkManager::free_chunks(). The only reason was that >>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free lists >>> to find out the size of the items of the free lists. >>> >>> This was solved by adding a new function, ChunkManager::size_by_index(), >>> which returns the size of the items of a chunk list given by its chunk >>> index. >>> >>> So ChunkManager::free_chunks() could be made private too. >>> >>> --- >>> >>> The rest are minor changes: >>> >>> - made ChunkManager::find_free_chunks_list() private because it was not >>> used from outside the class >>> - Fixed an assert in dec_free_chunks_total() >>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >>> to keep the existing logging intact or add useful logging where possible. >>> >>> ---- >>> >>> A large chunk of the changes is the added gtests. There is a new class >>> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >>> code. >>> >>> Note that I dislike the fact that this test is implemented inside >>> metaspace.cpp itself. For now I kept my new tests like the existing tests >>> inside this file, but I think these tests should be moved to >>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>> separate fix though, because it needs a bit of remodeling (the tests need >>> to access internals in metaspace.cpp). >>> >>> ---- >>> >>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >>> AIX. No issues popped up which were associated with my change. >>> >>> Thanks for reviewing and Kind Regards, Thomas >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> From coleen.phillimore at oracle.com Mon Mar 13 20:04:21 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 16:04:21 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> Message-ID: On 3/13/17 3:42 PM, serguei.spitsyn at oracle.com wrote: > On 3/13/17 12:34, coleen.phillimore at oracle.com wrote: >> >> >> On 3/13/17 2:31 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> It looks good. >>> Nice cleanup! >>> >>> http://oklahoma.us.oracle.com/~cphillim/webrev/8155627.src.closed.02/webrev/share/vm/jfr/jfrClassAdapter.cpp.frames.html >>> >>> >>> The HandleMark's at 1743 & 1784 can be not needed anymore. >> >> Thank you for reviewing! I suspect many HandleMarks lying around >> the code aren't needed anymore, because KlassHandles (and even >> methodHandles) aren't cleaned up using HandleMark. I've removed >> those two (and ran some retests because this is getting dangerous). >> >> Are you going to review the open too? > > Yes, but now I'm waiting for webrev version 03. > This link is not resolved for me: > http://cr.openjdk.java.net/~coleenp/8155672.03/webrev It's there now. I transposed the digits in the last webrev. Thanks for letting me know. Coleen > > Thanks, > Serguei > >> >> thank you! >> Coleen >>> >>> Thanks, >>> Serguei >>> >>> >>> On 3/12/17 21:18, coleen.phillimore at oracle.com wrote: >>>> >>>> As requested by David on the closed part of this review, I fixed >>>> variable names of the form h_ik and klass_h where the 'h' indicated >>>> that the klass was in a handle. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>> >>>> Also, fixing InstanceKlass to not pass this_k can be done in a >>>> separate cleanup. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, >>>>> remove handles with no implementation. >>>>> >>>>> These are fairly extensive changes. The KlassHandle types have >>>>> been dummy types since permgen elimination and were thought to be >>>>> useful for future features. They aren't, so can be removed (see >>>>> bug for more details). This allows stricter typing because you >>>>> can use the more specific type, and using const more. I didn't >>>>> add const to these changes, because of the cascading effect. >>>>> >>>>> The change isn't hard to review, just tedious. The main bug that >>>>> I had was redeclaring a type inside a scope, and >>>>> InstanceKlass::cast(k) can't take a NULL k, whereas >>>>> instanceKlassHandle ik(THREAD, k) could. >>>>> >>>>> It's so nice being able to single step on gdb without going into >>>>> KlassHandle constructors! >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>>> >>>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>>> JPRT. >>>>> >>>>> I can continue to hold this change for a convenient time for >>>>> merging purposes with other people's JDK10 changes that they've >>>>> been holding, or if there are other jdk9 changes that are likely >>>>> to cause a problem for merging. I'll update the copyrights to 2017 >>>>> on commit. >>>>> >>>>> Thanks, >>>>> Coleen >>>> >>> >> > From serguei.spitsyn at oracle.com Mon Mar 13 20:05:44 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 13 Mar 2017 13:05:44 -0700 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> Message-ID: On 3/13/17 13:04, coleen.phillimore at oracle.com wrote: > > > On 3/13/17 3:42 PM, serguei.spitsyn at oracle.com wrote: >> On 3/13/17 12:34, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 3/13/17 2:31 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Coleen, >>>> >>>> It looks good. >>>> Nice cleanup! >>>> >>>> http://oklahoma.us.oracle.com/~cphillim/webrev/8155627.src.closed.02/webrev/share/vm/jfr/jfrClassAdapter.cpp.frames.html >>>> >>>> >>>> The HandleMark's at 1743 & 1784 can be not needed anymore. >>> >>> Thank you for reviewing! I suspect many HandleMarks lying around >>> the code aren't needed anymore, because KlassHandles (and even >>> methodHandles) aren't cleaned up using HandleMark. I've removed >>> those two (and ran some retests because this is getting dangerous). >>> >>> Are you going to review the open too? >> >> Yes, but now I'm waiting for webrev version 03. >> This link is not resolved for me: >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev > > It's there now. I transposed the digits in the last webrev. Thanks > for letting me know. Thanks! Serguei > > Coleen > >> >> Thanks, >> Serguei >> >>> >>> thank you! >>> Coleen >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 3/12/17 21:18, coleen.phillimore at oracle.com wrote: >>>>> >>>>> As requested by David on the closed part of this review, I fixed >>>>> variable names of the form h_ik and klass_h where the 'h' >>>>> indicated that the klass was in a handle. >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>> >>>>> Also, fixing InstanceKlass to not pass this_k can be done in a >>>>> separate cleanup. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, >>>>>> remove handles with no implementation. >>>>>> >>>>>> These are fairly extensive changes. The KlassHandle types have >>>>>> been dummy types since permgen elimination and were thought to be >>>>>> useful for future features. They aren't, so can be removed (see >>>>>> bug for more details). This allows stricter typing because you >>>>>> can use the more specific type, and using const more. I didn't >>>>>> add const to these changes, because of the cascading effect. >>>>>> >>>>>> The change isn't hard to review, just tedious. The main bug that >>>>>> I had was redeclaring a type inside a scope, and >>>>>> InstanceKlass::cast(k) can't take a NULL k, whereas >>>>>> instanceKlassHandle ik(THREAD, k) could. >>>>>> >>>>>> It's so nice being able to single step on gdb without going into >>>>>> KlassHandle constructors! >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>>>> >>>>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>>>> JPRT. >>>>>> >>>>>> I can continue to hold this change for a convenient time for >>>>>> merging purposes with other people's JDK10 changes that they've >>>>>> been holding, or if there are other jdk9 changes that are likely >>>>>> to cause a problem for merging. I'll update the copyrights to >>>>>> 2017 on commit. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>> >>>> >>> >> > From sybersnake at gmail.com Mon Mar 13 23:10:42 2017 From: sybersnake at gmail.com (Jon V.) Date: Mon, 13 Mar 2017 19:10:42 -0400 Subject: AW: AW: How to pin/avoid gc for an oop? In-Reply-To: <000001d29c16$a6b85ec0$f4291c40$@hispeed.ch> References: <001501d298fa$35f3ecf0$a1dbc6d0$@hispeed.ch> <39c99202-129c-47a1-85e2-2dcc1836e8c8@typeapp.com> <5d6a216d-371c-4241-85d8-d6e21a13b5a8@typeapp.com> <000001d29a57$d9c7fc00$8d57f400$@hispeed.ch> <000301d29aca$fe862080$fb926180$@hispeed.ch> <000001d29b95$1353d820$39fb8860$@hispeed.ch> <58C68F32.7070108@oracle.com> <05517514-3125-c81a-58f7-9d2d12a7f16c@redhat.com> <58C693DD.40604@oracle.com> <000001d29c16$a6b85ec0$f4291c40$@hispeed.ch> Message-ID: Just a quick note: JNI is always a safe-point for GC. JNI Critical Access prevents the GC from running. JNI API calls always have to acquire and release the GC lock for every method. NewGlobalRef must be explicitly deleted otherwise they will exist forever. On Mon, Mar 13, 2017 at 12:27 PM, Christian Hagedorn wrote: > Thank you all for your help and your suggestions! :) > > That is right I only need a way to still reach it but not actually pin it. > It seems to be working by creating a global ref with > JNIHandles::make_global(handle), which is exactly what the > jni_NewGlobalRef function does pointed out by Ioi. > The GC did not remove the oops. > > Christian > > -----Urspr?ngliche Nachricht----- > Von: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] Im Auftrag > von Ioi Lam > Gesendet: Montag, 13. M?rz 2017 13:43 > An: Aleksey Shipilev; hotspot-dev at openjdk.java.net > Betreff: Re: AW: AW: How to pin/avoid gc for an oop? > > I don't think Christian actually needs object pinning (as in avoiding > relocation of an object due to GC). > > - Ioi > > On 3/13/17 8:36 PM, Aleksey Shipilev wrote: > > Handle-izing / JNIRef-ing the oop != pinning. > > > > a) If you want pinning, you have to cooperate with GC somehow. JNI has > > "critical" operations for this, so it would be instructive to see > > jni_GetPrimitiveArrayCritical does this with GCLocker. > > > > b) If you just want the reachability, and are in native code, then JNI > > GlobalRef would probably be the cleanest choice. > > > > c) If you just want the reachability, and are in Java code, then > > calling java.lang.ref.Reference.reachabilityFence would retain the > > local var in the thread roots. > > > > -Aleksey > > > > On 03/13/2017 01:23 PM, Ioi Lam wrote: > >> How about using JNI NewGlobalRef/DeleteGlobalRef? Unlike the C++ > >> "Handle" class (which is scoped and automatically reclaimed), a > >> reference created by NewGlobalRef remains valid until it's explicitly > deleted using DeleteGlobalRef. > >> > >> - Ioi > >> > >> On 3/13/17 8:59 AM, Christian Hagedorn wrote: > >>> So handles are not really an option. What else could I do to make my > >>> undo log a GC root to ensure reachability of the oops in it? > >>> > >>> Christian > >>> > >>> -----Urspr?ngliche Nachricht----- > >>> Von: David Holmes [mailto:david.holmes at oracle.com] > >>> Gesendet: Sonntag, 12. M?rz 2017 06:05 > >>> An: Jon V.; Christian Hagedorn; hotspot-dev at openjdk.java.net > >>> Betreff: Re: AW: How to pin/avoid gc for an oop? > >>> > >>> On 12/03/2017 10:59 AM, Jon V. wrote: > >>>> I don't believe that handles will keep the objects from being > >>>> garbage collected. > >>> Handles are GC roots. Or rather the per-thread handle area is a GC > root. > >>> > >>>> Hotspot does not support object pinning. > >>> This isn't pinning - the object can still be moved. But the Handle > >>> ensures it remains reachable. > >>> > >>>> Need input from a GC dev > >>>> > >>>> On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn > >>>> > >>>> wrote: > >>>> > >>>>> The undo log would be stored as an array of handles inside a > >>>>> thread and does not refer to other threads. Would that work then? > >>> Handles are intended to be scoped and reclaimed using HandleMarks. > >>> This seem to require a different kind of usage. > >>> > >>> David > >>> ----- > >>> > >>>>> Christian > >>>>> > >>>>> -----Urspr?ngliche Nachricht----- > >>>>> Von: David Holmes [mailto:david.holmes at oracle.com] > >>>>> Gesendet: Sonntag, 12. M?rz 2017 00:39 > >>>>> An: Christian Hagedorn; hotspot-dev at openjdk.java.net > >>>>> Betreff: Re: AW: How to pin/avoid gc for an oop? > >>>>> > >>>>> On 11/03/2017 9:08 PM, Christian Hagedorn wrote: > >>>>>> Thank you for your answers. If it is possible to do it in the JVM > >>>>>> I > >>>>> would prefer to do that. > >>>>>> I have tried to maintain an array of handles. However I think I > >>>>>> have not > >>>>> understood its usage correctly. I only saw examples by allocating > >>>>> it on the stack with Handle h1(thread, oop). But how can I create > >>>>> a handle on the heap such that it is kept alive for an access > >>>>> later? It asserts not to call the global operator "new" such that > "new Handle(..)" > >>>>> does not work. > >>>>> > >>>>> Sorry that was incorrect advice. Handles are per-thread so you > >>>>> can't have a data structure storing them across threads. > >>>>> > >>>>>> How could I make the log a GC root? > >>>>> GC folk should chime in on the details of that - I don't want to > >>>>> give more bad advice. > >>>>> > >>>>> David > >>>>> > >>>>>> Christian > >>>>>> > >>>>>> -----Urspr?ngliche Nachricht----- > >>>>>> Von: David Holmes [mailto:david.holmes at oracle.com] > >>>>>> Gesendet: Samstag, 11. M?rz 2017 04:58 > >>>>>> An: Christian; Jon V. > >>>>>> Cc: hotspot-dev at openjdk.java.net > >>>>>> Betreff: Re: How to pin/avoid gc for an oop? > >>>>>> > >>>>>> On 10/03/2017 6:56 PM, Christian wrote: > >>>>>>> Hi Jon, > >>>>>>> > >>>>>>> The STM is to be integrated into the JVM, but the transactional > >>>>>>> execution only affects modifications to the managed memory done > >>>>>>> by bytecode instructions. JVM internal structures are unaffected. > >>>>>>> > >>>>>>> Here an example: > >>>>>>> > >>>>>>> class X { > >>>>>>> int f; > >>>>>>> > >>>>>>> public static update(X x, int _new) { > >>>>>>> atomic { // tx start > >>>>>>> // (1) update field f of x: > >>>>>>> > >>>>>>> // undo log maintains a reference to x and old value of > f > >>>>>>> // note: x contains a direct reference to the lock > >>>>>>> > >>>>>>> x.f = _new; > >>>>>>> > >>>>>>> // (2) assume GC is run > >>>>>>> // Note: The atomic section can be large and blocking, > >>>>>>> // thus, inhibiting a GC run is not an option. > >>>>>>> > >>>>>>> // (3) End of the atomic section (tx end): > >>>>>>> // 1. In case of abort: restore old value > >>>>>>> // 2. Release lock (obtained via reference to x) > >>>>>>> } > >>>>>>> } > >>>>>>> } > >>>>>>> > >>>>>>> How can I assure that the reference to x in the undo log that I > >>>>>>> made at > >>>>>>> (1) remains valid at (3), even though a GC run may occur at (2)? > >>>>>>> I > >>>>>> My making sure the GC sees everything in the undo log. The > >>>>>> details will > >>>>> depend on how you propose to implement that log. > >>>>>>> thought that Handles (share/vm/runtime/handles.hpp) may be an > >>>>>>> option, but I might have misunderstood what they are used for. I > >>>>>>> can of course > >>>>>> Handles are used inside the VM to protect oops when we need to > >>>>>> use them > >>>>> across calls that might lead to safepoint checks and thus to GC > occurring. > >>>>>>> make the undo log in the managed heap as well, but I hoped that > >>>>>>> this can be done directly in the JVM. > >>>>>> There are a number of ways to do this. If your log is a native VM > >>>>>> data > >>>>> structure then it might store Handles. Or you could make the log a > >>>>> GC root and make it known to the GC, and store oops directly. > >>>>>> Or you could implement it all in Java - which is probably a lot > >>>>>> simpler, > >>>>> albeit likely less performant. > >>>>>> David > >>>>>> > >>>>>>> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." > >>>>>>> > >>>>> schrieb: > >>>>>>>> Are you trying to build a transactional memory system for the > >>>>>>>> JVM or your own Java code? > >>>>>>>> > >>>>>>>> On Thu, Mar 9, 2017 at 2:33 PM Christian > >>>>>>>> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi Jon, > >>>>>>>>> > >>>>>>>>> I need this for an undo log of a transactional memory system > >>>>>>>>> with pessimistic locking/eager version management. > >>>>>>>>> Am 9. M?rz 2017, um 19:00, "Jon V." > schrieb: > >>>>>>>>> > >>>>>>>>> Can you clarify what you mean by Undo and why you think this > >>>>>>>>> should > >>>>>>>> be > >>>>>>>>> done at the VM level so we can better understand the request. > >>>>>>>>> > >>>>>>>>> If you don't want something to be garbage collected then > >>>>>>>>> simply don't dereference it. > >>>>>>>>> > >>>>>>>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn > >>>>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> Hi, > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> I want to implement an undo functionality and thus need to > >>>>>>>>> access an > >>>>>>>> oop > >>>>>>>>> even when it is out of scope in the Java code. How can I pin > >>>>>>>>> such an > >>>>>>>> oop to > >>>>>>>>> avoid garbage collection of its used memory? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Best regards, > >>>>>>>>> > >>>>>>>>> Christian > >>>>>>>>> > >>>>>>>>> > > > From ch-hagedorn at hispeed.ch Mon Mar 13 23:36:19 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Tue, 14 Mar 2017 00:36:19 +0100 Subject: AW: AW: How to pin/avoid gc for an oop? Message-ID: <002401d29c52$9ee7ac00$dcb70400$@hispeed.ch> Thanks for the clarifications. I have ensured to delete them properly afterwards. Christian Von: Jon V. [mailto:sybersnake at gmail.com] Gesendet: Dienstag, 14. M?rz 2017 00:11 An: Christian Hagedorn Cc: hotspot-dev Betreff: Re: AW: AW: How to pin/avoid gc for an oop? Just a quick note: JNI is always a safe-point for GC. JNI Critical Access prevents the GC from running. JNI API calls always have to acquire and release the GC lock for every method. NewGlobalRef must be explicitly deleted otherwise they will exist forever. On Mon, Mar 13, 2017 at 12:27 PM, Christian Hagedorn wrote: Thank you all for your help and your suggestions! :) That is right I only need a way to still reach it but not actually pin it. It seems to be working by creating a global ref with JNIHandles::make_global(handle), which is exactly what the jni_NewGlobalRef function does pointed out by Ioi. The GC did not remove the oops. Christian -----Urspr?ngliche Nachricht----- Von: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] Im Auftrag von Ioi Lam Gesendet: Montag, 13. M?rz 2017 13:43 An: Aleksey Shipilev; hotspot-dev at openjdk.java.net Betreff: Re: AW: AW: How to pin/avoid gc for an oop? I don't think Christian actually needs object pinning (as in avoiding relocation of an object due to GC). - Ioi On 3/13/17 8:36 PM, Aleksey Shipilev wrote: > Handle-izing / JNIRef-ing the oop != pinning. > > a) If you want pinning, you have to cooperate with GC somehow. JNI has > "critical" operations for this, so it would be instructive to see > jni_GetPrimitiveArrayCritical does this with GCLocker. > > b) If you just want the reachability, and are in native code, then JNI > GlobalRef would probably be the cleanest choice. > > c) If you just want the reachability, and are in Java code, then > calling java.lang.ref.Reference.reachabilityFence would retain the > local var in the thread roots. > > -Aleksey > > On 03/13/2017 01:23 PM, Ioi Lam wrote: >> How about using JNI NewGlobalRef/DeleteGlobalRef? Unlike the C++ >> "Handle" class (which is scoped and automatically reclaimed), a >> reference created by NewGlobalRef remains valid until it's explicitly deleted using DeleteGlobalRef. >> >> - Ioi >> >> On 3/13/17 8:59 AM, Christian Hagedorn wrote: >>> So handles are not really an option. What else could I do to make my >>> undo log a GC root to ensure reachability of the oops in it? >>> >>> Christian >>> >>> -----Urspr?ngliche Nachricht----- >>> Von: David Holmes [mailto:david.holmes at oracle.com] >>> Gesendet: Sonntag, 12. M?rz 2017 06:05 >>> An: Jon V.; Christian Hagedorn; hotspot-dev at openjdk.java.net >>> Betreff: Re: AW: How to pin/avoid gc for an oop? >>> >>> On 12/03/2017 10:59 AM, Jon V. wrote: >>>> I don't believe that handles will keep the objects from being >>>> garbage collected. >>> Handles are GC roots. Or rather the per-thread handle area is a GC root. >>> >>>> Hotspot does not support object pinning. >>> This isn't pinning - the object can still be moved. But the Handle >>> ensures it remains reachable. >>> >>>> Need input from a GC dev >>>> >>>> On Sat, Mar 11, 2017 at 7:53 PM Christian Hagedorn >>>> >>>> wrote: >>>> >>>>> The undo log would be stored as an array of handles inside a >>>>> thread and does not refer to other threads. Would that work then? >>> Handles are intended to be scoped and reclaimed using HandleMarks. >>> This seem to require a different kind of usage. >>> >>> David >>> ----- >>> >>>>> Christian >>>>> >>>>> -----Urspr?ngliche Nachricht----- >>>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>>> Gesendet: Sonntag, 12. M?rz 2017 00:39 >>>>> An: Christian Hagedorn; hotspot-dev at openjdk.java.net >>>>> Betreff: Re: AW: How to pin/avoid gc for an oop? >>>>> >>>>> On 11/03/2017 9:08 PM, Christian Hagedorn wrote: >>>>>> Thank you for your answers. If it is possible to do it in the JVM >>>>>> I >>>>> would prefer to do that. >>>>>> I have tried to maintain an array of handles. However I think I >>>>>> have not >>>>> understood its usage correctly. I only saw examples by allocating >>>>> it on the stack with Handle h1(thread, oop). But how can I create >>>>> a handle on the heap such that it is kept alive for an access >>>>> later? It asserts not to call the global operator "new" such that "new Handle(..)" >>>>> does not work. >>>>> >>>>> Sorry that was incorrect advice. Handles are per-thread so you >>>>> can't have a data structure storing them across threads. >>>>> >>>>>> How could I make the log a GC root? >>>>> GC folk should chime in on the details of that - I don't want to >>>>> give more bad advice. >>>>> >>>>> David >>>>> >>>>>> Christian >>>>>> >>>>>> -----Urspr?ngliche Nachricht----- >>>>>> Von: David Holmes [mailto:david.holmes at oracle.com] >>>>>> Gesendet: Samstag, 11. M?rz 2017 04:58 >>>>>> An: Christian; Jon V. >>>>>> Cc: hotspot-dev at openjdk.java.net >>>>>> Betreff: Re: How to pin/avoid gc for an oop? >>>>>> >>>>>> On 10/03/2017 6:56 PM, Christian wrote: >>>>>>> Hi Jon, >>>>>>> >>>>>>> The STM is to be integrated into the JVM, but the transactional >>>>>>> execution only affects modifications to the managed memory done >>>>>>> by bytecode instructions. JVM internal structures are unaffected. >>>>>>> >>>>>>> Here an example: >>>>>>> >>>>>>> class X { >>>>>>> int f; >>>>>>> >>>>>>> public static update(X x, int _new) { >>>>>>> atomic { // tx start >>>>>>> // (1) update field f of x: >>>>>>> >>>>>>> // undo log maintains a reference to x and old value of f >>>>>>> // note: x contains a direct reference to the lock >>>>>>> >>>>>>> x.f = _new; >>>>>>> >>>>>>> // (2) assume GC is run >>>>>>> // Note: The atomic section can be large and blocking, >>>>>>> // thus, inhibiting a GC run is not an option. >>>>>>> >>>>>>> // (3) End of the atomic section (tx end): >>>>>>> // 1. In case of abort: restore old value >>>>>>> // 2. Release lock (obtained via reference to x) >>>>>>> } >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> How can I assure that the reference to x in the undo log that I >>>>>>> made at >>>>>>> (1) remains valid at (3), even though a GC run may occur at (2)? >>>>>>> I >>>>>> My making sure the GC sees everything in the undo log. The >>>>>> details will >>>>> depend on how you propose to implement that log. >>>>>>> thought that Handles (share/vm/runtime/handles.hpp) may be an >>>>>>> option, but I might have misunderstood what they are used for. I >>>>>>> can of course >>>>>> Handles are used inside the VM to protect oops when we need to >>>>>> use them >>>>> across calls that might lead to safepoint checks and thus to GC occurring. >>>>>>> make the undo log in the managed heap as well, but I hoped that >>>>>>> this can be done directly in the JVM. >>>>>> There are a number of ways to do this. If your log is a native VM >>>>>> data >>>>> structure then it might store Handles. Or you could make the log a >>>>> GC root and make it known to the GC, and store oops directly. >>>>>> Or you could implement it all in Java - which is probably a lot >>>>>> simpler, >>>>> albeit likely less performant. >>>>>> David >>>>>> >>>>>>> Am 9. M?rz 2017, 21:12, um 21:12, "Jon V." >>>>>>> >>>>> schrieb: >>>>>>>> Are you trying to build a transactional memory system for the >>>>>>>> JVM or your own Java code? >>>>>>>> >>>>>>>> On Thu, Mar 9, 2017 at 2:33 PM Christian >>>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Jon, >>>>>>>>> >>>>>>>>> I need this for an undo log of a transactional memory system >>>>>>>>> with pessimistic locking/eager version management. >>>>>>>>> Am 9. M?rz 2017, um 19:00, "Jon V." schrieb: >>>>>>>>> >>>>>>>>> Can you clarify what you mean by Undo and why you think this >>>>>>>>> should >>>>>>>> be >>>>>>>>> done at the VM level so we can better understand the request. >>>>>>>>> >>>>>>>>> If you don't want something to be garbage collected then >>>>>>>>> simply don't dereference it. >>>>>>>>> >>>>>>>>> On Thu, Mar 9, 2017 at 12:38 PM Christian Hagedorn >>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I want to implement an undo functionality and thus need to >>>>>>>>> access an >>>>>>>> oop >>>>>>>>> even when it is out of scope in the Java code. How can I pin >>>>>>>>> such an >>>>>>>> oop to >>>>>>>>> avoid garbage collection of its used memory? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> >>>>>>>>> Christian >>>>>>>>> >>>>>>>>> From david.holmes at oracle.com Mon Mar 13 23:46:42 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 14 Mar 2017 09:46:42 +1000 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> Message-ID: On 13/03/2017 10:37 PM, coleen.phillimore at oracle.com wrote: > > David, I'm so sorry, I should have pointed to > > open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev Oops I should have noticed I was re-checking the same .01 version :) Never mind. src/share/vm/jvmci/jvmciEnv.cpp 282 methodHandle JVMCIEnv::lookup_method(InstanceKlass* h_accessor, 283 InstanceKlass* h_holder, the .hpp was fixed but not the .cpp. Otherwise I think they have all been fixed. Thanks, David > On 3/13/17 2:23 AM, David Holmes wrote: >> Hi Coleen, >> >> Wow that was a hard slog! :) >> >> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>> >>> As requested by David on the closed part of this review, I fixed >>> variable names of the form h_ik and klass_h where the 'h' indicated that >>> the klass was in a handle. >> >> Thanks for that. But I'm afraid there are quite a number of other >> patterns including the "h" still present: >> >> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved (though >> that code uses h_ when there doesn't appear to be a handle in use) >> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >> InstanceKlass* h_cp_entry_f1 >> src/share/vm/prims/jni.cpp: Klass* h_k >> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh >> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >> _ikh, InstanceKlass* ikh() >> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >> instanceK_h >> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >> src/share/vm/prims/nativeLookup.cpp: Klass* kh >> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >> > > These were the ones I fixed and retested. >> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >> >> There are numerous code indentation issues with some of the changed >> code (mainly around parameter/argument lists) - but they all seem to >> be pre-existing so I decided not to try and call them out. You did fix >> quite a few, and I don't think you introduced any new ones. :) >> >> The main thing to look for with use of handles internal to methods is >> whether once changed to a direct Klass/InstanceKlass*, the variable >> becomes redundant. You picked up on a few of these but I spotted a >> couple of others (reading the patch file the context isn't always >> sufficient to spot these): >> >> share/vm/c1/c1_Runtime1.cpp >> >> { Klass* klass = resolve_field_return_klass(caller_method, >> bci, CHECK); >> - init_klass = KlassHandle(THREAD, klass); >> + init_klass = klass; >> >> You don't need the klass local but can assign direct to init_klass. >> >> // convert to handle >> - load_klass = KlassHandle(THREAD, k); >> + load_klass = k; >> >> Comment should be deleted, plus it seems you should be able to assign >> to load_klass directly earlier through the switch instead of >> introducing 'k'. > > This logic is not obvious and I'd rather not change it. init_klass and > load_klass are used further down for different purposes. I removed the > comment though. >> >> --- >> >> src/share/vm/jvmci/jvmciEnv.cpp >> >> 117 Klass* kls; >> >> This local is not needed as you can assign directly to found_klass now. > > Yes, that's better. >> >> --- >> >> Finally a meta-question re the changes in: >> >> src/share/vm/classfile/dictionary.hpp >> >> and the flow out from that. I'm unclear about all the changes from >> Klass to InstanceKlass. Are all dictionary/hashtable entries >> guaranteed to be instance classes? > > Yes, they are. This avoided multiple InstanceKlass::cast() calls. >> >> --- >> >> >>> Also, fixing InstanceKlass to not pass this_k can be done in a separate >>> cleanup. >> >> Ok. > > Thank you for slogging through all of this. > > Coleen >> >> Thanks, >> David >> >>> Thanks, >>> Coleen >>> >>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>> handles with no implementation. >>>> >>>> These are fairly extensive changes. The KlassHandle types have been >>>> dummy types since permgen elimination and were thought to be useful >>>> for future features. They aren't, so can be removed (see bug for more >>>> details). This allows stricter typing because you can use the more >>>> specific type, and using const more. I didn't add const to these >>>> changes, because of the cascading effect. >>>> >>>> The change isn't hard to review, just tedious. The main bug that I >>>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>>> >>>> It's so nice being able to single step on gdb without going into >>>> KlassHandle constructors! >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>> >>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>> java/lang/instrument tests, all closed tonga colocated tests, and JPRT. >>>> >>>> I can continue to hold this change for a convenient time for merging >>>> purposes with other people's JDK10 changes that they've been holding, >>>> or if there are other jdk9 changes that are likely to cause a problem >>>> for merging. I'll update the copyrights to 2017 on commit. >>>> >>>> Thanks, >>>> Coleen >>> > From coleen.phillimore at oracle.com Tue Mar 14 00:01:49 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 20:01:49 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <31332df0-9074-92d8-785b-e77134b152fb@oracle.com> Message-ID: On 3/13/17 7:46 PM, David Holmes wrote: > On 13/03/2017 10:37 PM, coleen.phillimore at oracle.com wrote: >> >> David, I'm so sorry, I should have pointed to >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.02/webrev > > Oops I should have noticed I was re-checking the same .01 version :) > > Never mind. > > src/share/vm/jvmci/jvmciEnv.cpp > > 282 methodHandle JVMCIEnv::lookup_method(InstanceKlass* h_accessor, > 283 InstanceKlass* h_holder, > > the .hpp was fixed but not the .cpp. got it! Thanks!! Coleen > > Otherwise I think they have all been fixed. > > Thanks, > David > >> On 3/13/17 2:23 AM, David Holmes wrote: >>> Hi Coleen, >>> >>> Wow that was a hard slog! :) >>> >>> On 13/03/2017 2:18 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> As requested by David on the closed part of this review, I fixed >>>> variable names of the form h_ik and klass_h where the 'h' indicated >>>> that >>>> the klass was in a handle. >>> >>> Thanks for that. But I'm afraid there are quite a number of other >>> patterns including the "h" still present: >>> >>> src/share/vm/aot/aotCodeHeap.cpp/.hpp: InstanceKlass* kh >>> src/share/vm/c1/c1_Runtime1.cpp: InstanceKlass* h >>> src/share/vm/ci/ciArrayKlass.cpp: Klass* h_k >>> src/share/vm/ci/ciInstanceKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciMethod.cpp: Klass* h_recv, Klass* h_resolved (though >>> that code uses h_ when there doesn't appear to be a handle in use) >>> src/share/vm/ci/ciObjArrayKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/ci/ciTypeArrayKlass.cpp/.hpp: Klass* h_k >>> src/share/vm/interpreter/interpreterRuntime.cpp: Klass* h_klass, >>> InstanceKlass* h_cp_entry_f1 >>> src/share/vm/prims/jni.cpp: Klass* h_k >>> src/share/vm/prims/jvm.cpp: InstanceKlass* kh, InstanceKlass* ik_h >>> src/share/vm/prims/jvmtiClassFileReconstituter.cpp: InstanceKlass* iikh >>> src/share/vm/prims/jvmtiClassFileReconstituter.hpp: InstanceKlass* >>> _ikh, InstanceKlass* ikh() >>> src/share/vm/prims/jvmtiEnv.cpp: InstanceKlass* ikh, InstanceKlass* >>> instanceK_h >>> src/share/vm/prims/jvmtiEnvBase.cpp: Klass* ob_kh >>> src/share/vm/prims/jvmtiExport.cpp: Klass* _h_class_being_redefined >>> src/share/vm/prims/jvmtiImpl.cpp: InstanceKlass* ikh, Klass* ob_kh >>> src/share/vm/prims/jvmtiRedefineClasses.cpp: InstanceKlass* ikh >>> src/share/vm/prims/jvmtiTagMap.cpp: InstanceKlass* ikh >>> src/share/vm/prims/jvmtiThreadState.hpp: Klass* h_class >>> src/share/vm/prims/nativeLookup.cpp: Klass* kh >>> src/share/vm/prims/whitebox.cpp: InstanceKlass* ikh >>> src/share/vm/runtime/sharedRuntime.cpp: Klass* h_klass >>> src/share/vm/services/gcNotifier.cpp: InstanceKlass* mu_kh >>> src/share/vm/services/heapDumper.cpp: InstanceKlass* ikh >>> >> >> These were the ones I fixed and retested. >>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>> >>> There are numerous code indentation issues with some of the changed >>> code (mainly around parameter/argument lists) - but they all seem to >>> be pre-existing so I decided not to try and call them out. You did fix >>> quite a few, and I don't think you introduced any new ones. :) >>> >>> The main thing to look for with use of handles internal to methods is >>> whether once changed to a direct Klass/InstanceKlass*, the variable >>> becomes redundant. You picked up on a few of these but I spotted a >>> couple of others (reading the patch file the context isn't always >>> sufficient to spot these): >>> >>> share/vm/c1/c1_Runtime1.cpp >>> >>> { Klass* klass = resolve_field_return_klass(caller_method, >>> bci, CHECK); >>> - init_klass = KlassHandle(THREAD, klass); >>> + init_klass = klass; >>> >>> You don't need the klass local but can assign direct to init_klass. >>> >>> // convert to handle >>> - load_klass = KlassHandle(THREAD, k); >>> + load_klass = k; >>> >>> Comment should be deleted, plus it seems you should be able to assign >>> to load_klass directly earlier through the switch instead of >>> introducing 'k'. >> >> This logic is not obvious and I'd rather not change it. init_klass and >> load_klass are used further down for different purposes. I removed the >> comment though. >>> >>> --- >>> >>> src/share/vm/jvmci/jvmciEnv.cpp >>> >>> 117 Klass* kls; >>> >>> This local is not needed as you can assign directly to found_klass now. >> >> Yes, that's better. >>> >>> --- >>> >>> Finally a meta-question re the changes in: >>> >>> src/share/vm/classfile/dictionary.hpp >>> >>> and the flow out from that. I'm unclear about all the changes from >>> Klass to InstanceKlass. Are all dictionary/hashtable entries >>> guaranteed to be instance classes? >> >> Yes, they are. This avoided multiple InstanceKlass::cast() calls. >>> >>> --- >>> >>> >>>> Also, fixing InstanceKlass to not pass this_k can be done in a >>>> separate >>>> cleanup. >>> >>> Ok. >> >> Thank you for slogging through all of this. >> >> Coleen >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Coleen >>>> >>>> On 3/12/17 11:32 AM, coleen.phillimore at oracle.com wrote: >>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, remove >>>>> handles with no implementation. >>>>> >>>>> These are fairly extensive changes. The KlassHandle types have been >>>>> dummy types since permgen elimination and were thought to be useful >>>>> for future features. They aren't, so can be removed (see bug for >>>>> more >>>>> details). This allows stricter typing because you can use the more >>>>> specific type, and using const more. I didn't add const to these >>>>> changes, because of the cascading effect. >>>>> >>>>> The change isn't hard to review, just tedious. The main bug that I >>>>> had was redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) could. >>>>> >>>>> It's so nice being able to single step on gdb without going into >>>>> KlassHandle constructors! >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8155672.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8155672 >>>>> >>>>> Tested with all hotspot jtreg tests, java/lang/invoke tests, >>>>> java/lang/instrument tests, all closed tonga colocated tests, and >>>>> JPRT. >>>>> >>>>> I can continue to hold this change for a convenient time for merging >>>>> purposes with other people's JDK10 changes that they've been holding, >>>>> or if there are other jdk9 changes that are likely to cause a problem >>>>> for merging. I'll update the copyrights to 2017 on commit. >>>>> >>>>> Thanks, >>>>> Coleen >>>> >> From serguei.spitsyn at oracle.com Tue Mar 14 00:10:12 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 13 Mar 2017 17:10:12 -0700 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> Message-ID: Coleen, It looks good to me. Below are some minor comments. I'm Ok to separate the HandleMark cleanup if you prefer it. http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html 2581 if ((name() == inner_name)) { Unneeded extra brackets (existed before the fix). http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/oops/klassVtable.cpp.frames.html 1080 HandleMark hm(THREAD); Possibly a redundant HandleMark. http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/prims/jvmtiImpl.cpp.frames.html 695 HandleMark hm(cur_thread); Possibly a redundant HandleMark. http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/prims/jvmtiRedefineClasses.cpp.frames.html 3853 InstanceKlass *ik = (InstanceKlass *)the_class; 4049 increment_class_counter((InstanceKlass *)the_class, THREAD); No need to cast to (InstanceKlass *) as the_class isInstanceKlass* . http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/runtime/vframe.cpp.frames.html 519 HandleMark hm; It seems to be redundant now. http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/services/classLoadingService.hpp.frames.html 124 // _current_thread is for creating a Klass* with a faster version constructor 125 static Thread* _current_thread; Does the _current_thread become redundant now? http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/services/heapDumper.cpp.frames.html It seems, the HandleMark's at 816, 849, 879, 894 are redundant. Thanks, Serguei On 3/13/17 13:04, coleen.phillimore at oracle.com wrote: > On 3/13/17 3:42 PM, serguei.spitsyn at oracle.com wrote: >> On 3/13/17 12:34, coleen.phillimore at oracle.com wrote: >>> On 3/13/17 2:31 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Coleen, It looks good. Nice cleanup! >>>> http://oklahoma.us.oracle.com/~cphillim/webrev/8155627.src.closed.02/webrev/share/vm/jfr/jfrClassAdapter.cpp.frames.html >>>> The HandleMark's at 1743 & 1784 can be not needed anymore. >>> Thank you for reviewing! I suspect many HandleMarks lying around >>> the code aren't needed anymore, because KlassHandles (and even >>> methodHandles) aren't cleaned up using HandleMark. I've removed >>> those two (and ran some retests because this is getting dangerous). >>> Are you going to review the open too? >> Yes, but now I'm waiting for webrev version 03. This link is not >> resolved for me: http://cr.openjdk.java.net/~coleenp/8155672.03/webrev > It's there now. I transposed the digits in the last webrev. Thanks > for letting me know. Coleen >> Thanks, Serguei >>> thank you! Coleen >>>> Thanks, Serguei On 3/12/17 21:18, coleen.phillimore at oracle.com wrote: >>>>> As requested by David on the closed part of this review, I fixed >>>>> variable names of the form h_ik and klass_h where the 'h' >>>>> indicated that the klass was in a handle. open webrev at >>>>> http://cr.openjdk.java.net/~coleenp/8155672.01/webrev Also, fixing >>>>> InstanceKlass to not pass this_k can be done in a separate >>>>> cleanup. Thanks, Coleen On 3/12/17 11:32 AM, >>>>> coleen.phillimore at oracle.com wrote: >>>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, >>>>>> remove handles with no implementation. These are fairly extensive >>>>>> changes. The KlassHandle types have been dummy types since >>>>>> permgen elimination and were thought to be useful for future >>>>>> features. They aren't, so can be removed (see bug for more >>>>>> details). This allows stricter typing because you can use the >>>>>> more specific type, and using const more. I didn't add const to >>>>>> these changes, because of the cascading effect. The change isn't >>>>>> hard to review, just tedious. The main bug that I had was >>>>>> redeclaring a type inside a scope, and InstanceKlass::cast(k) >>>>>> can't take a NULL k, whereas instanceKlassHandle ik(THREAD, k) >>>>>> could. It's so nice being able to single step on gdb without >>>>>> going into KlassHandle constructors! open webrev at >>>>>> http://cr.openjdk.java.net/~coleenp/8155672.01/webrev bug link >>>>>> https://bugs.openjdk.java.net/browse/JDK-8155672 Tested with all >>>>>> hotspot jtreg tests, java/lang/invoke tests, java/lang/instrument >>>>>> tests, all closed tonga colocated tests, and JPRT. I can continue >>>>>> to hold this change for a convenient time for merging purposes >>>>>> with other people's JDK10 changes that they've been holding, or >>>>>> if there are other jdk9 changes that are likely to cause a >>>>>> problem for merging. I'll update the copyrights to 2017 on >>>>>> commit. Thanks, Coleen From coleen.phillimore at oracle.com Tue Mar 14 00:23:06 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 13 Mar 2017 20:23:06 -0400 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> Message-ID: <10583606-0889-b4b6-dcce-4f3e93d44660@oracle.com> Hi Serguei, Thank you for getting through this big change. On 3/13/17 8:10 PM, serguei.spitsyn at oracle.com wrote: > Coleen, > > It looks good to me. > Below are some minor comments. > I'm Ok to separate the HandleMark cleanup if you prefer it. > Yes, I would like to clean up the HandleMarks with a different change. They're somewhat complicated because they used to clean up Handles for oops and Handles for metadata. Now they only apply to oops. But some of the places below do accumulate oops so they might be needed there. I think Handle should be a scoped object with a destructor, etc so that we don't need HandleMarks, but I haven't filed an RFE for that yet. > > http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html > > > 2581 if ((name() == inner_name)) { > > Unneeded extra brackets (existed before the fix). > > fixed. > > http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/oops/klassVtable.cpp.frames.html > > > 1080 HandleMark hm(THREAD); > > Possibly a redundant HandleMark. > > > http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/prims/jvmtiImpl.cpp.frames.html > > > 695 HandleMark hm(cur_thread); > > Possibly a redundant HandleMark. > > > > http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/prims/jvmtiRedefineClasses.cpp.frames.html > > > 3853 InstanceKlass *ik = (InstanceKlass *)the_class; 4049 > increment_class_counter((InstanceKlass *)the_class, THREAD); > > No need to cast to (InstanceKlass *) as the_class isInstanceKlass* . This was in commented out code but I fixed it. > http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/runtime/vframe.cpp.frames.html > > > 519 HandleMark hm; > > It seems to be redundant now. > > http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/services/classLoadingService.hpp.frames.html > > > 124 // _current_thread is for creating a Klass* with a faster version > constructor > 125 static Thread* _current_thread; > > Does the _current_thread become redundant now? Yes, it does! Good find. I removed this and am rerunning monitoring tests. > > http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/services/heapDumper.cpp.frames.html > It seems, the HandleMark's at 816, 849, 879, 894 are redundant. These might have Handles for oops in them. TBD cleanup. Thank you for the code review, Coleen > Thanks, Serguei On 3/13/17 13:04, coleen.phillimore at oracle.com wrote: >> On 3/13/17 3:42 PM, serguei.spitsyn at oracle.com wrote: >>> On 3/13/17 12:34, coleen.phillimore at oracle.com wrote: >>>> On 3/13/17 2:31 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Coleen, It looks good. Nice cleanup! >>>>> http://oklahoma.us.oracle.com/~cphillim/webrev/8155627.src.closed.02/webrev/share/vm/jfr/jfrClassAdapter.cpp.frames.html >>>>> The HandleMark's at 1743 & 1784 can be not needed anymore. >>>> Thank you for reviewing! I suspect many HandleMarks lying around >>>> the code aren't needed anymore, because KlassHandles (and even >>>> methodHandles) aren't cleaned up using HandleMark. I've removed >>>> those two (and ran some retests because this is getting dangerous). >>>> Are you going to review the open too? >>> Yes, but now I'm waiting for webrev version 03. This link is not >>> resolved for me: http://cr.openjdk.java.net/~coleenp/8155672.03/webrev >> It's there now. I transposed the digits in the last webrev. Thanks >> for letting me know. Coleen >>> Thanks, Serguei >>>> thank you! Coleen >>>>> Thanks, Serguei On 3/12/17 21:18, coleen.phillimore at oracle.com wrote: >>>>>> As requested by David on the closed part of this review, I fixed >>>>>> variable names of the form h_ik and klass_h where the 'h' >>>>>> indicated that the klass was in a handle. open webrev at >>>>>> http://cr.openjdk.java.net/~coleenp/8155672.01/webrev Also, >>>>>> fixing InstanceKlass to not pass this_k can be done in a separate >>>>>> cleanup. Thanks, Coleen On 3/12/17 11:32 AM, >>>>>> coleen.phillimore at oracle.com wrote: >>>>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, >>>>>>> remove handles with no implementation. These are fairly >>>>>>> extensive changes. The KlassHandle types have been dummy types >>>>>>> since permgen elimination and were thought to be useful for >>>>>>> future features. They aren't, so can be removed (see bug for >>>>>>> more details). This allows stricter typing because you can use >>>>>>> the more specific type, and using const more. I didn't add >>>>>>> const to these changes, because of the cascading effect. The >>>>>>> change isn't hard to review, just tedious. The main bug that I >>>>>>> had was redeclaring a type inside a scope, and >>>>>>> InstanceKlass::cast(k) can't take a NULL k, whereas >>>>>>> instanceKlassHandle ik(THREAD, k) could. It's so nice being able >>>>>>> to single step on gdb without going into KlassHandle >>>>>>> constructors! open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/8155672.01/webrev bug link >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8155672 Tested with all >>>>>>> hotspot jtreg tests, java/lang/invoke tests, >>>>>>> java/lang/instrument tests, all closed tonga colocated tests, >>>>>>> and JPRT. I can continue to hold this change for a convenient >>>>>>> time for merging purposes with other people's JDK10 changes that >>>>>>> they've been holding, or if there are other jdk9 changes that >>>>>>> are likely to cause a problem for merging. I'll update the >>>>>>> copyrights to 2017 on commit. Thanks, Coleen From serguei.spitsyn at oracle.com Tue Mar 14 00:28:05 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 13 Mar 2017 17:28:05 -0700 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: <10583606-0889-b4b6-dcce-4f3e93d44660@oracle.com> References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <10583606-0889-b4b6-dcce-4f3e93d44660@oracle.com> Message-ID: Hi Coleen, On 3/13/17 17:23, coleen.phillimore at oracle.com wrote: > > Hi Serguei, > Thank you for getting through this big change. Amazing work! Thank you for making an update. Thanks, Serguei > > On 3/13/17 8:10 PM, serguei.spitsyn at oracle.com wrote: >> Coleen, >> >> It looks good to me. >> Below are some minor comments. >> I'm Ok to separate the HandleMark cleanup if you prefer it. >> > > Yes, I would like to clean up the HandleMarks with a different > change. They're somewhat complicated because they used to clean up > Handles for oops and Handles for metadata. Now they only apply to > oops. But some of the places below do accumulate oops so they might > be needed there. > > I think Handle should be a scoped object with a destructor, etc so > that we don't need HandleMarks, but I haven't filed an RFE for that yet. >> >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/oops/instanceKlass.cpp.frames.html >> >> >> 2581 if ((name() == inner_name)) { >> >> Unneeded extra brackets (existed before the fix). >> >> > fixed. > >> >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/oops/klassVtable.cpp.frames.html >> >> >> 1080 HandleMark hm(THREAD); >> >> Possibly a redundant HandleMark. >> >> >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/prims/jvmtiImpl.cpp.frames.html >> >> >> 695 HandleMark hm(cur_thread); >> >> Possibly a redundant HandleMark. >> >> >> >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/prims/jvmtiRedefineClasses.cpp.frames.html >> >> >> 3853 InstanceKlass *ik = (InstanceKlass *)the_class; 4049 >> increment_class_counter((InstanceKlass *)the_class, THREAD); >> >> No need to cast to (InstanceKlass *) as the_class isInstanceKlass* . > > This was in commented out code but I fixed it. > >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/runtime/vframe.cpp.frames.html >> >> >> 519 HandleMark hm; >> >> It seems to be redundant now. >> >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/services/classLoadingService.hpp.frames.html >> >> >> 124 // _current_thread is for creating a Klass* with a faster version >> constructor >> 125 static Thread* _current_thread; >> >> Does the _current_thread become redundant now? > > Yes, it does! Good find. I removed this and am rerunning monitoring > tests. >> >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/services/heapDumper.cpp.frames.html >> It seems, the HandleMark's at 816, 849, 879, 894 are redundant. > > These might have Handles for oops in them. TBD cleanup. > > Thank you for the code review, > Coleen > >> Thanks, Serguei On 3/13/17 13:04, coleen.phillimore at oracle.com wrote: >>> On 3/13/17 3:42 PM, serguei.spitsyn at oracle.com wrote: >>>> On 3/13/17 12:34, coleen.phillimore at oracle.com wrote: >>>>> On 3/13/17 2:31 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Coleen, It looks good. Nice cleanup! >>>>>> http://oklahoma.us.oracle.com/~cphillim/webrev/8155627.src.closed.02/webrev/share/vm/jfr/jfrClassAdapter.cpp.frames.html >>>>>> The HandleMark's at 1743 & 1784 can be not needed anymore. >>>>> Thank you for reviewing! I suspect many HandleMarks lying around >>>>> the code aren't needed anymore, because KlassHandles (and even >>>>> methodHandles) aren't cleaned up using HandleMark. I've removed >>>>> those two (and ran some retests because this is getting >>>>> dangerous). Are you going to review the open too? >>>> Yes, but now I'm waiting for webrev version 03. This link is not >>>> resolved for me: http://cr.openjdk.java.net/~coleenp/8155672.03/webrev >>> It's there now. I transposed the digits in the last webrev. Thanks >>> for letting me know. Coleen >>>> Thanks, Serguei >>>>> thank you! Coleen >>>>>> Thanks, Serguei On 3/12/17 21:18, coleen.phillimore at oracle.com >>>>>> wrote: >>>>>>> As requested by David on the closed part of this review, I fixed >>>>>>> variable names of the form h_ik and klass_h where the 'h' >>>>>>> indicated that the klass was in a handle. open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/8155672.01/webrev Also, >>>>>>> fixing InstanceKlass to not pass this_k can be done in a >>>>>>> separate cleanup. Thanks, Coleen On 3/12/17 11:32 AM, >>>>>>> coleen.phillimore at oracle.com wrote: >>>>>>>> Summary: Use unhandled pointers for Klass and InstanceKlass, >>>>>>>> remove handles with no implementation. These are fairly >>>>>>>> extensive changes. The KlassHandle types have been dummy types >>>>>>>> since permgen elimination and were thought to be useful for >>>>>>>> future features. They aren't, so can be removed (see bug for >>>>>>>> more details). This allows stricter typing because you can >>>>>>>> use the more specific type, and using const more. I didn't add >>>>>>>> const to these changes, because of the cascading effect. The >>>>>>>> change isn't hard to review, just tedious. The main bug that I >>>>>>>> had was redeclaring a type inside a scope, and >>>>>>>> InstanceKlass::cast(k) can't take a NULL k, whereas >>>>>>>> instanceKlassHandle ik(THREAD, k) could. It's so nice being >>>>>>>> able to single step on gdb without going into KlassHandle >>>>>>>> constructors! open webrev at >>>>>>>> http://cr.openjdk.java.net/~coleenp/8155672.01/webrev bug link >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8155672 Tested with >>>>>>>> all hotspot jtreg tests, java/lang/invoke tests, >>>>>>>> java/lang/instrument tests, all closed tonga colocated tests, >>>>>>>> and JPRT. I can continue to hold this change for a convenient >>>>>>>> time for merging purposes with other people's JDK10 changes >>>>>>>> that they've been holding, or if there are other jdk9 changes >>>>>>>> that are likely to cause a problem for merging. I'll update the >>>>>>>> copyrights to 2017 on commit. Thanks, Coleen > From david.holmes at oracle.com Tue Mar 14 00:44:33 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 14 Mar 2017 10:44:33 +1000 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: <10583606-0889-b4b6-dcce-4f3e93d44660@oracle.com> References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <10583606-0889-b4b6-dcce-4f3e93d44660@oracle.com> Message-ID: <8a96bd92-6620-0b01-00b0-baeba545f39a@oracle.com> Something caught my eye ... On 14/03/2017 10:23 AM, coleen.phillimore at oracle.com wrote: > On 3/13/17 8:10 PM, serguei.spitsyn at oracle.com wrote: >> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/services/classLoadingService.hpp.frames.html >> >> >> 124 // _current_thread is for creating a Klass* with a faster version >> constructor >> 125 static Thread* _current_thread; >> >> Does the _current_thread become redundant now? > > Yes, it does! Good find. I removed this and am rerunning monitoring > tests. Current thread stored in a static! That was a bug! David ----- From serguei.spitsyn at oracle.com Tue Mar 14 00:50:45 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 13 Mar 2017 17:50:45 -0700 Subject: RFR (XL) 8155672: Remove instanceKlassHandles and KlassHandles In-Reply-To: <8a96bd92-6620-0b01-00b0-baeba545f39a@oracle.com> References: <47373b97-b6a3-b133-b415-ca413e00928f@oracle.com> <10583606-0889-b4b6-dcce-4f3e93d44660@oracle.com> <8a96bd92-6620-0b01-00b0-baeba545f39a@oracle.com> Message-ID: On 3/13/17 17:44, David Holmes wrote: > Something caught my eye ... > > > > On 14/03/2017 10:23 AM, coleen.phillimore at oracle.com wrote: >> On 3/13/17 8:10 PM, serguei.spitsyn at oracle.com wrote: >>> http://cr.openjdk.java.net/~coleenp/8155672.03/webrev/src/share/vm/services/classLoadingService.hpp.frames.html >>> >>> >>> >>> 124 // _current_thread is for creating a Klass* with a faster version >>> constructor >>> 125 static Thread* _current_thread; >>> >>> Does the _current_thread become redundant now? >> >> Yes, it does! Good find. I removed this and am rerunning monitoring >> tests. > > Current thread stored in a static! That was a bug! Right. Good catch! Thanks, Serguei > > David > ----- From zhuyong.me at gmail.com Tue Mar 14 02:10:18 2017 From: zhuyong.me at gmail.com (Zhu Yong) Date: Tue, 14 Mar 2017 10:10:18 +0800 Subject: Fwd: cross compile OpenJDK-9+158 minimal variant failed when link libjvm.so In-Reply-To: References: <30b93f7d-c442-82ab-75b4-61ab10c3e3eb@oracle.com> <6cb3f741-f665-30ce-f5fe-e3127f43f55b@oracle.com> <82ce925e-64e0-fff5-0e35-c681a8b9d00d@oracle.com> <1fa9ab73-b3dd-9fa3-8d36-0e90c7e33127@oracle.com> <976215a3-ca51-dc8b-9f78-658707c54d2e@oracle.com> <51e53933-2860-49e5-7ae6-53f069576cb2@oracle.com> <1d2586eb-0257-a330-2ff6-5028de510b4b@oracle.com> <289fc22c-3faa-26f8-5cce-affd3b184998@oracle.com> Message-ID: I confirmed the floating point issue is due to the solution I applied for https://bugs.openjdk.java.net/browse/JDK-8175567 If I use the initial version of solution, can see the floating point issue, if use the final solution won't see. Thank you very much for helping. On Thu, Mar 9, 2017 at 10:17 AM, Zhu Yong wrote: > Yes, it's easy to test. > > I am doing benchmark (SciMark and SPECjvm2008) on target system with the > newly built server, client and minimal VM. > > After done the benchmarks, I will build a test code and update again. > > On Thu, Mar 9, 2017 at 9:49 AM, David Holmes > wrote: > >> On 9/03/2017 11:25 AM, Zhu Yong wrote: >> >>> Here is my latest update with good news >>> >>> Yesterday pulled tips code from jdk9-dev and built with my current >>> toolchain (no change), this time client and minimal VM finished the test >>> code without any issue. >>> >>> The issue might relate to >>> https://bugs.openjdk.java.net/browse/JDK-8175567, the final solution >>> in http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/d6a0fe7ae0e8 is >>> different from the very initial proposed solution (which I used in my >>> previous build). >>> >> >> Easy enough to test that theory. :) >> >> Glad things are now working. >> >> David >> ----- >> >> On Tue, Mar 7, 2017 at 5:54 AM, Chris Plummer >> > wrote: >>> >>> I haven't been following this too closely, but one thing to consider >>> is that you are not getting proper floating point calling >>> conventions. You mentioned your configure target ABI is >>> arm-vfp-sflt. This means you have VFP support, but use the old >>> softfloat calling conventions (FP values passed in GPRs). Most >>> platforms these days with VFP support use the new hardfloat calling >>> conventions (FP values passed in FPRs). So this could simply be the >>> case of building the wrong binary for your platform. >>> >>> - Use arm-vfp-hflt is you have VFP support and want to use hard >>> float calling >>> conventions. >>> - Use arm-vfp-sflt if you have VFP support and want to use the >>> older soft float >>> calling conventions. >>> - Use arm-sflt is you have a true soft float platform. >>> >>> There's also a very old arm sflt ABI that I don't think we support. >>> The main difference from arm-sflt is the word endianess of of 64-bit >>> values. I believe the presence of -mfpu=vfp is what is used to get >>> the more modern endianess, which to say the least is very >>> non-obvious option for doing this, but is the reason you will see >>> -mfpu=vfp for all our supported arm platforms (arm-vfp-hflt, >>> arm-vfp-sflt, and arm-sflt). Here's the logic from the configure >>> script: >>> >>> if test "x$ARM_FLOAT_TYPE" = xvfp-sflt; then >>> ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=softfp -mfpu=vfp >>> -DFLOAT_ARCH=-vfp-sflt' >>> elif test "x$ARM_FLOAT_TYPE" = xvfp-hflt; then >>> ARM_FLOAT_TYPE_FLAGS='-mfloat-abi=hard -mfpu=vfp >>> -DFLOAT_ARCH=-vfp-hflt' >>> elif test "x$ARM_FLOAT_TYPE" = xsflt; then >>> ARM_FLOAT_TYPE_FLAGS='-msoft-float -mfpu=vfp' >>> fi >>> >>> BTW, for the arm-sflt case, FLOAT_ARCH gets set internally to >>> "-sflt". I'm not sure why for arm-sflt it's handled in the source >>> and the other two platforms set it in the configure script. >>> >>> cheers, >>> >>> Chris >>> >>> >>> On 3/5/17 8:30 PM, Zhu Yong wrote: >>> >>> Sometime later this week, I might get newer version of Marvell >>> toolchain, I >>> will do a build and test again then. >>> I will update this thread when I get result. >>> >>> Thank you very much for helping. >>> >>> On Mon, Mar 6, 2017 at 9:09 AM, David Holmes >>> > >>> wrote: >>> >>> On 3/03/2017 6:59 PM, Zhu Yong wrote: >>> >>> Thank you very much for helping. >>> >>> Today I tried to debug the test code, but without luck, >>> still can't >>> figure out why. >>> >>> First, I followed the instruction >>> http://stackoverflow.com/quest >>> ions/29916995/intellij-idea- >>> >> tions/29916995/intellij-idea-> >>> remotely-debug-java-console-program, >>> changed suspend=y because the test code is short, it >>> will exit before >>> debug start if not suspend first. Unfortunately, with >>> remote debug, I >>> was not able to reproduce the issue. Same class file, >>> directly run, will >>> produce the issue, remote debug won't. >>> >>> Then, I tried *jdb *from my OpenJDK9 client vm, run >>> directly on my >>> target system to debug the code. If I set breakpoint, >>> program run >>> without issue, however, if don't set breakpoint, just >>> "run" in jdb, >>> problem reproduced. >>> >>> This suggests to me that a floating-point register is not >>> being preserved >>> when needed. When you run with the breakpoint you will get >>> full >>> save/restore of all the register state. Just a theory. >>> >>> Next, I tried to use *javap *to decompile the output class >>> file, run >>> >>> OpenJDK9 client vm javap and host side javap on same >>> class file, the >>> output is same for getTime function. >>> >>> It is a native code issue not a bytecode issue, so javap >>> wont help here. >>> >>> One interesting point I noticed today is the output like >>> below when >>> >>> problem happen, 2nd call of getTime() has value of 1st >>> call result / >>> 1000. Looks like 2nd call getTime(), double variable q >>> was not updated >>> startTime: 284092.063000 endTime: 284.092063 runTime: >>> -283807.970937 >>> >>> What should I do so that I can come a conclusion that >>> it's due to >>> compiler issue? >>> >>> Can you get access to a different gcc to build with? >>> >>> David >>> ----- >>> >>> >>> On Thu, Mar 2, 2017 at 12:32 PM, David Holmes >>> >> > >>> >> >> wrote: >>> >>> I built your sample program with latest JDK 9 for >>> arm-vfp-sflt and >>> ran on an emulator I have available, and I had no >>> issues. >>> >>> So I still have concerns this may be a >>> build/compiler issue. >>> >>> David >>> >>> >>> On 2/03/2017 12:51 PM, David Holmes wrote: >>> >>> I've moved this discussion over to hotspot-dev >>> as this seems no >>> longer a >>> build issue but a C1 soft-float issue. >>> >>> David >>> >>> On 2/03/2017 12:35 PM, Zhu Yong wrote: >>> >>> If run with -Xint, it works. >>> >>> I have created simplified FP test by remove >>> all non-related >>> code from >>> Whetstone test code. >>> The test code is here: >>> >>> https://gist.github.com/yongz >>> hy/d65c26d39fe5d549d1b406c7c84e >>> >> hy/d65c26d39fe5d549d1b406c7c84e> >>> acd4 >>> >>> >> zhy/d65c26d39fe5d549d1b406c7c84 >>> >> hy/d65c26d39fe5d549d1b406c7c84> >>> eacd4> >>> >>> I am not sure if the test code can help or >>> not. The >>> behaviour is weird : >>> - If I print both itime and q, program will >>> run OK >>> - inside condition block if(q<1.0f), if I >>> use exit code >>> 2,3,4,5, problem >>> appears, however, if I use exit code >=6, >>> program run OK. >>> >>> I always get exit code 5, the line q = >>> (double)itime is wrong? >>> >>> public static double getTime() >>> { >>> double q; >>> long itime; >>> itime = >>> System.currentTimeMillis(); >>> if(itime<1000000) { >>> System.exit(1); >>> } >>> //System.out.printf("time long >>> value %d\n", >>> itime); >>> q = (double) itime; >>> if(q < 1.0f) { >>> System.exit(5); // I >>> always get exit code 5 >>> } >>> //System.out.printf("time float >>> value %f\n", q); >>> return q / 1000.0; >>> } >>> >>> >>> >>> On Wed, Mar 1, 2017 at 5:31 PM, David Holmes >>> >> >>> >> > >>> >> >>> >> >>> wrote: >>> >>> On 1/03/2017 6:45 PM, Zhu Yong wrote: >>> >>> OK, for the Whetstone Benchmark, I >>> added debug >>> printing and >>> found it's >>> due to function getTime(), it only >>> get good value on >>> 1st call, >>> all >>> following calls get 0s. >>> Debug printing of itime value is >>> correct. So reason >>> should be >>> the return >>> division line. Could it because >>> toolchain's floating >>> point >>> operation??? >>> But why server VM Ok? >>> >>> >>> Server and client implement FP logic >>> differently, and >>> particularly >>> as this is soft-fp, they may well not >>> be in sync. Does >>> -Xint work? >>> >>> Can you isolate to a simple FP test? >>> >>> David >>> >>> public static double getTime() >>> { >>> double q; >>> long itime; >>> itime = >>> System.currentTimeMillis(); >>> q = (double) itime; >>> return q / 1000.0; >>> } >>> >>> On Wed, Mar 1, 2017 at 3:14 PM, >>> David Holmes >>> >> >>> >> > >>> >> >>> >> >> >>> >> >>> >> > >>> >> >>> >> >>>> wrote: >>> >>> On 1/03/2017 4:26 PM, Zhu Yong >>> wrote: >>> >>> We use >>> armv7-marvell-linux-gnueabi-softfp >>> toolchain >>> >>> gcc version 4.6.4 (Marvell >>> GCC release >>> 20150204-c4af733b 64K >>> MAXPAGESIZE >>> ALIGN CVE-2015-0235) >>> >>> Is this toolchain too old? >>> I am asking >>> because I saw >>> other strange >>> issues as well: >>> >>> >>> We last used gcc 4.7.2. I can't >>> really say if >>> 4.6.4 is "too >>> old". >>> >>> I have successfully build >>> server, client and >>> minimal VM, >>> when I run >>> Dhrystone benchmark file >>> (the same jar file >>> from >>> >>> http://www.okayan.jp/DhrystoneApplet/ >>> >>> >> > >>> >>> >> >>> >> >> >>> >>> >> >>> >> > >>> >>> >> >>> >> >>>), the >>> performance >>> from >>> server VM >>> is much better than client >>> and minimal VM. >>> (minimal: 1629852, client: >>> 1615508, server: >>> 2338871 ) >>> >>> >>> That's expected. The server VM >>> is high >>> performance. >>> >>> And when run Whetstone >>> Benchmark >>> from >>> >>> >>> http://www.roylongbottom.org.uk/online/whetjava2.html >>> >>> >>> >> > >>> >>> >>> >> >>> >>> >> >> >>> >>> >>> >>> >> >>> >>> >> > >>> >>> >>> >> >>> >>> >> >> >>>>, >>> server VM >>> finished with good result, >>> client and >>> minimal VM not >>> able to finish >>> (program stuck there >>> forever after print 1st >>> line of >>> output). >>> >>> >>> That needs investigating. You >>> need to try and >>> generate a >>> stackdump >>> to see where things are stuck. >>> >>> David >>> >>> >>> On Wed, Mar 1, 2017 at 1:56 >>> PM, David Holmes >>> >> >>> >> > >>> >> >>> >> >> >>> >> >> david.holmes at oracle.com >> >> >>> >> >>> >> >>> >>> >>> >> >>> >> > >>> >> >>> >> >> >>> >>> >> >>> >> > >>> >> >>> >> >>>>> wrote: >>> >>> On 1/03/2017 3:39 PM, >>> Zhu Yong wrote: >>> >>> Thank you for the >>> information, I >>> managed to make >>> minimal >>> build >>> pass now. >>> >>> Initially I though >>> JVM_EXCLUDE_FILES >>> need to add >>> additional 3 >>> generated >>> files (they appeared >>> in >>> _BUILD_LIBJVM_objectfilenames.txt) >>> : >>> >>> bytecodeInterpreterWithChecks.cpp >>> jvmtiEnter.cpp >>> jvmtiEnterTrace.cpp >>> But build still >>> fail with same error >>> message. >>> >>> Finally I figured >>> out it's due to >>> those 8 doit() >>> functions >>> implementation in >>> jvmtiEnvBase.hpp >>> file. After >>> move all >>> those 8 >>> doit() >>> implementations to >>> jvmtiEnvBase.cpp, >>> build of >>> minimal VM >>> passed >>> without >>> any issue. >>> >>> >>> That seems to be a >>> compiler issue. >>> jvmtiEnvBase.hpp is >>> unconditionally >>> included in a couple of >>> places >>> because we >>> have to >>> have enough of the >>> JVMTI code to say >>> JVMTI is not >>> available. >>> Those >>> doit() implementations, >>> though arguably >>> could be >>> conditional on >>> INCLUDE_JVMTI, are not >>> referenced by any >>> called code >>> and so >>> should >>> not linked in. But in >>> your case it seems >>> they are. >>> >>> What toolchain are you >>> using? >>> >>> David >>> ----- >>> >>> Changes appended >>> ============= >>> >>> --- >>> >>> .../src/share/vm/prims/jvmtiE >>> nvBase.cpp >>> | 74 >>> >>> ++++++++++++++++++++++ >>> >>> .../src/share/vm/prims/jvmtiE >>> nvBase.hpp >>> | 68 >>> +++----------------- >>> 2 files changed, >>> 82 insertions(+), 60 >>> deletions(-) >>> mode change 100644 >>> => 100755 >>> >>> hotspot-9211c2e89c1c/src/share >>> /vm/prims/jvmtiEnvBase.cpp >>> mode change 100644 >>> => 100755 >>> >>> hotspot-9211c2e89c1c/src/share >>> /vm/prims/jvmtiEnvBase.hpp >>> >>> diff --git >>> >>> >>> >>> a/hotspot-9211c2e89c1c/src/sh >>> are/vm/prims/jvmtiEnvBase.cpp >>> >>> >>> >>> b/hotspot-9211c2e89c1c/src/sh >>> are/vm/prims/jvmtiEnvBase.cpp >>> old mode 100644 >>> new mode 100755 >>> index >>> dd241a0..e5832ba >>> --- >>> >>> >>> a/hotspot-9211c2e89c1c/src/sh >>> are/vm/prims/jvmtiEnvBase.cpp >>> +++ >>> >>> >>> b/hotspot-9211c2e89c1c/src/sh >>> are/vm/prims/jvmtiEnvBase.cpp >>> @@ -1283,6 +1283,80 >>> @@ >>> >>> VM_GetMultipleStackTraces::all >>> ocate_and_fill_stacks(jint >>> thread_count) { >>> "the last >>> copied frame >>> info must be >>> the last >>> record"); >>> } >>> >>> +void >>> >>> +VM_UpdateForPopTopFrame::doit() { >>> + JavaThread* jt = >>> _state->get_thread(); >>> + if >>> (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + >>> _state->update_for_pop_top_fra >>> me(); >>> + } else { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + } >>> +} >>> + >>> +void >>> >>> +VM_SetFramePop::doit() { >>> + JavaThread* jt = >>> _state->get_thread(); >>> + if >>> (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + int >>> frame_number = >>> _state->count_frames() - >>> _depth; >>> + >>> >>> >>> >>> >>> _state->env_thread_state((Jvm >>> tiEnvBase*)_env)->set_frame_ >>> pop(frame_number); >>> >>> + } else { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + } >>> +} >>> + >>> +void >>> >>> +VM_GetOwnedMonitorInfo::doit() { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if >>> (Threads::includes(_java_thread) && >>> >>> !_java_thread->is_exiting() >>> + >>> && >>> >>> _java_thread->threadObj() != >>> NULL) { >>> + _result = >>> ((JvmtiEnvBase >>> >>> *)_env)->get_owned_monitors(_ >>> calling_thread, >>> _java_thread, >>> + >>> >>> _owned_monitors_list); >>> + } >>> +} >>> + >>> +void >>> >>> +VM_GetObjectMonitorUsage::doit() { >>> + _result = >>> ((JvmtiEnvBase*) >>> >>> >>> _env)->get_object_monitor_usage(_calling_thread, >>> _object, >>> _info_ptr); >>> +} >>> + >>> +void >>> >>> +VM_GetCurrentContendedMonitor::doit() >>> { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if >>> (Threads::includes(_java_thread) && >>> >>> !_java_thread->is_exiting() && >>> + >>> _java_thread->threadObj() != >>> NULL) { >>> + _result = >>> ((JvmtiEnvBase >>> >>> >>> >>> >>> *)_env)->get_current_contende >>> d_monitor(_calling_thread,_ >>> java_thread,_owned_monitor_ptr); >>> >>> + } >>> +} >>> + >>> +void >>> >>> +VM_GetStackTrace::doit() { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if >>> (Threads::includes(_java_thread) && >>> >>> !_java_thread->is_exiting() >>> + >>> && >>> >>> _java_thread->threadObj() != >>> NULL) { >>> + _result = >>> ((JvmtiEnvBase >>> >>> *)_env)->get_stack_trace(_java_thread, >>> + >>> _start_depth, >>> _max_count, >>> + >>> _frame_buffer, >>> _count_ptr); >>> + } >>> +} >>> + >>> +void >>> >>> +VM_GetFrameCount::doit() { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + JavaThread* jt = >>> _state->get_thread(); >>> + if >>> (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() != >>> NULL) { >>> + _result = >>> >>> ((JvmtiEnvBase*)_env)->get_fra >>> me_count(_state, >>> _count_ptr); >>> + } >>> +} >>> + >>> +void >>> >>> +VM_GetFrameLocation::doit() { >>> + _result = >>> JVMTI_ERROR_THREAD_NOT_ALIVE; >>> + if >>> (Threads::includes(_java_thread) && >>> >>> !_java_thread->is_exiting() && >>> + >>> _java_thread->threadObj() != >>> NULL) { >>> + _result = >>> >>> ((JvmtiEnvBase*)_env)->get_fra >>> me_location(_java_thread, >>> _depth, >>> + >>> _method_ptr, >>> _location_ptr); >>> + } >>> +} >>> >>> void >>> >>> VM_GetThreadListStackTraces::doit() >>> { >>> diff --git >>> >>> >>> >>> a/hotspot-9211c2e89c1c/src/sh >>> are/vm/prims/jvmtiEnvBase.hpp >>> >>> >>> >>> b/hotspot-9211c2e89c1c/src/sh >>> are/vm/prims/jvmtiEnvBase.hpp >>> old mode 100644 >>> new mode 100755 >>> index >>> 04e6869..00b9890 >>> --- >>> >>> >>> a/hotspot-9211c2e89c1c/src/sh >>> are/vm/prims/jvmtiEnvBase.hpp >>> +++ >>> >>> >>> b/hotspot-9211c2e89c1c/src/sh >>> are/vm/prims/jvmtiEnvBase.hpp >>> @@ -359,14 +359,7 >>> @@ public: >>> } >>> VMOp_Type type() >>> const { return >>> VMOp_UpdateForPopTopFrame; } >>> jvmtiError >>> result() { return >>> _result; } >>> - void doit() { >>> - JavaThread* jt >>> = >>> _state->get_thread(); >>> - if >>> (Threads::includes(jt) && >>> !jt->is_exiting() && >>> jt->threadObj() >>> != NULL) { >>> - >>> _state->update_for_pop_top_frame(); >>> - } else { >>> - _result = >> >> > From thomas.stuefe at gmail.com Tue Mar 14 08:34:04 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 14 Mar 2017 09:34:04 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> References: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> Message-ID: Hi Coleen, thank you for reviewing! I'll do the proposed change. Kind Regards, Thomas On Mon, Mar 13, 2017 at 8:54 PM, wrote: > > Hi Thomas, Thank you for cleaning up the metaspace code. I think the > change looks good. I have only one comment and that is: can you put > ChunkManager::size_by_index() function definition up in the file with the > other ChunkManager functions? > > If Mikael reviews the test, I'd be happy to sponsor this for you. Sorry > about the inattention. > > thanks! > Coleen > > > On 3/11/17 4:08 AM, Thomas St?fe wrote: > >> Ping... Did I send this to the wrong mailing list? >> >> On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe >> wrote: >> >> (Resent to hotspot-dev) >>> >>> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >>> wrote: >>> >>> Hi all, >>>> >>>> I would like to get a review for this cleanup change for the >>>> metaspace.cpp. This has been a long time brewing in my backlog, but I >>>> had >>>> to wait until jdk10 is open. The cleanup is actually quite small, the >>>> largest part of the change is the added test coding. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>>> >>>> The change implements a suggestion made by Mikael Gerdin when >>>> reviewing JDK-8170520 last year, who suggested that the ChunkManager >>>> class >>>> should handle returns of both non-humongous and humongous chunks, see >>>> original discussion here: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>>> 016-November/021949.html >>>> >>>> "It would be great if you could have a look at unifying the >>>> ChunkManager::return_chunks API for standard and humongous chunks so >>>> that we wouldn't need the special casing for this in ~SpaceManager >>>> That way we could also make humongous_dictionary() private to >>>> ChunkManager which would improve encapsulation." >>>> >>>> The cleanup does this and a bit more, all changes having to do with the >>>> fact that the ChunkManager class unnecessarily exposes internals to the >>>> other classes in metaspace.cpp and that with a little bit of >>>> reorganization >>>> this can be avoided. The benefit is simpler coding and way better >>>> encapsulation, which will help a lot with future changes (in preparation >>>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>>> >>>> -------- >>>> >>>> The changes in detail: >>>> >>>> The main issue was that ChunkManager::return_chunks() did not handle >>>> humongous chunks. To return humongous chunks, one had to access the >>>> humongous chunk dictionary inside the ChunkManager and add the chunk >>>> yourself, including maintaining all counters. This happened in >>>> ~SpaceManager and made this destructor very complicated. >>>> >>>> This is solved by moving the handling of humongous chunk returns to the >>>> ChunkManager class itself. For convenience, I split the old >>>> "ChunkManager::return_chunks" method into two new ones, >>>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>>> the ChunkManager without walking the chunk chain, and >>>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>>> the ChunkManager. Both functions are now able to handle both >>>> non-humongous >>>> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >>>> simplified) and just calls "ChunkManager::return_chunk_list()" for all >>>> its chunk lists. >>>> >>>> So now we can remove the public access to the humongous chunk dictionary >>>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>>> function private. >>>> >>>> ---- >>>> >>>> Then there was the fact that the non-humongous chunk lists were also >>>> exposed to public via ChunkManager::free_chunks(). The only reason was >>>> that >>>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free >>>> lists >>>> to find out the size of the items of the free lists. >>>> >>>> This was solved by adding a new function, ChunkManager::size_by_index(), >>>> which returns the size of the items of a chunk list given by its chunk >>>> index. >>>> >>>> So ChunkManager::free_chunks() could be made private too. >>>> >>>> --- >>>> >>>> The rest are minor changes: >>>> >>>> - made ChunkManager::find_free_chunks_list() private because it was not >>>> used from outside the class >>>> - Fixed an assert in dec_free_chunks_total() >>>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >>>> to keep the existing logging intact or add useful logging where >>>> possible. >>>> >>>> ---- >>>> >>>> A large chunk of the changes is the added gtests. There is a new class >>>> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >>>> code. >>>> >>>> Note that I dislike the fact that this test is implemented inside >>>> metaspace.cpp itself. For now I kept my new tests like the existing >>>> tests >>>> inside this file, but I think these tests should be moved to >>>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>>> separate fix though, because it needs a bit of remodeling (the tests >>>> need >>>> to access internals in metaspace.cpp). >>>> >>>> ---- >>>> >>>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >>>> AIX. No issues popped up which were associated with my change. >>>> >>>> Thanks for reviewing and Kind Regards, Thomas >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> > From uschindler at apache.org Tue Mar 14 09:25:44 2017 From: uschindler at apache.org (Uwe Schindler) Date: Tue, 14 Mar 2017 10:25:44 +0100 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: References: <010401d299ca$c198b620$44ca2260$@apache.org> <011201d29be5$8250dfc0$86f29f40$@apache.org> Message-ID: <028c01d29ca4$f7c07300$e7415900$@apache.org> Hallo Mike, thank you for the update! The comparison with NIOFSDirectory is helpful, as we can now conclude: * There is a minor slowdown implied by Java 9 in comparison to Java 8, which can have to do with other changed due to Jigsaw module system, and is not related to the ByteBuffer issue. As NIOFSDirectory does not really use ByteBuffers in a performance critical way (it is just used to read buffers of 8 or 16 KiB size from FileChannel into a HeapByteBuffer wrapped around a byte[] that is used for BufferedIndexInput), Andrew Haley?s investigation of byte buffer slowdowns cannot have any impact here. The ByteBuffer API is never used to access the byte[] contents, ByteBuffer is just used to pass the Lucene byte[] between FileChannel and our code! * NIOFSDirectory is still much slower for accessing the Lucene index than mmap (as expected, especially for sorting where positional reads are really needed), so the current Lucene default of using MMapDirectory of 64 bit platforms still outweights any other Lucene directory implementation. * As there is a larger slowdown if Lucene uses the MMapDirectory implementation on Java 9 in comparison to Java 8, we should really look into the issues! So Andrew Haley is right, there is some slowdown not only affecting HeapByteBuffers! Maybe it is caused by missing vectorization or because the code calling the ByteBuffers is much more complicated than Andrew?s tests, Hotspot is failing to properly optimize the access to DirectByteBuffer, too. Andrew & others, if you have a patch ready, we can try to benchmark it! Uwe From: Michael McCandless [mailto:lucene at mikemccandless.com] Sent: Monday, March 13, 2017 4:44 PM To: Uwe Schindler Cc: Andrew Haley ; Lucene/Solr dev ; jdk9-dev at openjdk.java.net; hotspot-dev at openjdk.java.net Subject: Re: ByteBuffer performance issue in Java 9? I reran the same test as before, this time using Lucene's NIOFSDirectory (java.nio's FileChannel for positional reads). There is still some slowdown, though a bit less than with MMapDirectory: Task QPS base StdDev QPS comp StdDev Pct diff LowSloppyPhrase 9.12 (5.2%) 7.89 (3.6%) -13.5% ( -21% - -4%) HighSloppyPhrase 6.25 (4.1%) 5.57 (3.3%) -10.8% ( -17% - -3%) MedSloppyPhrase 4.29 (3.8%) 3.85 (3.0%) -10.1% ( -16% - -3%) BrowseDayOfYearSSDVFacets 4.72 (8.8%) 4.32 (5.4%) -8.4% ( -20% - 6%) BrowseMonthTaxoFacets 1.31 (2.9%) 1.20 (4.7%) -8.3% ( -15% - 0%) BrowseDayOfYearTaxoFacets 1.17 (4.0%) 1.08 (4.1%) -7.7% ( -15% - 0%) BrowseDateTaxoFacets 1.18 (4.0%) 1.09 (4.1%) -7.6% ( -15% - 0%) BrowseMonthSSDVFacets 5.39 (4.3%) 4.98 (10.7%) -7.6% ( -21% - 7%) HighTerm 29.47 (5.3%) 27.45 (4.2%) -6.9% ( -15% - 2%) HighTermDayOfYearSort 14.24 (4.6%) 13.35 (5.5%) -6.3% ( -15% - 3%) MedTerm 44.47 (4.0%) 42.02 (3.4%) -5.5% ( -12% - 1%) OrHighNotLow 33.13 (5.0%) 31.39 (4.3%) -5.2% ( -13% - 4%) OrHighLow 26.84 (4.2%) 25.49 (4.0%) -5.0% ( -12% - 3%) HighPhrase 7.51 (5.4%) 7.17 (4.0%) -4.5% ( -13% - 5%) Fuzzy2 51.32 (0.9%) 49.03 (1.2%) -4.5% ( -6% - -2%) OrHighNotHigh 13.18 (3.5%) 12.64 (3.7%) -4.2% ( -10% - 3%) IntNRQ 6.28 (6.3%) 6.03 (9.9%) -4.0% ( -18% - 12%) OrHighNotMed 27.69 (2.9%) 26.65 (3.3%) -3.8% ( -9% - 2%) Fuzzy1 42.32 (0.8%) 40.88 (1.1%) -3.4% ( -5% - -1%) MedSpanNear 27.44 (2.4%) 26.57 (2.8%) -3.2% ( -8% - 2%) OrNotHighHigh 17.58 (2.8%) 17.04 (3.3%) -3.1% ( -8% - 3%) HighSpanNear 26.83 (2.2%) 26.03 (2.5%) -3.0% ( -7% - 1%) Respell 49.07 (1.2%) 47.62 (0.8%) -3.0% ( -4% - 0%) LowPhrase 31.19 (1.5%) 30.34 (1.2%) -2.7% ( -5% - 0%) Wildcard 51.30 (4.8%) 49.93 (4.3%) -2.7% ( -11% - 6%) LowSpanNear 31.05 (1.2%) 30.40 (1.5%) -2.1% ( -4% - 0%) LowTerm 105.83 (1.1%) 103.78 (1.4%) -1.9% ( -4% - 0%) OrNotHighMed 49.08 (1.8%) 48.16 (2.1%) -1.9% ( -5% - 2%) OrHighMed 18.44 (5.8%) 18.09 (5.1%) -1.9% ( -12% - 9%) AndHighHigh 21.22 (1.0%) 20.82 (1.3%) -1.9% ( -4% - 0%) HighTermMonthSort 32.01 (3.9%) 31.43 (5.3%) -1.8% ( -10% - 7%) OrHighHigh 12.89 (7.0%) 12.66 (6.0%) -1.8% ( -13% - 12%) MedPhrase 24.09 (2.3%) 23.80 (2.2%) -1.2% ( -5% - 3%) AndHighLow 447.74 (1.5%) 443.85 (1.7%) -0.9% ( -3% - 2%) Prefix3 18.41 (6.8%) 18.28 (5.6%) -0.7% ( -12% - 12%) OrNotHighLow 254.77 (1.4%) 254.59 (1.2%) -0.1% ( -2% - 2%) AndHighMed 63.15 (1.5%) 63.47 (0.9%) 0.5% ( -1% - 3%) PKLookup 347.80 (2.3%) 349.93 (2.1%) 0.6% ( -3% - 5%) Mike McCandless http://blog.mikemccandless.com On Mon, Mar 13, 2017 at 10:16 AM, Michael McCandless < lucene at mikemccandless.com> wrote: Hi Uwe, OK, I'll test with NIOFSDirectory as well ... that's a good idea. I do remember testing earlier Java 9 builds long ago, but I can't remember what the outcome was. Mike McCandless http://blog.mikemccandless.com On Mon, Mar 13, 2017 at 6:35 AM, Uwe Schindler < uschindler at apache.org> wrote: Hi Andrew, yes that was my impression, too. Just for cross-checking: Mike, is it possible to also add a perf comparison between Java 8 and Java 9 when using SimpleFSDirectory or NIOFSDirectory (which are both FileChannel based since Java 7, the name is just backwards-compatibility)? If we see a slowdown there (maybe even larger) than it is not related to ByteBuffer positional/byte-wise reads and there is a general performance issue somewhere else. > Right, but ByteBuffers were significantly rewritten for a significant > performance *increase*. Any slowdown shows that something has gone > very wrong indeed. That would be a pity, because of that we should check the above case with non-mmap based, conventional index access. As far as I remember: at the time when you announced the bytebuffer improvements we did some performance measurements and were impressed by the speedup. I think Robert Muir did something. Mike, do you know? Maybe we should check with a Java 9 preview build from that time. I know that you can download older builds by changing the build number in the download URL. Uwe From aph at redhat.com Tue Mar 14 09:58:31 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 14 Mar 2017 09:58:31 +0000 Subject: ByteBuffer performance issue in Java 9? In-Reply-To: <028c01d29ca4$f7c07300$e7415900$@apache.org> References: <010401d299ca$c198b620$44ca2260$@apache.org> <011201d29be5$8250dfc0$86f29f40$@apache.org> <028c01d29ca4$f7c07300$e7415900$@apache.org> Message-ID: On 14/03/17 09:25, Uwe Schindler wrote: > * As there is a larger slowdown if Lucene uses the MMapDirectory > implementation on Java 9 in comparison to Java 8, we should really > look into the issues! So Andrew Haley is right, there is some > slowdown not only affecting HeapByteBuffers! Maybe it is caused by > missing vectorization or because the code calling the ByteBuffers is > much more complicated than Andrew?s tests, Hotspot is failing to > properly optimize the access to DirectByteBuffer, too. I'm happy to look at this, but you'll have to provide me with some sort of test case. "Read the Lucene manual. Install this server." probably isn't such a test case. Andrew. From mikael.gerdin at oracle.com Tue Mar 14 13:11:21 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Tue, 14 Mar 2017 14:11:21 +0100 Subject: RFR(M) 8176100: [REDO][REDO] G1 Needs pre barrier on dereference of weak JNI handles Message-ID: Here we go again... This time around the fix is exactly the same but since I changed the lock ranks in 8176363 we shouldn't be hitting any lock ordering assertions. Hi all, Please review this revised change to jweak to support G1. I've copied Kim's original description of the fix below for reference: > > The problem being addressed is that, when using G1, dereferencing a > jweak must ensure the obtained referent is ensured live, just as for > other weak reference types. This has always been a requirement, and > failure to do so appears to be a since-day1 G1 bug. > > There are two categories of places that need to address this issue: > JNIHandles::resolve and friends, and returning a jobject from native > code to Java. In both of these places the jobject whose contained oop > is being extracted might really be a jweak. jweak is > representationally indistinguishable from jobject; jweak is just a > typedef for jobject. The documentation says a jweak can be used > anywhere a jobject is permitted, making that type equivalence > necessary for a C API. (A C++ API might be able to have distinct > types via inheritance, though would still need a method for > distinguishing them at runtime. But since JNI is a C API...). > > The basic approach being taken is to "tag" jweaks by setting the low > order bit (recall that jweak == jobject == _jobject*), in order to > distinguish them from non-jweak jobjects. This tagging is only being > done when the selected GC needs the distinction, e.g. G1. This gives > us a representational difference between jobject and jweak on which to > base the different behavior, hiding that distinction behind the > existing API. > > JNIHandles::resolve and friends are changed to unconditionally strip > off any potential weak tag when translating from jobject to to oop*. > That's cheaper than testing for the conditional use of tagging and > then stripping. Also stripped when deleting JNI handles. > > TemplateInterpreterGenerator::generate_native_entry and > SharedRuntime::generate_native_wrapper are changed to conditionally > emit code to apply the G1 pre-barrier to the value obtained from a > jobject result, when the result is tagged as a jweak, which only > occurs when using G1. > > For arm32/arm64, this required moving the g1_write_barrier_pre > definitions from InterpreterMacroAssembler to MacroAssembler. Also > moved g1_write_barrier_post, for consistency. > In addition to Kim's final version of the change (which was backed out) this updated version adds code to mask out the weak tag bit in the fast JNI field getters on all platforms where fast JNI getters are used. Since the fast JNI getters only read primitive values from objects me and Kim decided that there was no need to invoke the G1 pre-barrier if a jweak was used as the "receiver" for the getters, this simplifies the change to the fast getters and I was able to come up with a seemingly efficient instruction encoding on all platforms. A couple of comments about the implementations: My goal was to encode clearing of the weak_tag_mask (which at the moment is 1). On SPARC and 32 bit arm there is an instruction that can be used straight off to perform an and with a value that is first negated, ANDN on SPARC and BTC on ARM. On 64 bit ARM the encoding for immediates allows encoding of the bitwise inverse of 1 (0xfffffffffffffffe) but if weak_tag_mask is changed the encoding may fail. On x86 I pass the mask as a signed 32 bit immediate but note that the assembler encodes it as an 8 bit immediate and sets a sign-extension bit in the opcode. Testing: Standard JPRT and tier2-3 HotSpot tests. I've modified the CallWithJNIWeak test to call all the primitive getters and some other interesting cases I came up with. I've also run the CallWithJNIWeak test on all platforms on both product and fastdebug builds (since fast JNI getters are implicitly disabled in debug builds due to VerifyJNIFields being enabled by default in debug builds. The aarch64 change has been tested in the previous round of this change. Full webrev: http://cr.openjdk.java.net/~mgerdin/8176100/webrev.0/ Bugs: https://bugs.openjdk.java.net/browse/JDK-8176100 https://bugs.openjdk.java.net/browse/JDK-8175085 https://bugs.openjdk.java.net/browse/JDK-8166188 Thanks /Mikael From michael.rasmussen at zeroturnaround.com Tue Mar 14 13:24:22 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Tue, 14 Mar 2017 15:24:22 +0200 Subject: @HotSpotIntrinsicCandidate and native prefixes Message-ID: Hi If you set a native prefix using SetNativeMethodPrefix, in order to wrap native methods. If those methods are annotated with @HotSpotIntrinsicCandidate you get a warning when running. For instance for Thread::isInterrupted: Compiler intrinsic is defined for method [java.lang.Thread.isInterrupted(Z)Z], but the method is not annotated with @HotSpotIntrinsicCandidate. Method will not be inlined. Method [java.lang.Thread.$prefix$isInterrupted(Z)Z] is annotated with @HotSpotIntrinsicCandidate, but no compiler intrinsic is defined for the method. Shouldn't the native prefix be taken into account for this? Kind regards Michael Rasmussen JRebel, ZeroTurnaround From thomas.stuefe at gmail.com Tue Mar 14 15:20:13 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 14 Mar 2017 16:20:13 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> References: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> Message-ID: Hi Coleen, Mikael, thanks again for looking over this change, I know its a lot of stuff. Coleen, thanks for offering sponsorship! Here the new webrev after your first input: webrev 01: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-8170933-unify-treatment-of-humongous-and-non-humongous-chunks-on-chunkmanager-return/jdk10-webrev.01/webrev/ delta 00-01: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-8170933-unify-treatment-of-humongous-and-non-humongous-chunks-on-chunkmanager-return/jdk10-webrev.00-to-01/webrev/ I addressed your issues: @Coleen > can you put ChunkManager::size_by_index() function definition up in the file with the other ChunkManager functions? I moved this function and three stray others up to the rest of the ChunkManager functions (size_by_index(), list_index(), and the two new ones, ChunkManager::return_single_chunk ChunkManager::return_chunk_list). @Mikael: >While you're at it, would you mind breaking > >assert(_free_chunks_count > 0 && > _free_chunks_total >= v, > "About to go negative"); > >into two separate asserts and have them print the values? Done. Note that the counters (.. _count) being of size_t kind of irritates me :) but I refrained from changing the type because touching that would spread all over. Maybe in a separate cleanup. >In ChunkManager::list_index you can now use size_by_index (and maybe make size_by_index inline in the ChunkManager class declaration? Done. >In ChunkManager::return_chunk_list you might as well use >LogTarget(Trace, gc, metaspace, freelist) log; >since you only log to the trace level anyway. Ok, did that. Note that there are a couple of more places where this could be done, but I did not touch those to keep the change small and reviewable. Kind Regards, Thomas On Mon, Mar 13, 2017 at 8:54 PM, wrote: > > Hi Thomas, Thank you for cleaning up the metaspace code. I think the > change looks good. I have only one comment and that is: can you put > ChunkManager::size_by_index() function definition up in the file with the > other ChunkManager functions? > > If Mikael reviews the test, I'd be happy to sponsor this for you. Sorry > about the inattention. > > thanks! > Coleen > > > On 3/11/17 4:08 AM, Thomas St?fe wrote: > >> Ping... Did I send this to the wrong mailing list? >> >> On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe >> wrote: >> >> (Resent to hotspot-dev) >>> >>> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >>> wrote: >>> >>> Hi all, >>>> >>>> I would like to get a review for this cleanup change for the >>>> metaspace.cpp. This has been a long time brewing in my backlog, but I >>>> had >>>> to wait until jdk10 is open. The cleanup is actually quite small, the >>>> largest part of the change is the added test coding. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>>> >>>> The change implements a suggestion made by Mikael Gerdin when >>>> reviewing JDK-8170520 last year, who suggested that the ChunkManager >>>> class >>>> should handle returns of both non-humongous and humongous chunks, see >>>> original discussion here: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>>> 016-November/021949.html >>>> >>>> "It would be great if you could have a look at unifying the >>>> ChunkManager::return_chunks API for standard and humongous chunks so >>>> that we wouldn't need the special casing for this in ~SpaceManager >>>> That way we could also make humongous_dictionary() private to >>>> ChunkManager which would improve encapsulation." >>>> >>>> The cleanup does this and a bit more, all changes having to do with the >>>> fact that the ChunkManager class unnecessarily exposes internals to the >>>> other classes in metaspace.cpp and that with a little bit of >>>> reorganization >>>> this can be avoided. The benefit is simpler coding and way better >>>> encapsulation, which will help a lot with future changes (in preparation >>>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>>> >>>> -------- >>>> >>>> The changes in detail: >>>> >>>> The main issue was that ChunkManager::return_chunks() did not handle >>>> humongous chunks. To return humongous chunks, one had to access the >>>> humongous chunk dictionary inside the ChunkManager and add the chunk >>>> yourself, including maintaining all counters. This happened in >>>> ~SpaceManager and made this destructor very complicated. >>>> >>>> This is solved by moving the handling of humongous chunk returns to the >>>> ChunkManager class itself. For convenience, I split the old >>>> "ChunkManager::return_chunks" method into two new ones, >>>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>>> the ChunkManager without walking the chunk chain, and >>>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>>> the ChunkManager. Both functions are now able to handle both >>>> non-humongous >>>> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >>>> simplified) and just calls "ChunkManager::return_chunk_list()" for all >>>> its chunk lists. >>>> >>>> So now we can remove the public access to the humongous chunk dictionary >>>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>>> function private. >>>> >>>> ---- >>>> >>>> Then there was the fact that the non-humongous chunk lists were also >>>> exposed to public via ChunkManager::free_chunks(). The only reason was >>>> that >>>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free >>>> lists >>>> to find out the size of the items of the free lists. >>>> >>>> This was solved by adding a new function, ChunkManager::size_by_index(), >>>> which returns the size of the items of a chunk list given by its chunk >>>> index. >>>> >>>> So ChunkManager::free_chunks() could be made private too. >>>> >>>> --- >>>> >>>> The rest are minor changes: >>>> >>>> - made ChunkManager::find_free_chunks_list() private because it was not >>>> used from outside the class >>>> - Fixed an assert in dec_free_chunks_total() >>>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >>>> to keep the existing logging intact or add useful logging where >>>> possible. >>>> >>>> ---- >>>> >>>> A large chunk of the changes is the added gtests. There is a new class >>>> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >>>> code. >>>> >>>> Note that I dislike the fact that this test is implemented inside >>>> metaspace.cpp itself. For now I kept my new tests like the existing >>>> tests >>>> inside this file, but I think these tests should be moved to >>>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>>> separate fix though, because it needs a bit of remodeling (the tests >>>> need >>>> to access internals in metaspace.cpp). >>>> >>>> ---- >>>> >>>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >>>> AIX. No issues popped up which were associated with my change. >>>> >>>> Thanks for reviewing and Kind Regards, Thomas >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> > From kim.barrett at oracle.com Tue Mar 14 15:29:54 2017 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 14 Mar 2017 11:29:54 -0400 Subject: RFR(M) 8176100: [REDO][REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: Message-ID: > On Mar 14, 2017, at 9:11 AM, Mikael Gerdin wrote: > > Here we go again... > This time around the fix is exactly the same but since I changed the lock ranks in 8176363 we shouldn't be hitting any lock ordering assertions. > > > [?] > Full webrev: > http://cr.openjdk.java.net/~mgerdin/8176100/webrev.0/ > > Bugs: > https://bugs.openjdk.java.net/browse/JDK-8176100 > https://bugs.openjdk.java.net/browse/JDK-8175085 > https://bugs.openjdk.java.net/browse/JDK-8166188 > > Thanks > /Mikael Looks good. But I've said that before. Hopefully we've run out of places that cheat about the "equivalence" of jobject and oop*. From serguei.spitsyn at oracle.com Tue Mar 14 17:34:50 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Mar 2017 10:34:50 -0700 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: References: Message-ID: I've added the seviceability-dev mailing list as the SetNativeMethodPrefix() is in the JVM TI interface. Thanks, Serguei On 3/14/17 06:24, Michael Rasmussen wrote: > Hi > > If you set a native prefix using SetNativeMethodPrefix, in order to wrap > native methods. If those methods are annotated with @HotSpotIntrinsicCandidate > you get a warning when running. > > For instance for Thread::isInterrupted: > Compiler intrinsic is defined for method > [java.lang.Thread.isInterrupted(Z)Z], but the method is not annotated with > @HotSpotIntrinsicCandidate. Method will not be inlined. > Method [java.lang.Thread.$prefix$isInterrupted(Z)Z] is annotated with > @HotSpotIntrinsicCandidate, but no compiler intrinsic is defined for the > method. > > Shouldn't the native prefix be taken into account for this? > > Kind regards > > Michael Rasmussen > JRebel, ZeroTurnaround From vladimir.x.ivanov at oracle.com Tue Mar 14 18:30:57 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 14 Mar 2017 21:30:57 +0300 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: References: Message-ID: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> Nice catch, Michael! @HotSpotIntrinsicCandidate starts to pay off :-) So, if a native method w/ an intrinsic is instrumented, it won't be intrinsified anymore. Looking into the code, it looks like it worked that way from the very beginning. Filed an RFE: https://bugs.openjdk.java.net/browse/JDK-8176771 Do you have any problems when the JVM issues such warnings? Best regards, Vladimir Ivanov > On 3/14/17 06:24, Michael Rasmussen wrote: >> Hi >> >> If you set a native prefix using SetNativeMethodPrefix, in order to wrap >> native methods. If those methods are annotated with >> @HotSpotIntrinsicCandidate >> you get a warning when running. >> >> For instance for Thread::isInterrupted: >> Compiler intrinsic is defined for method >> [java.lang.Thread.isInterrupted(Z)Z], but the method is not annotated >> with >> @HotSpotIntrinsicCandidate. Method will not be inlined. >> Method [java.lang.Thread.$prefix$isInterrupted(Z)Z] is annotated with >> @HotSpotIntrinsicCandidate, but no compiler intrinsic is defined for the >> method. >> >> Shouldn't the native prefix be taken into account for this? >> >> Kind regards >> >> Michael Rasmussen >> JRebel, ZeroTurnaround > From michael.rasmussen at zeroturnaround.com Tue Mar 14 18:45:30 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Tue, 14 Mar 2017 20:45:30 +0200 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> References: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> Message-ID: On 14 March 2017 at 20:30, Vladimir Ivanov wrote: > Do you have any problems when the JVM issues such warnings? We only just recently resumed our integration with JDK9 (was waiting for the API (especially jigsaw) to stabilize), so I don't yet know the full extend of these warnings, or if they will cause some actual problems as well. The warnings seems more of a nuisance than a problem, as we do add a native-prefix to a lot of native methods, there are bounds to be some hotspot intrinsic among them. But for now, I'll just add it to the list of warnings that can be ignored, most noticeably the "JavaLaunchHelper is implemented in both ... " warning on macOS, when launching with an agent. /Michael From vladimir.x.ivanov at oracle.com Tue Mar 14 18:48:59 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 14 Mar 2017 21:48:59 +0300 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: References: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> Message-ID: Just in case it will cause problems for you, there's a workaround: you can prune @HotSpotIntrinsicCandidate during instrumentation. Best regards, Vladimir Ivanov On 3/14/17 9:45 PM, Michael Rasmussen wrote: > On 14 March 2017 at 20:30, Vladimir Ivanov wrote: >> Do you have any problems when the JVM issues such warnings? > > We only just recently resumed our integration with JDK9 (was waiting > for the API (especially jigsaw) to stabilize), so I don't yet know the > full extend of these warnings, or if they will cause some actual > problems as well. > > The warnings seems more of a nuisance than a problem, as we do add a > native-prefix to a lot of native methods, there are bounds to be some > hotspot intrinsic among them. > But for now, I'll just add it to the list of warnings that can be > ignored, most noticeably the "JavaLaunchHelper is implemented in both > ... " warning on macOS, when launching with an agent. > > /Michael > From michael.rasmussen at zeroturnaround.com Tue Mar 14 18:50:26 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Tue, 14 Mar 2017 20:50:26 +0200 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: References: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> Message-ID: On 14 March 2017 at 20:48, Vladimir Ivanov wrote: > Just in case it will cause problems for you, there's a workaround: you can > prune @HotSpotIntrinsicCandidate during instrumentation. Wouldn't that just cause the 2nd warning to disappear, but the first one would remain? /Michael From vladimir.x.ivanov at oracle.com Tue Mar 14 19:03:42 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 14 Mar 2017 22:03:42 +0300 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: References: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> Message-ID: <1fef2eae-529f-4c01-2faf-30481a04e1d9@oracle.com> >> Just in case it will cause problems for you, there's a workaround: you can >> prune @HotSpotIntrinsicCandidate during instrumentation. > > Wouldn't that just cause the 2nd warning to disappear, but the first > one would remain? Yes, you are right. Then the only option is: -XX:+UnlockDiagnosticVMOptions -XX:-CheckIntrinsics Best regards, Vladimir Ivanov From coleen.phillimore at oracle.com Tue Mar 14 19:35:37 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 14 Mar 2017 15:35:37 -0400 Subject: RFR(M) 8176100: [REDO][REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: Message-ID: <55a1e130-9f0f-ac2b-faff-d59e1b0553e2@oracle.com> This looks good still. For https://bugs.openjdk.java.net/browse/JDK-8176363 why are the two locks "special"? Is that because there is no leaf-2? Just curious. Thanks, Coleen On 3/14/17 9:11 AM, Mikael Gerdin wrote: > Here we go again... > This time around the fix is exactly the same but since I changed the > lock ranks in 8176363 we shouldn't be hitting any lock ordering > assertions. > > > Hi all, > Please review this revised change to jweak to support G1. > > I've copied Kim's original description of the fix below for reference: > > > > > The problem being addressed is that, when using G1, dereferencing a > > jweak must ensure the obtained referent is ensured live, just as for > > other weak reference types. This has always been a requirement, and > > failure to do so appears to be a since-day1 G1 bug. > > > > There are two categories of places that need to address this issue: > > JNIHandles::resolve and friends, and returning a jobject from native > > code to Java. In both of these places the jobject whose contained oop > > is being extracted might really be a jweak. jweak is > > representationally indistinguishable from jobject; jweak is just a > > typedef for jobject. The documentation says a jweak can be used > > anywhere a jobject is permitted, making that type equivalence > > necessary for a C API. (A C++ API might be able to have distinct > > types via inheritance, though would still need a method for > > distinguishing them at runtime. But since JNI is a C API...). > > > > The basic approach being taken is to "tag" jweaks by setting the low > > order bit (recall that jweak == jobject == _jobject*), in order to > > distinguish them from non-jweak jobjects. This tagging is only being > > done when the selected GC needs the distinction, e.g. G1. This gives > > us a representational difference between jobject and jweak on which to > > base the different behavior, hiding that distinction behind the > > existing API. > > > > JNIHandles::resolve and friends are changed to unconditionally strip > > off any potential weak tag when translating from jobject to to oop*. > > That's cheaper than testing for the conditional use of tagging and > > then stripping. Also stripped when deleting JNI handles. > > > > TemplateInterpreterGenerator::generate_native_entry and > > SharedRuntime::generate_native_wrapper are changed to conditionally > > emit code to apply the G1 pre-barrier to the value obtained from a > > jobject result, when the result is tagged as a jweak, which only > > occurs when using G1. > > > > For arm32/arm64, this required moving the g1_write_barrier_pre > > definitions from InterpreterMacroAssembler to MacroAssembler. Also > > moved g1_write_barrier_post, for consistency. > > > > In addition to Kim's final version of the change (which was backed > out) this updated version adds code to mask out the weak tag bit in > the fast JNI field getters on all platforms where fast JNI getters are > used. > > Since the fast JNI getters only read primitive values from objects me > and Kim decided that there was no need to invoke the G1 pre-barrier if > a jweak was used as the "receiver" for the getters, this simplifies > the change to the fast getters and I was able to come up with a > seemingly efficient instruction encoding on all platforms. > > A couple of comments about the implementations: > My goal was to encode clearing of the weak_tag_mask (which at the > moment is 1). > On SPARC and 32 bit arm there is an instruction that can be used > straight off to perform an and with a value that is first negated, > ANDN on SPARC and BTC on ARM. > On 64 bit ARM the encoding for immediates allows encoding of the > bitwise inverse of 1 (0xfffffffffffffffe) but if weak_tag_mask is > changed the encoding may fail. > On x86 I pass the mask as a signed 32 bit immediate but note that the > assembler encodes it as an 8 bit immediate and sets a sign-extension > bit in the opcode. > > > Testing: > Standard JPRT and tier2-3 HotSpot tests. > I've modified the CallWithJNIWeak test to call all the primitive > getters and some other interesting cases I came up with. > I've also run the CallWithJNIWeak test on all platforms on both > product and fastdebug builds (since fast JNI getters are implicitly > disabled in debug builds due to VerifyJNIFields being enabled by > default in debug builds. > > The aarch64 change has been tested in the previous round of this change. > > Full webrev: > http://cr.openjdk.java.net/~mgerdin/8176100/webrev.0/ > > Bugs: > https://bugs.openjdk.java.net/browse/JDK-8176100 > https://bugs.openjdk.java.net/browse/JDK-8175085 > https://bugs.openjdk.java.net/browse/JDK-8166188 > > Thanks > /Mikael From serguei.spitsyn at oracle.com Tue Mar 14 20:11:34 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 14 Mar 2017 13:11:34 -0700 Subject: [8u-dev] RFR JDK-8171194: Exception "Duplicate field name&signature in class file" should report the name and signature of the field In-Reply-To: <5b0128b5-54e7-4042-943f-37f552ab8f88@default> References: <9d0fe9e5-21e4-b88d-6a9a-263c9eb26d1e@oracle.com> <5b0128b5-54e7-4042-943f-37f552ab8f88@default> Message-ID: Hi Shafi, It looks good to me. Thanks, Serguei On 3/12/17 20:52, Shafi Ahmad wrote: > Hi, > > May I get the second review for this simple backport. > > Regards, > Shafi > >> -----Original Message----- >> From: Shafi Ahmad >> Sent: Thursday, March 09, 2017 3:39 PM >> To: David Holmes ; hotspot- >> dev at openjdk.java.net >> Subject: RE: [8u-dev] RFR JDK-8171194: Exception "Duplicate field >> name&signature in class file" should report the name and signature of the >> field >> >> Hi David, >> >> Thank you for the review and letting me know about the enhancement >> approval process. >> >> Regards, >> Shafi >> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Thursday, March 09, 2017 11:46 AM >>> To: Shafi Ahmad ; hotspot- >>> dev at openjdk.java.net >>> Subject: Re: [8u-dev] RFR JDK-8171194: Exception "Duplicate field >>> name&signature in class file" should report the name and signature of >>> the field >>> >>> Hi Shafi, >>> >>> On 9/03/2017 3:39 PM, Shafi Ahmad wrote: >>>> Hi, >>>> >>>> Please review the backport of "JDK-8171194: Exception "Duplicate >>>> field >>> name&signature in class file" should report the name and signature of >>> the field" to jdk8u-dev. >>>> The backport is not clean even though the changes are very small. >>>> >>>> Webrev: http://cr.openjdk.java.net/~shshahma/8176150/webrev.00/ >>> The code changes look fine to me. >>> >>> But note that as this is an enhancement you will need to get an >>> enhancement approval for it to into 8u, as well as the actual push approval: >>> >>> http://openjdk.java.net/projects/jdk8u/enhancement-template.html >>> >>> Thanks, >>> David >>> >>>> Jdk10 bug: https://bugs.openjdk.java.net/browse/JDK-8171194 >>>> Jdk10 review: >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025 >>>> 87 >>>> 5.html >>>> >>>> Testing: jprt and jtreg test. >>>> >>>> >>>> Regards, >>>> Shafi >>>> From mikael.gerdin at oracle.com Wed Mar 15 12:27:25 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 15 Mar 2017 13:27:25 +0100 Subject: RFR(M) 8176100: [REDO][REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: <55a1e130-9f0f-ac2b-faff-d59e1b0553e2@oracle.com> References: <55a1e130-9f0f-ac2b-faff-d59e1b0553e2@oracle.com> Message-ID: <3b766df2-a243-e6ec-51a5-d970bf0fb07d@oracle.com> Hi Coleen, On 2017-03-14 20:35, coleen.phillimore at oracle.com wrote: > This looks good still. For Thanks. > https://bugs.openjdk.java.net/browse/JDK-8176363 > > why are the two locks "special"? Is that because there is no leaf-2? > Just curious. Unclear. I guess it's because they are guarding a very small region and someone wanted to make sure that no other locks are taken when they are held. /Mikael > > Thanks, > Coleen > > On 3/14/17 9:11 AM, Mikael Gerdin wrote: >> Here we go again... >> This time around the fix is exactly the same but since I changed the >> lock ranks in 8176363 we shouldn't be hitting any lock ordering >> assertions. >> >> >> Hi all, >> Please review this revised change to jweak to support G1. >> >> I've copied Kim's original description of the fix below for reference: >> >> > >> > The problem being addressed is that, when using G1, dereferencing a >> > jweak must ensure the obtained referent is ensured live, just as for >> > other weak reference types. This has always been a requirement, and >> > failure to do so appears to be a since-day1 G1 bug. >> > >> > There are two categories of places that need to address this issue: >> > JNIHandles::resolve and friends, and returning a jobject from native >> > code to Java. In both of these places the jobject whose contained oop >> > is being extracted might really be a jweak. jweak is >> > representationally indistinguishable from jobject; jweak is just a >> > typedef for jobject. The documentation says a jweak can be used >> > anywhere a jobject is permitted, making that type equivalence >> > necessary for a C API. (A C++ API might be able to have distinct >> > types via inheritance, though would still need a method for >> > distinguishing them at runtime. But since JNI is a C API...). >> > >> > The basic approach being taken is to "tag" jweaks by setting the low >> > order bit (recall that jweak == jobject == _jobject*), in order to >> > distinguish them from non-jweak jobjects. This tagging is only being >> > done when the selected GC needs the distinction, e.g. G1. This gives >> > us a representational difference between jobject and jweak on which to >> > base the different behavior, hiding that distinction behind the >> > existing API. >> > >> > JNIHandles::resolve and friends are changed to unconditionally strip >> > off any potential weak tag when translating from jobject to to oop*. >> > That's cheaper than testing for the conditional use of tagging and >> > then stripping. Also stripped when deleting JNI handles. >> > >> > TemplateInterpreterGenerator::generate_native_entry and >> > SharedRuntime::generate_native_wrapper are changed to conditionally >> > emit code to apply the G1 pre-barrier to the value obtained from a >> > jobject result, when the result is tagged as a jweak, which only >> > occurs when using G1. >> > >> > For arm32/arm64, this required moving the g1_write_barrier_pre >> > definitions from InterpreterMacroAssembler to MacroAssembler. Also >> > moved g1_write_barrier_post, for consistency. >> > >> >> In addition to Kim's final version of the change (which was backed >> out) this updated version adds code to mask out the weak tag bit in >> the fast JNI field getters on all platforms where fast JNI getters are >> used. >> >> Since the fast JNI getters only read primitive values from objects me >> and Kim decided that there was no need to invoke the G1 pre-barrier if >> a jweak was used as the "receiver" for the getters, this simplifies >> the change to the fast getters and I was able to come up with a >> seemingly efficient instruction encoding on all platforms. >> >> A couple of comments about the implementations: >> My goal was to encode clearing of the weak_tag_mask (which at the >> moment is 1). >> On SPARC and 32 bit arm there is an instruction that can be used >> straight off to perform an and with a value that is first negated, >> ANDN on SPARC and BTC on ARM. >> On 64 bit ARM the encoding for immediates allows encoding of the >> bitwise inverse of 1 (0xfffffffffffffffe) but if weak_tag_mask is >> changed the encoding may fail. >> On x86 I pass the mask as a signed 32 bit immediate but note that the >> assembler encodes it as an 8 bit immediate and sets a sign-extension >> bit in the opcode. >> >> >> Testing: >> Standard JPRT and tier2-3 HotSpot tests. >> I've modified the CallWithJNIWeak test to call all the primitive >> getters and some other interesting cases I came up with. >> I've also run the CallWithJNIWeak test on all platforms on both >> product and fastdebug builds (since fast JNI getters are implicitly >> disabled in debug builds due to VerifyJNIFields being enabled by >> default in debug builds. >> >> The aarch64 change has been tested in the previous round of this change. >> >> Full webrev: >> http://cr.openjdk.java.net/~mgerdin/8176100/webrev.0/ >> >> Bugs: >> https://bugs.openjdk.java.net/browse/JDK-8176100 >> https://bugs.openjdk.java.net/browse/JDK-8175085 >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> Thanks >> /Mikael > From mikael.gerdin at oracle.com Wed Mar 15 12:28:18 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 15 Mar 2017 13:28:18 +0100 Subject: RFR(M) 8176100: [REDO][REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: Message-ID: <5b92d3a8-0d63-e2cd-f177-02153a5efbcc@oracle.com> Hi Kim, On 2017-03-14 16:29, Kim Barrett wrote: >> On Mar 14, 2017, at 9:11 AM, Mikael Gerdin wrote: >> >> Here we go again... >> This time around the fix is exactly the same but since I changed the lock ranks in 8176363 we shouldn't be hitting any lock ordering assertions. >> >> >> [?] >> Full webrev: >> http://cr.openjdk.java.net/~mgerdin/8176100/webrev.0/ >> >> Bugs: >> https://bugs.openjdk.java.net/browse/JDK-8176100 >> https://bugs.openjdk.java.net/browse/JDK-8175085 >> https://bugs.openjdk.java.net/browse/JDK-8166188 >> >> Thanks >> /Mikael > > Looks good. But I've said that before. Hopefully we've run out of > places that cheat about the "equivalence" of jobject and oop*. Thanks. I'm re-running a bunch of tests upon request so I'll integrate this once testing is completed. /Mikael > From thomas.schatzl at oracle.com Wed Mar 15 12:34:24 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 15 Mar 2017 13:34:24 +0100 Subject: RFR(M) 8176100: [REDO][REDO] G1 Needs pre barrier on dereference of weak JNI handles In-Reply-To: References: Message-ID: <1489581264.3398.23.camel@oracle.com> Hi Mikael, On Tue, 2017-03-14 at 14:11 +0100, Mikael Gerdin wrote: > Here we go again... > This time around the fix is exactly the same but since I changed the? > lock ranks in 8176363 we shouldn't be hitting any lock ordering > assertions. ? (still) looks good to me. As discussed privately, please wait for the current test runs to complete before pushing to minimize the risk for a [REDO]^3. Thanks, ? Thomas From mikael.gerdin at oracle.com Wed Mar 15 12:34:55 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 15 Mar 2017 13:34:55 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: References: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> Message-ID: <38d7b4be-7ab3-d243-95ed-83f64c6ec959@oracle.com> Hi Thomas, On 2017-03-14 16:20, Thomas St?fe wrote: > Hi Coleen, Mikael, > > thanks again for looking over this change, I know its a lot of stuff. > Coleen, thanks for offering sponsorship! > > Here the new webrev after your first input: > > webrev 01: > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-8170933-unify-treatment-of-humongous-and-non-humongous-chunks-on-chunkmanager-return/jdk10-webrev.01/webrev/ > > delta 00-01: > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-8170933-unify-treatment-of-humongous-and-non-humongous-chunks-on-chunkmanager-return/jdk10-webrev.00-to-01/webrev/ > > I addressed your issues: > > @Coleen > >> can you put ChunkManager::size_by_index() function definition up in the > file with the other ChunkManager functions? > > I moved this function and three stray others up to the rest of the > ChunkManager functions (size_by_index(), list_index(), and the two new > ones, ChunkManager::return_single_chunk ChunkManager::return_chunk_list). > > @Mikael: > >> While you're at it, would you mind breaking >> >> assert(_free_chunks_count > 0 && >> _free_chunks_total >= v, >> "About to go negative"); >> >> into two separate asserts and have them print the values? > > Done. Note that the counters (.. _count) being of size_t kind of irritates > me :) but I refrained from changing the type because touching that would > spread all over. Maybe in a separate cleanup. Sounds good. > >> In ChunkManager::list_index you can now use size_by_index (and maybe make > size_by_index inline in the ChunkManager class declaration? > > Done. > >> In ChunkManager::return_chunk_list you might as well use >> LogTarget(Trace, gc, metaspace, freelist) log; >> since you only log to the trace level anyway. > > Ok, did that. Note that there are a couple of more places where this could > be done, but I did not touch those to keep the change small and reviewable. > Ok. I've had a look at the test now and while I think it's valuable to have I'd like us to have some sort of actual unit test also. The test you added is more of a stress test as you state in the comment. Perhaps now is also the time to rip out the classes local to metaspace.cpp in order to make them actually testable without having to do this complex dance with calling extern functions from the test code. I'm not going to force you to do the move in this change but I'd really like us to do that in 10 now that we have the chance to do some cleanups, what do you think Thomas? Coleen? /Mikael > Kind Regards, Thomas > > > On Mon, Mar 13, 2017 at 8:54 PM, wrote: > >> >> Hi Thomas, Thank you for cleaning up the metaspace code. I think the >> change looks good. I have only one comment and that is: can you put >> ChunkManager::size_by_index() function definition up in the file with the >> other ChunkManager functions? >> >> If Mikael reviews the test, I'd be happy to sponsor this for you. Sorry >> about the inattention. >> >> thanks! >> Coleen >> >> >> On 3/11/17 4:08 AM, Thomas St?fe wrote: >> >>> Ping... Did I send this to the wrong mailing list? >>> >>> On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe >>> wrote: >>> >>> (Resent to hotspot-dev) >>>> >>>> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >>>> wrote: >>>> >>>> Hi all, >>>>> >>>>> I would like to get a review for this cleanup change for the >>>>> metaspace.cpp. This has been a long time brewing in my backlog, but I >>>>> had >>>>> to wait until jdk10 is open. The cleanup is actually quite small, the >>>>> largest part of the change is the added test coding. >>>>> >>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>>>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>>>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>>>> >>>>> The change implements a suggestion made by Mikael Gerdin when >>>>> reviewing JDK-8170520 last year, who suggested that the ChunkManager >>>>> class >>>>> should handle returns of both non-humongous and humongous chunks, see >>>>> original discussion here: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>>>> 016-November/021949.html >>>>> >>>>> "It would be great if you could have a look at unifying the >>>>> ChunkManager::return_chunks API for standard and humongous chunks so >>>>> that we wouldn't need the special casing for this in ~SpaceManager >>>>> That way we could also make humongous_dictionary() private to >>>>> ChunkManager which would improve encapsulation." >>>>> >>>>> The cleanup does this and a bit more, all changes having to do with the >>>>> fact that the ChunkManager class unnecessarily exposes internals to the >>>>> other classes in metaspace.cpp and that with a little bit of >>>>> reorganization >>>>> this can be avoided. The benefit is simpler coding and way better >>>>> encapsulation, which will help a lot with future changes (in preparation >>>>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>>>> >>>>> -------- >>>>> >>>>> The changes in detail: >>>>> >>>>> The main issue was that ChunkManager::return_chunks() did not handle >>>>> humongous chunks. To return humongous chunks, one had to access the >>>>> humongous chunk dictionary inside the ChunkManager and add the chunk >>>>> yourself, including maintaining all counters. This happened in >>>>> ~SpaceManager and made this destructor very complicated. >>>>> >>>>> This is solved by moving the handling of humongous chunk returns to the >>>>> ChunkManager class itself. For convenience, I split the old >>>>> "ChunkManager::return_chunks" method into two new ones, >>>>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>>>> the ChunkManager without walking the chunk chain, and >>>>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>>>> the ChunkManager. Both functions are now able to handle both >>>>> non-humongous >>>>> and humongous chunks. ~SpaceManager() was reimplemented (actually, quite >>>>> simplified) and just calls "ChunkManager::return_chunk_list()" for all >>>>> its chunk lists. >>>>> >>>>> So now we can remove the public access to the humongous chunk dictionary >>>>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>>>> function private. >>>>> >>>>> ---- >>>>> >>>>> Then there was the fact that the non-humongous chunk lists were also >>>>> exposed to public via ChunkManager::free_chunks(). The only reason was >>>>> that >>>>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free >>>>> lists >>>>> to find out the size of the items of the free lists. >>>>> >>>>> This was solved by adding a new function, ChunkManager::size_by_index(), >>>>> which returns the size of the items of a chunk list given by its chunk >>>>> index. >>>>> >>>>> So ChunkManager::free_chunks() could be made private too. >>>>> >>>>> --- >>>>> >>>>> The rest are minor changes: >>>>> >>>>> - made ChunkManager::find_free_chunks_list() private because it was not >>>>> used from outside the class >>>>> - Fixed an assert in dec_free_chunks_total() >>>>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I tried >>>>> to keep the existing logging intact or add useful logging where >>>>> possible. >>>>> >>>>> ---- >>>>> >>>>> A large chunk of the changes is the added gtests. There is a new class >>>>> now, ChunkManagerReturnTest, which stresses the ChunkManager take/return >>>>> code. >>>>> >>>>> Note that I dislike the fact that this test is implemented inside >>>>> metaspace.cpp itself. For now I kept my new tests like the existing >>>>> tests >>>>> inside this file, but I think these tests should be moved to >>>>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>>>> separate fix though, because it needs a bit of remodeling (the tests >>>>> need >>>>> to access internals in metaspace.cpp). >>>>> >>>>> ---- >>>>> >>>>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc and >>>>> AIX. No issues popped up which were associated with my change. >>>>> >>>>> Thanks for reviewing and Kind Regards, Thomas >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >> From thomas.stuefe at gmail.com Wed Mar 15 13:34:01 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 15 Mar 2017 14:34:01 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: <38d7b4be-7ab3-d243-95ed-83f64c6ec959@oracle.com> References: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> <38d7b4be-7ab3-d243-95ed-83f64c6ec959@oracle.com> Message-ID: Hi Mikael, Coleen, On Wed, Mar 15, 2017 at 1:34 PM, Mikael Gerdin wrote: > Hi Thomas, > > > On 2017-03-14 16:20, Thomas St?fe wrote: > >> Hi Coleen, Mikael, >> >> thanks again for looking over this change, I know its a lot of stuff. >> Coleen, thanks for offering sponsorship! >> >> Here the new webrev after your first input: >> >> webrev 01: >> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-81709 >> 33-unify-treatment-of-humongous-and-non-humongous-chunks-on- >> chunkmanager-return/jdk10-webrev.01/webrev/ >> >> delta 00-01: >> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-81709 >> 33-unify-treatment-of-humongous-and-non-humongous-chunks-on- >> chunkmanager-return/jdk10-webrev.00-to-01/webrev/ >> >> I addressed your issues: >> >> @Coleen >> >> can you put ChunkManager::size_by_index() function definition up in the >>> >> file with the other ChunkManager functions? >> >> I moved this function and three stray others up to the rest of the >> ChunkManager functions (size_by_index(), list_index(), and the two new >> ones, ChunkManager::return_single_chunk ChunkManager::return_chunk_list). >> >> @Mikael: >> >> While you're at it, would you mind breaking >>> >>> assert(_free_chunks_count > 0 && >>> _free_chunks_total >= v, >>> "About to go negative"); >>> >>> into two separate asserts and have them print the values? >>> >> >> Done. Note that the counters (.. _count) being of size_t kind of irritates >> me :) but I refrained from changing the type because touching that would >> spread all over. Maybe in a separate cleanup. >> > > Sounds good. > > >> In ChunkManager::list_index you can now use size_by_index (and maybe make >>> >> size_by_index inline in the ChunkManager class declaration? >> >> Done. >> >> In ChunkManager::return_chunk_list you might as well use >>> LogTarget(Trace, gc, metaspace, freelist) log; >>> since you only log to the trace level anyway. >>> >> >> Ok, did that. Note that there are a couple of more places where this could >> be done, but I did not touch those to keep the change small and >> reviewable. >> >> > Ok. > > I've had a look at the test now and while I think it's valuable to have > I'd like us to have some sort of actual unit test also. The test you added > is more of a stress test as you state in the comment. > How would they differ from the tests I did? I am all in favor of more tests, it makes sense to have them now before larger changes to metaspace.cpp happens. I am just curious about how you would test, and what, exactly. > Perhaps now is also the time to rip out the classes local to metaspace.cpp > in order to make them actually testable without having to do this complex > dance with calling extern functions from the test code. > > Yes! This can be easily isolated as an own separate issue. To track it, I opened: https://bugs.openjdk.java.net/browse/JDK-8176808 > I'm not going to force you to do the move in this change but I'd really > like us to do that in 10 now that we have the chance to do some cleanups, > what do you think Thomas? Coleen? I really would like to wrap this change here up. I prefer to do both moving the test code out of metaspace.cpp (JDK-8176808) and adding the unit tests you requested as separate follow up issues. I still have three more changes to metaspace on the back burner: A) the delayed https://bugs.openjdk.java.net/browse/JDK-8170520 ("Make ChunkManager counters non-atomic") - this one is a very small change, especially after the cleanup done in this change. I will post this once this change got comitted. B) two more changes, one to add something called a "metaspace map", a way to print an ascii representation of the metaspace. And as follow up, the prototype for the new allocator. The "metaspace map" will be a way to easily see the effect of the new allocator (inspect the metaspace fragmentation). I would propose to add the unit tests you wanted in between (A) and (B). Kind Regards, Thomas > > > /Mikael > > > Kind Regards, Thomas >> >> >> On Mon, Mar 13, 2017 at 8:54 PM, wrote: >> >> >>> Hi Thomas, Thank you for cleaning up the metaspace code. I think the >>> change looks good. I have only one comment and that is: can you put >>> ChunkManager::size_by_index() function definition up in the file with the >>> other ChunkManager functions? >>> >>> If Mikael reviews the test, I'd be happy to sponsor this for you. Sorry >>> about the inattention. >>> >>> thanks! >>> Coleen >>> >>> >>> On 3/11/17 4:08 AM, Thomas St?fe wrote: >>> >>> Ping... Did I send this to the wrong mailing list? >>>> >>>> On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe >>>> wrote: >>>> >>>> (Resent to hotspot-dev) >>>> >>>>> >>>>> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >>>>> wrote: >>>>> >>>>> Hi all, >>>>> >>>>>> >>>>>> I would like to get a review for this cleanup change for the >>>>>> metaspace.cpp. This has been a long time brewing in my backlog, but I >>>>>> had >>>>>> to wait until jdk10 is open. The cleanup is actually quite small, the >>>>>> largest part of the change is the added test coding. >>>>>> >>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>>>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>>>>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>>>>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>>>>> >>>>>> The change implements a suggestion made by Mikael Gerdin when >>>>>> reviewing JDK-8170520 last year, who suggested that the ChunkManager >>>>>> class >>>>>> should handle returns of both non-humongous and humongous chunks, see >>>>>> original discussion here: >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>>>>> 016-November/021949.html >>>>>> >>>>>> "It would be great if you could have a look at unifying the >>>>>> ChunkManager::return_chunks API for standard and humongous chunks so >>>>>> that we wouldn't need the special casing for this in ~SpaceManager >>>>>> That way we could also make humongous_dictionary() private to >>>>>> ChunkManager which would improve encapsulation." >>>>>> >>>>>> The cleanup does this and a bit more, all changes having to do with >>>>>> the >>>>>> fact that the ChunkManager class unnecessarily exposes internals to >>>>>> the >>>>>> other classes in metaspace.cpp and that with a little bit of >>>>>> reorganization >>>>>> this can be avoided. The benefit is simpler coding and way better >>>>>> encapsulation, which will help a lot with future changes (in >>>>>> preparation >>>>>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>>>>> >>>>>> -------- >>>>>> >>>>>> The changes in detail: >>>>>> >>>>>> The main issue was that ChunkManager::return_chunks() did not handle >>>>>> humongous chunks. To return humongous chunks, one had to access the >>>>>> humongous chunk dictionary inside the ChunkManager and add the chunk >>>>>> yourself, including maintaining all counters. This happened in >>>>>> ~SpaceManager and made this destructor very complicated. >>>>>> >>>>>> This is solved by moving the handling of humongous chunk returns to >>>>>> the >>>>>> ChunkManager class itself. For convenience, I split the old >>>>>> "ChunkManager::return_chunks" method into two new ones, >>>>>> "ChunkManager::return_single_chunk()" which returns a single chunk to >>>>>> the ChunkManager without walking the chunk chain, and >>>>>> "ChunkManger::return_chunk_list()" which returns a chain of chunks to >>>>>> the ChunkManager. Both functions are now able to handle both >>>>>> non-humongous >>>>>> and humongous chunks. ~SpaceManager() was reimplemented (actually, >>>>>> quite >>>>>> simplified) and just calls "ChunkManager::return_chunk_list()" for >>>>>> all >>>>>> its chunk lists. >>>>>> >>>>>> So now we can remove the public access to the humongous chunk >>>>>> dictionary >>>>>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>>>>> function private. >>>>>> >>>>>> ---- >>>>>> >>>>>> Then there was the fact that the non-humongous chunk lists were also >>>>>> exposed to public via ChunkManager::free_chunks(). The only reason was >>>>>> that >>>>>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free >>>>>> lists >>>>>> to find out the size of the items of the free lists. >>>>>> >>>>>> This was solved by adding a new function, >>>>>> ChunkManager::size_by_index(), >>>>>> which returns the size of the items of a chunk list given by its chunk >>>>>> index. >>>>>> >>>>>> So ChunkManager::free_chunks() could be made private too. >>>>>> >>>>>> --- >>>>>> >>>>>> The rest are minor changes: >>>>>> >>>>>> - made ChunkManager::find_free_chunks_list() private because it was >>>>>> not >>>>>> used from outside the class >>>>>> - Fixed an assert in dec_free_chunks_total() >>>>>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I >>>>>> tried >>>>>> to keep the existing logging intact or add useful logging where >>>>>> possible. >>>>>> >>>>>> ---- >>>>>> >>>>>> A large chunk of the changes is the added gtests. There is a new class >>>>>> now, ChunkManagerReturnTest, which stresses the ChunkManager >>>>>> take/return >>>>>> code. >>>>>> >>>>>> Note that I dislike the fact that this test is implemented inside >>>>>> metaspace.cpp itself. For now I kept my new tests like the existing >>>>>> tests >>>>>> inside this file, but I think these tests should be moved to >>>>>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>>>>> separate fix though, because it needs a bit of remodeling (the tests >>>>>> need >>>>>> to access internals in metaspace.cpp). >>>>>> >>>>>> ---- >>>>>> >>>>>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc >>>>>> and >>>>>> AIX. No issues popped up which were associated with my change. >>>>>> >>>>>> Thanks for reviewing and Kind Regards, Thomas >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>> From coleen.phillimore at oracle.com Wed Mar 15 14:56:11 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 15 Mar 2017 10:56:11 -0400 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: References: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> <38d7b4be-7ab3-d243-95ed-83f64c6ec959@oracle.com> Message-ID: <7ab4ced7-284b-2cf6-749f-34fe760a9974@oracle.com> On 3/15/17 9:34 AM, Thomas St?fe wrote: > Hi Mikael, Coleen, > > On Wed, Mar 15, 2017 at 1:34 PM, Mikael Gerdin > > wrote: > > Hi Thomas, > > > On 2017-03-14 16:20, Thomas St?fe wrote: > > Hi Coleen, Mikael, > > thanks again for looking over this change, I know its a lot of > stuff. > Coleen, thanks for offering sponsorship! > > Here the new webrev after your first input: > > webrev 01: > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-8170933-unify-treatment-of-humongous-and-non-humongous-chunks-on-chunkmanager-return/jdk10-webrev.01/webrev/ > > > delta 00-01: > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-8170933-unify-treatment-of-humongous-and-non-humongous-chunks-on-chunkmanager-return/jdk10-webrev.00-to-01/webrev/ > > > I addressed your issues: > > @Coleen > > can you put ChunkManager::size_by_index() function > definition up in the > > file with the other ChunkManager functions? > > I moved this function and three stray others up to the rest of the > ChunkManager functions (size_by_index(), list_index(), and the > two new > ones, ChunkManager::return_single_chunk > ChunkManager::return_chunk_list). > > @Mikael: > > While you're at it, would you mind breaking > > assert(_free_chunks_count > 0 && > _free_chunks_total >= v, > "About to go negative"); > > into two separate asserts and have them print the values? > > > Done. Note that the counters (.. _count) being of size_t kind > of irritates > me :) but I refrained from changing the type because touching > that would > spread all over. Maybe in a separate cleanup. > > > Sounds good. > > > In ChunkManager::list_index you can now use size_by_index > (and maybe make > > size_by_index inline in the ChunkManager class declaration? > > Done. > > In ChunkManager::return_chunk_list you might as well use > LogTarget(Trace, gc, metaspace, freelist) log; > since you only log to the trace level anyway. > > > Ok, did that. Note that there are a couple of more places > where this could > be done, but I did not touch those to keep the change small > and reviewable. > > > Ok. > > I've had a look at the test now and while I think it's valuable to > have I'd like us to have some sort of actual unit test also. The > test you added is more of a stress test as you state in the comment. > > > How would they differ from the tests I did? I am all in favor of more > tests, it makes sense to have them now before larger changes to > metaspace.cpp happens. I am just curious about how you would test, and > what, exactly. > > Perhaps now is also the time to rip out the classes local to > metaspace.cpp in order to make them actually testable without > having to do this complex dance with calling extern functions from > the test code. > > > Yes! This can be easily isolated as an own separate issue. To track > it, I opened: https://bugs.openjdk.java.net/browse/JDK-8176808 > > I'm not going to force you to do the move in this change but I'd > really like us to do that in 10 now that we have the chance to do > some cleanups, what do you think Thomas? Coleen? > YES, please! > > I really would like to wrap this change here up. I prefer to do both > moving the test code out of metaspace.cpp (JDK-8176808) and adding the > unit tests you requested as separate follow up issues. I think it would be a good follow up fix to break up the tests. The reason I didn't review it very well was because it's too big. I'm happy to sponsor this change as-is. Thank you Thomas for working on metaspace.cpp. Compared to some of the other code in hotspot, it's a new piece of code and still needs shaking out and cleaning up. Coleen > > I still have three more changes to metaspace on the back burner: > A) the delayed https://bugs.openjdk.java.net/browse/JDK-8170520 ("Make > ChunkManager counters non-atomic") - this one is a very small change, > especially after the cleanup done in this change. I will post this > once this change got comitted. > B) two more changes, one to add something called a "metaspace map", a > way to print an ascii representation of the metaspace. And as follow > up, the prototype for the new allocator. The "metaspace map" will be a > way to easily see the effect of the new allocator (inspect the > metaspace fragmentation). > > I would propose to add the unit tests you wanted in between (A) and (B). > > Kind Regards, Thomas > > > > /Mikael > > > Kind Regards, Thomas > > > On Mon, Mar 13, 2017 at 8:54 PM, > wrote: > > > Hi Thomas, Thank you for cleaning up the metaspace code. > I think the > change looks good. I have only one comment and that is: > can you put > ChunkManager::size_by_index() function definition up in > the file with the > other ChunkManager functions? > > If Mikael reviews the test, I'd be happy to sponsor this > for you. Sorry > about the inattention. > > thanks! > Coleen > > > On 3/11/17 4:08 AM, Thomas St?fe wrote: > > Ping... Did I send this to the wrong mailing list? > > On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe > > > wrote: > > (Resent to hotspot-dev) > > > On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe > > > wrote: > > Hi all, > > > I would like to get a review for this cleanup > change for the > metaspace.cpp. This has been a long time > brewing in my backlog, but I > had > to wait until jdk10 is open. The cleanup is > actually quite small, the > largest part of the change is the added test > coding. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8170933 > > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE > > -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch > unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ > > The change implements a suggestion made by > Mikael Gerdin when > reviewing JDK-8170520 last year, who suggested > that the ChunkManager > class > should handle returns of both non-humongous > and humongous chunks, see > original discussion here: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 > > 016-November/021949.html > > "It would be great if you could have a look at > unifying the > ChunkManager::return_chunks API for standard > and humongous chunks so > that we wouldn't need the special casing for > this in ~SpaceManager > That way we could also make > humongous_dictionary() private to > ChunkManager which would improve encapsulation." > > The cleanup does this and a bit more, all > changes having to do with the > fact that the ChunkManager class unnecessarily > exposes internals to the > other classes in metaspace.cpp and that with a > little bit of > reorganization > this can be avoided. The benefit is simpler > coding and way better > encapsulation, which will help a lot with > future changes (in preparation > for > https://bugs.openjdk.java.net/browse/JDK-8166690 > ). > > -------- > > The changes in detail: > > The main issue was that > ChunkManager::return_chunks() did not handle > humongous chunks. To return humongous chunks, > one had to access the > humongous chunk dictionary inside the > ChunkManager and add the chunk > yourself, including maintaining all counters. > This happened in > ~SpaceManager and made this destructor very > complicated. > > This is solved by moving the handling of > humongous chunk returns to the > ChunkManager class itself. For convenience, I > split the old > "ChunkManager::return_chunks" method into two > new ones, > "ChunkManager::return_single_chunk()" which > returns a single chunk to > the ChunkManager without walking the chunk > chain, and > "ChunkManger::return_chunk_list()" which > returns a chain of chunks to > the ChunkManager. Both functions are now able > to handle both > non-humongous > and humongous chunks. ~SpaceManager() was > reimplemented (actually, quite > simplified) and just calls > "ChunkManager::return_chunk_list()" for all > its chunk lists. > > So now we can remove the public access to the > humongous chunk dictionary > in ChunkManger > (ChunkManager::humongous_dictionary()) and > make this > function private. > > ---- > > Then there was the fact that the non-humongous > chunk lists were also > exposed to public via > ChunkManager::free_chunks(). The only reason was > that > when a VirtualSpaceNode is retired, it > accessed the ChunkManager free > lists > to find out the size of the items of the free > lists. > > This was solved by adding a new function, > ChunkManager::size_by_index(), > which returns the size of the items of a chunk > list given by its chunk > index. > > So ChunkManager::free_chunks() could be made > private too. > > --- > > The rest are minor changes: > > - made ChunkManager::find_free_chunks_list() > private because it was not > used from outside the class > - Fixed an assert in dec_free_chunks_total() > - I added logging (UL, with the tags "gc, > metaspace, freelist"). I tried > to keep the existing logging intact or add > useful logging where > possible. > > ---- > > A large chunk of the changes is the added > gtests. There is a new class > now, ChunkManagerReturnTest, which stresses > the ChunkManager take/return > code. > > Note that I dislike the fact that this test is > implemented inside > metaspace.cpp itself. For now I kept my new > tests like the existing > tests > inside this file, but I think these tests > should be moved to > test/native/memory/test_chunkManager.cpp. I > suggest doing this in a > separate fix though, because it needs a bit of > remodeling (the tests > need > to access internals in metaspace.cpp). > > ---- > > I ran gtests and the jtreg hotspot tests on > Linux x64, Solaris sparc and > AIX. No issues popped up which were associated > with my change. > > Thanks for reviewing and Kind Regards, Thomas > > > > > > > > > > > > > > > From thomas.stuefe at gmail.com Wed Mar 15 15:32:14 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 15 Mar 2017 16:32:14 +0100 Subject: (jdk10) RFR(m): 8170933: Cleanup Metaspace Chunk manager: Unify treatment of humongous and non-humongous chunks In-Reply-To: <7ab4ced7-284b-2cf6-749f-34fe760a9974@oracle.com> References: <90ef2a18-7732-7512-e24c-ad428cfb2650@oracle.com> <38d7b4be-7ab3-d243-95ed-83f64c6ec959@oracle.com> <7ab4ced7-284b-2cf6-749f-34fe760a9974@oracle.com> Message-ID: On Wed, Mar 15, 2017 at 3:56 PM, wrote: > > > On 3/15/17 9:34 AM, Thomas St?fe wrote: > > Hi Mikael, Coleen, > > On Wed, Mar 15, 2017 at 1:34 PM, Mikael Gerdin > wrote: > >> Hi Thomas, >> >> >> On 2017-03-14 16:20, Thomas St?fe wrote: >> >>> Hi Coleen, Mikael, >>> >>> thanks again for looking over this change, I know its a lot of stuff. >>> Coleen, thanks for offering sponsorship! >>> >>> Here the new webrev after your first input: >>> >>> webrev 01: >>> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-81709 >>> 33-unify-treatment-of-humongous-and-non-humongous-chunks-on- >>> chunkmanager-return/jdk10-webrev.01/webrev/ >>> >>> delta 00-01: >>> http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-1-81709 >>> 33-unify-treatment-of-humongous-and-non-humongous-chunks-on- >>> chunkmanager-return/jdk10-webrev.00-to-01/webrev/ >>> >>> I addressed your issues: >>> >>> @Coleen >>> >>> can you put ChunkManager::size_by_index() function definition up in the >>>> >>> file with the other ChunkManager functions? >>> >>> I moved this function and three stray others up to the rest of the >>> ChunkManager functions (size_by_index(), list_index(), and the two new >>> ones, ChunkManager::return_single_chunk ChunkManager::return_chunk_lis >>> t). >>> >>> @Mikael: >>> >>> While you're at it, would you mind breaking >>>> >>>> assert(_free_chunks_count > 0 && >>>> _free_chunks_total >= v, >>>> "About to go negative"); >>>> >>>> into two separate asserts and have them print the values? >>>> >>> >>> Done. Note that the counters (.. _count) being of size_t kind of >>> irritates >>> me :) but I refrained from changing the type because touching that would >>> spread all over. Maybe in a separate cleanup. >>> >> >> Sounds good. >> >> >>> In ChunkManager::list_index you can now use size_by_index (and maybe make >>>> >>> size_by_index inline in the ChunkManager class declaration? >>> >>> Done. >>> >>> In ChunkManager::return_chunk_list you might as well use >>>> LogTarget(Trace, gc, metaspace, freelist) log; >>>> since you only log to the trace level anyway. >>>> >>> >>> Ok, did that. Note that there are a couple of more places where this >>> could >>> be done, but I did not touch those to keep the change small and >>> reviewable. >>> >>> >> Ok. >> >> I've had a look at the test now and while I think it's valuable to have >> I'd like us to have some sort of actual unit test also. The test you added >> is more of a stress test as you state in the comment. >> > > How would they differ from the tests I did? I am all in favor of more > tests, it makes sense to have them now before larger changes to > metaspace.cpp happens. I am just curious about how you would test, and > what, exactly. > > >> Perhaps now is also the time to rip out the classes local to >> metaspace.cpp in order to make them actually testable without having to do >> this complex dance with calling extern functions from the test code. >> >> > Yes! This can be easily isolated as an own separate issue. To track it, I > opened: https://bugs.openjdk.java.net/browse/JDK-8176808 > > >> I'm not going to force you to do the move in this change but I'd really >> like us to do that in 10 now that we have the chance to do some cleanups, >> what do you think Thomas? Coleen? > > > YES, please! > > > I really would like to wrap this change here up. I prefer to do both > moving the test code out of metaspace.cpp (JDK-8176808) and adding the unit > tests you requested as separate follow up issues. > > > I think it would be a good follow up fix to break up the tests. The > reason I didn't review it very well was because it's too big. > > I'm happy to sponsor this change as-is. > > Thank you Thomas for working on metaspace.cpp. Compared to some of the > other code in hotspot, it's a new piece of code and still needs shaking out > and cleaning up. > > Coleen > > Thank you both for the review work! We are getting there. I sometimes wonder whether it would be simpler to plug in a completely new allocator, but I suspect after a while we would arrive at the same complex coding, because the usage scenario is complex and this would be reflected in the allocator coding. So, we would not have gained much. Kind Regards, Thomas > > I still have three more changes to metaspace on the back burner: > A) the delayed https://bugs.openjdk.java.net/browse/JDK-8170520 ("Make > ChunkManager counters non-atomic") - this one is a very small change, > especially after the cleanup done in this change. I will post this once > this change got comitted. > B) two more changes, one to add something called a "metaspace map", a way > to print an ascii representation of the metaspace. And as follow up, the > prototype for the new allocator. The "metaspace map" will be a way to > easily see the effect of the new allocator (inspect the metaspace > fragmentation). > > I would propose to add the unit tests you wanted in between (A) and (B). > > Kind Regards, Thomas > > >> >> >> /Mikael >> >> >> Kind Regards, Thomas >>> >>> >>> On Mon, Mar 13, 2017 at 8:54 PM, wrote: >>> >>> >>>> Hi Thomas, Thank you for cleaning up the metaspace code. I think the >>>> change looks good. I have only one comment and that is: can you put >>>> ChunkManager::size_by_index() function definition up in the file with >>>> the >>>> other ChunkManager functions? >>>> >>>> If Mikael reviews the test, I'd be happy to sponsor this for you. Sorry >>>> about the inattention. >>>> >>>> thanks! >>>> Coleen >>>> >>>> >>>> On 3/11/17 4:08 AM, Thomas St?fe wrote: >>>> >>>> Ping... Did I send this to the wrong mailing list? >>>>> >>>>> On Tue, Mar 7, 2017 at 3:01 PM, Thomas St?fe >>>>> wrote: >>>>> >>>>> (Resent to hotspot-dev) >>>>> >>>>>> >>>>>> On Tue, Mar 7, 2017 at 7:45 AM, Thomas St?fe >>>>> > >>>>>> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>>> >>>>>>> I would like to get a review for this cleanup change for the >>>>>>> metaspace.cpp. This has been a long time brewing in my backlog, but I >>>>>>> had >>>>>>> to wait until jdk10 is open. The cleanup is actually quite small, the >>>>>>> largest part of the change is the added test coding. >>>>>>> >>>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8170933 >>>>>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE >>>>>>> -1-8170933-unify-treatment-of-humongous-and-non-humongous-ch >>>>>>> unks-on-chunkmanager-return/jdk10-webrev.00/webrev/ >>>>>>> >>>>>>> The change implements a suggestion made by Mikael Gerdin when >>>>>>> reviewing JDK-8170520 last year, who suggested that the ChunkManager >>>>>>> class >>>>>>> should handle returns of both non-humongous and humongous chunks, see >>>>>>> original discussion here: >>>>>>> >>>>>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2 >>>>>>> 016-November/021949.html >>>>>>> >>>>>>> "It would be great if you could have a look at unifying the >>>>>>> ChunkManager::return_chunks API for standard and humongous chunks so >>>>>>> that we wouldn't need the special casing for this in ~SpaceManager >>>>>>> That way we could also make humongous_dictionary() private to >>>>>>> ChunkManager which would improve encapsulation." >>>>>>> >>>>>>> The cleanup does this and a bit more, all changes having to do with >>>>>>> the >>>>>>> fact that the ChunkManager class unnecessarily exposes internals to >>>>>>> the >>>>>>> other classes in metaspace.cpp and that with a little bit of >>>>>>> reorganization >>>>>>> this can be avoided. The benefit is simpler coding and way better >>>>>>> encapsulation, which will help a lot with future changes (in >>>>>>> preparation >>>>>>> for https://bugs.openjdk.java.net/browse/JDK-8166690). >>>>>>> >>>>>>> -------- >>>>>>> >>>>>>> The changes in detail: >>>>>>> >>>>>>> The main issue was that ChunkManager::return_chunks() did not handle >>>>>>> humongous chunks. To return humongous chunks, one had to access the >>>>>>> humongous chunk dictionary inside the ChunkManager and add the chunk >>>>>>> yourself, including maintaining all counters. This happened in >>>>>>> ~SpaceManager and made this destructor very complicated. >>>>>>> >>>>>>> This is solved by moving the handling of humongous chunk returns to >>>>>>> the >>>>>>> ChunkManager class itself. For convenience, I split the old >>>>>>> "ChunkManager::return_chunks" method into two new ones, >>>>>>> "ChunkManager::return_single_chunk()" which returns a single chunk >>>>>>> to >>>>>>> the ChunkManager without walking the chunk chain, and >>>>>>> "ChunkManger::return_chunk_list()" which returns a chain of chunks >>>>>>> to >>>>>>> the ChunkManager. Both functions are now able to handle both >>>>>>> non-humongous >>>>>>> and humongous chunks. ~SpaceManager() was reimplemented (actually, >>>>>>> quite >>>>>>> simplified) and just calls "ChunkManager::return_chunk_list()" for >>>>>>> all >>>>>>> its chunk lists. >>>>>>> >>>>>>> So now we can remove the public access to the humongous chunk >>>>>>> dictionary >>>>>>> in ChunkManger (ChunkManager::humongous_dictionary()) and make this >>>>>>> function private. >>>>>>> >>>>>>> ---- >>>>>>> >>>>>>> Then there was the fact that the non-humongous chunk lists were also >>>>>>> exposed to public via ChunkManager::free_chunks(). The only reason >>>>>>> was >>>>>>> that >>>>>>> when a VirtualSpaceNode is retired, it accessed the ChunkManager free >>>>>>> lists >>>>>>> to find out the size of the items of the free lists. >>>>>>> >>>>>>> This was solved by adding a new function, >>>>>>> ChunkManager::size_by_index(), >>>>>>> which returns the size of the items of a chunk list given by its >>>>>>> chunk >>>>>>> index. >>>>>>> >>>>>>> So ChunkManager::free_chunks() could be made private too. >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> The rest are minor changes: >>>>>>> >>>>>>> - made ChunkManager::find_free_chunks_list() private because it was >>>>>>> not >>>>>>> used from outside the class >>>>>>> - Fixed an assert in dec_free_chunks_total() >>>>>>> - I added logging (UL, with the tags "gc, metaspace, freelist"). I >>>>>>> tried >>>>>>> to keep the existing logging intact or add useful logging where >>>>>>> possible. >>>>>>> >>>>>>> ---- >>>>>>> >>>>>>> A large chunk of the changes is the added gtests. There is a new >>>>>>> class >>>>>>> now, ChunkManagerReturnTest, which stresses the ChunkManager >>>>>>> take/return >>>>>>> code. >>>>>>> >>>>>>> Note that I dislike the fact that this test is implemented inside >>>>>>> metaspace.cpp itself. For now I kept my new tests like the existing >>>>>>> tests >>>>>>> inside this file, but I think these tests should be moved to >>>>>>> test/native/memory/test_chunkManager.cpp. I suggest doing this in a >>>>>>> separate fix though, because it needs a bit of remodeling (the tests >>>>>>> need >>>>>>> to access internals in metaspace.cpp). >>>>>>> >>>>>>> ---- >>>>>>> >>>>>>> I ran gtests and the jtreg hotspot tests on Linux x64, Solaris sparc >>>>>>> and >>>>>>> AIX. No issues popped up which were associated with my change. >>>>>>> >>>>>>> Thanks for reviewing and Kind Regards, Thomas >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>> > > From dean.long at oracle.com Wed Mar 15 21:28:51 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 15 Mar 2017 14:28:51 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 Message-ID: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8158168 http://cr.openjdk.java.net/~dlong/8158168/ This crash is caused by missing array bounds checks on compact string intrinsics. It shows up when unsynchronized access to a StringBuilder object causes inconsistent field values. To convince myself that all the necessary bounds checks are being done, I put callers into two groups, trusted and untrusted. Untrusted callers are all directed through StringUTF16 methods, so that bounds checks are done in one place and can be tested easily. Trusted callers bypass the bounds checks, so they must do their own checking. As a safety net, I added asserts around the intrinsic calls, and a try/catch that so any out of bounds exception turns into an assert error as well. Finally, I restored some C2 debug code that was previously removed, and I use it to do bounds checking in debug builds. In a product build C2 will remove all of these. See the bug report for tests run. There are some unavoidable performance regressions on micro benchmarks, because now we are doing bounds checks that we weren't before. dl From david.holmes at oracle.com Thu Mar 16 01:19:49 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Mar 2017 11:19:49 +1000 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> Message-ID: Hi Dean, I fixed the cc from core-libs-dev-request at openjdk.java.net to core-libs-dev at openjdk.java.net On 16/03/2017 7:28 AM, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8158168 > > http://cr.openjdk.java.net/~dlong/8158168/ Not a full review sorry, just a couple of comments. src/share/vm/classfile/javaClasses.hpp Given the the only call to java_lang_String::set_debug_intrinsics is within an ifdef, shouldn't the declaration and definition of the method also be guarded the same way? > This crash is caused by missing array bounds checks on compact string > intrinsics. It shows up when unsynchronized access to a StringBuilder > object causes inconsistent field values. > > To convince myself that all the necessary bounds checks are being done, > I put callers into two groups, trusted and untrusted. Untrusted callers > are all directed through StringUTF16 methods, so that bounds checks are > done in one place and can be tested easily. Trusted callers bypass the > bounds checks, so they must do their own checking. The changes to the JDK core classes are quite extensive. This will need rigorous functional and performance testing and it is very late in the release cycle to make these kinds of changes. But I'll leave that to the core-libs folk to comment on. David ----- > As a safety net, I added asserts around the intrinsic calls, and a > try/catch that so any out of bounds exception turns into an assert error > as well. Finally, I restored some C2 debug code that was previously > removed, and I use it to do bounds checking in debug builds. In a > product build C2 will remove all of these. > > See the bug report for tests run. > > There are some unavoidable performance regressions on micro benchmarks, > because now we are doing bounds checks that we weren't before. > > dl > From chris.plummer at oracle.com Thu Mar 16 05:03:11 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 15 Mar 2017 22:03:11 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads Message-ID: Hello, Please review the following: https://bugs.openjdk.java.net/browse/JDK-8176768 http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot While working on 8175342 I noticed our stack size on xgene was 8mb even though I was specifying -Xss72k. It turns out the following code was failing: pthread_attr_setstacksize(&attr, stack_size); Although we computed a minimum stack size of 72k, so -Xss72k should be fine, pthreads on this platform requires the stack be at least 128k, so it failed the pthread_attr_setstacksize() call. The end result is pthread_attr_setstacksize() had no impact on the thread's stack size, and we ended up with the platform default of 8mb. The fix is to round up the following variables to PTHREAD_STACK_MIN after computing their new values: _java_thread_min_stack_allowed _compiler_thread_min_stack_allowed _vm_internal_thread_min_stack_allowed For solaris, there was an issue using PTHREAD_STACK_MIN. You need to #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN #defined, and this needs to be done before including OS header files. I noticed that on solaris we were using thr_min_stack() elsewhere instead of PTHREAD_STACK_MIN, so I decided to do the same with this fix. Either way is ugly (the #define or using thr_min_stack()). And speaking of the existing use of thr_min_stack(), I deleted it. It was being applied before any adjustments to the stack sizes had been made (rounding and adding red, yellow, and shadow zones). This mean the stack ended up being larger than necessary. With the above fix in place, we are now applying thr_min_stack() after recomputing the minimum stack sizes. If for any reason one of those stack sizes is now too small, the correct fix is to adjust the initial stack sizes, not apply thr_min_stack() to the initial stack sizes. However, it looks like no adjustment is needed. I did something close to our nightly testing on all affect platforms, and no new problems turned up. thanks, Chris From david.holmes at oracle.com Thu Mar 16 05:23:10 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Mar 2017 15:23:10 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: Message-ID: Hi Chris, On 16/03/2017 3:03 PM, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8176768 > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot Change looks good. > While working on 8175342 I noticed our stack size on xgene was 8mb even > though I was specifying -Xss72k. It turns out the following code was > failing: > > pthread_attr_setstacksize(&attr, stack_size); So these really should be checking return values, at least in debug builds. But we can leave that until we refactor the thread startup code into os_posix.cpp. Thanks, David ----- > Although we computed a minimum stack size of 72k, so -Xss72k should be > fine, pthreads on this platform requires the stack be at least 128k, so > it failed the pthread_attr_setstacksize() call. The end result is > pthread_attr_setstacksize() had no impact on the thread's stack size, > and we ended up with the platform default of 8mb. The fix is to round up > the following variables to PTHREAD_STACK_MIN after computing their new > values: > > _java_thread_min_stack_allowed > _compiler_thread_min_stack_allowed > _vm_internal_thread_min_stack_allowed > > For solaris, there was an issue using PTHREAD_STACK_MIN. You need to > #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN > #defined, and this needs to be done before including OS header files. I > noticed that on solaris we were using thr_min_stack() elsewhere instead > of PTHREAD_STACK_MIN, so I decided to do the same with this fix. Either > way is ugly (the #define or using thr_min_stack()). > > And speaking of the existing use of thr_min_stack(), I deleted it. It > was being applied before any adjustments to the stack sizes had been > made (rounding and adding red, yellow, and shadow zones). This mean the > stack ended up being larger than necessary. With the above fix in place, > we are now applying thr_min_stack() after recomputing the minimum stack > sizes. If for any reason one of those stack sizes is now too small, the > correct fix is to adjust the initial stack sizes, not apply > thr_min_stack() to the initial stack sizes. However, it looks like no > adjustment is needed. I did something close to our nightly testing on > all affect platforms, and no new problems turned up. > > thanks, > > Chris From chris.plummer at oracle.com Thu Mar 16 05:51:13 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 15 Mar 2017 22:51:13 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: Message-ID: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> On 3/15/17 10:23 PM, David Holmes wrote: > Hi Chris, > > On 16/03/2017 3:03 PM, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8176768 >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot > > Change looks good. > >> While working on 8175342 I noticed our stack size on xgene was 8mb even >> though I was specifying -Xss72k. It turns out the following code was >> failing: >> >> pthread_attr_setstacksize(&attr, stack_size); > > So these really should be checking return values, at least in debug > builds. But we can leave that until we refactor the thread startup > code into os_posix.cpp. I considered adding checks. I wasn't sure if we should abort or just print a warning if it failed. What refactoring is planned? Chris > > Thanks, > David > ----- > >> Although we computed a minimum stack size of 72k, so -Xss72k should be >> fine, pthreads on this platform requires the stack be at least 128k, so >> it failed the pthread_attr_setstacksize() call. The end result is >> pthread_attr_setstacksize() had no impact on the thread's stack size, >> and we ended up with the platform default of 8mb. The fix is to round up >> the following variables to PTHREAD_STACK_MIN after computing their new >> values: >> >> _java_thread_min_stack_allowed >> _compiler_thread_min_stack_allowed >> _vm_internal_thread_min_stack_allowed >> >> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >> #defined, and this needs to be done before including OS header files. I >> noticed that on solaris we were using thr_min_stack() elsewhere instead >> of PTHREAD_STACK_MIN, so I decided to do the same with this fix. Either >> way is ugly (the #define or using thr_min_stack()). >> >> And speaking of the existing use of thr_min_stack(), I deleted it. It >> was being applied before any adjustments to the stack sizes had been >> made (rounding and adding red, yellow, and shadow zones). This mean the >> stack ended up being larger than necessary. With the above fix in place, >> we are now applying thr_min_stack() after recomputing the minimum stack >> sizes. If for any reason one of those stack sizes is now too small, the >> correct fix is to adjust the initial stack sizes, not apply >> thr_min_stack() to the initial stack sizes. However, it looks like no >> adjustment is needed. I did something close to our nightly testing on >> all affect platforms, and no new problems turned up. >> >> thanks, >> >> Chris From david.holmes at oracle.com Thu Mar 16 06:11:10 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Mar 2017 16:11:10 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> Message-ID: <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> On 16/03/2017 3:51 PM, Chris Plummer wrote: > On 3/15/17 10:23 PM, David Holmes wrote: >> Hi Chris, >> >> On 16/03/2017 3:03 PM, Chris Plummer wrote: >>> Hello, >>> >>> Please review the following: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >> >> Change looks good. >> >>> While working on 8175342 I noticed our stack size on xgene was 8mb even >>> though I was specifying -Xss72k. It turns out the following code was >>> failing: >>> >>> pthread_attr_setstacksize(&attr, stack_size); >> >> So these really should be checking return values, at least in debug >> builds. But we can leave that until we refactor the thread startup >> code into os_posix.cpp. > I considered adding checks. I wasn't sure if we should abort or just > print a warning if it failed. When we check pthread lib routines we use: int status = pthread_mutex_lock(_mutex); assert_status(status == 0, status, "mutex_lock"); This is for things that should only fail if we have a programming error. > What refactoring is planned? "Planned" might be a bit strong :) I was thinking of a number of os_posix related cleanups for which issues exist, but also forgot that some of our general clean-up RFE's have been closed as WNF :( I may do some of them after hours anyway :) David ----- > Chris >> >> Thanks, >> David >> ----- >> >>> Although we computed a minimum stack size of 72k, so -Xss72k should be >>> fine, pthreads on this platform requires the stack be at least 128k, so >>> it failed the pthread_attr_setstacksize() call. The end result is >>> pthread_attr_setstacksize() had no impact on the thread's stack size, >>> and we ended up with the platform default of 8mb. The fix is to round up >>> the following variables to PTHREAD_STACK_MIN after computing their new >>> values: >>> >>> _java_thread_min_stack_allowed >>> _compiler_thread_min_stack_allowed >>> _vm_internal_thread_min_stack_allowed >>> >>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>> #defined, and this needs to be done before including OS header files. I >>> noticed that on solaris we were using thr_min_stack() elsewhere instead >>> of PTHREAD_STACK_MIN, so I decided to do the same with this fix. Either >>> way is ugly (the #define or using thr_min_stack()). >>> >>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>> was being applied before any adjustments to the stack sizes had been >>> made (rounding and adding red, yellow, and shadow zones). This mean the >>> stack ended up being larger than necessary. With the above fix in place, >>> we are now applying thr_min_stack() after recomputing the minimum stack >>> sizes. If for any reason one of those stack sizes is now too small, the >>> correct fix is to adjust the initial stack sizes, not apply >>> thr_min_stack() to the initial stack sizes. However, it looks like no >>> adjustment is needed. I did something close to our nightly testing on >>> all affect platforms, and no new problems turned up. >>> >>> thanks, >>> >>> Chris > > From chris.plummer at oracle.com Thu Mar 16 06:14:12 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 15 Mar 2017 23:14:12 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> Message-ID: <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> On 3/15/17 11:11 PM, David Holmes wrote: > On 16/03/2017 3:51 PM, Chris Plummer wrote: >> On 3/15/17 10:23 PM, David Holmes wrote: >>> Hi Chris, >>> >>> On 16/03/2017 3:03 PM, Chris Plummer wrote: >>>> Hello, >>>> >>>> Please review the following: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>> >>> Change looks good. >>> >>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>> even >>>> though I was specifying -Xss72k. It turns out the following code was >>>> failing: >>>> >>>> pthread_attr_setstacksize(&attr, stack_size); >>> >>> So these really should be checking return values, at least in debug >>> builds. But we can leave that until we refactor the thread startup >>> code into os_posix.cpp. >> I considered adding checks. I wasn't sure if we should abort or just >> print a warning if it failed. > > When we check pthread lib routines we use: > > int status = pthread_mutex_lock(_mutex); > assert_status(status == 0, status, "mutex_lock"); > > This is for things that should only fail if we have a programming error. Ok, but this is in the launcher, so I'll need to just use the built-in assert(). I'll add that if want. Chris > >> What refactoring is planned? > > "Planned" might be a bit strong :) I was thinking of a number of > os_posix related cleanups for which issues exist, but also forgot that > some of our general clean-up RFE's have been closed as WNF :( I may do > some of them after hours anyway :) > > David > ----- > >> Chris >>> >>> Thanks, >>> David >>> ----- >>> >>>> Although we computed a minimum stack size of 72k, so -Xss72k should be >>>> fine, pthreads on this platform requires the stack be at least >>>> 128k, so >>>> it failed the pthread_attr_setstacksize() call. The end result is >>>> pthread_attr_setstacksize() had no impact on the thread's stack size, >>>> and we ended up with the platform default of 8mb. The fix is to >>>> round up >>>> the following variables to PTHREAD_STACK_MIN after computing their new >>>> values: >>>> >>>> _java_thread_min_stack_allowed >>>> _compiler_thread_min_stack_allowed >>>> _vm_internal_thread_min_stack_allowed >>>> >>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>> #defined, and this needs to be done before including OS header >>>> files. I >>>> noticed that on solaris we were using thr_min_stack() elsewhere >>>> instead >>>> of PTHREAD_STACK_MIN, so I decided to do the same with this fix. >>>> Either >>>> way is ugly (the #define or using thr_min_stack()). >>>> >>>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>>> was being applied before any adjustments to the stack sizes had been >>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>> the >>>> stack ended up being larger than necessary. With the above fix in >>>> place, >>>> we are now applying thr_min_stack() after recomputing the minimum >>>> stack >>>> sizes. If for any reason one of those stack sizes is now too small, >>>> the >>>> correct fix is to adjust the initial stack sizes, not apply >>>> thr_min_stack() to the initial stack sizes. However, it looks like no >>>> adjustment is needed. I did something close to our nightly testing on >>>> all affect platforms, and no new problems turned up. >>>> >>>> thanks, >>>> >>>> Chris >> >> From david.holmes at oracle.com Thu Mar 16 06:18:26 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Mar 2017 16:18:26 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> Message-ID: On 16/03/2017 4:14 PM, Chris Plummer wrote: > On 3/15/17 11:11 PM, David Holmes wrote: >> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>> On 3/15/17 10:23 PM, David Holmes wrote: >>>> Hi Chris, >>>> >>>> On 16/03/2017 3:03 PM, Chris Plummer wrote: >>>>> Hello, >>>>> >>>>> Please review the following: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>> >>>> Change looks good. >>>> >>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>> even >>>>> though I was specifying -Xss72k. It turns out the following code was >>>>> failing: >>>>> >>>>> pthread_attr_setstacksize(&attr, stack_size); >>>> >>>> So these really should be checking return values, at least in debug >>>> builds. But we can leave that until we refactor the thread startup >>>> code into os_posix.cpp. >>> I considered adding checks. I wasn't sure if we should abort or just >>> print a warning if it failed. >> >> When we check pthread lib routines we use: >> >> int status = pthread_mutex_lock(_mutex); >> assert_status(status == 0, status, "mutex_lock"); >> >> This is for things that should only fail if we have a programming error. > Ok, but this is in the launcher, so I'll need to just use the built-in > assert(). I'll add that if want. Oops! I was forgetting that. Need to be consistent with launcher error checking or lack thereof. And ignore refactoring comments - not relevant. David > Chris >> >>> What refactoring is planned? >> >> "Planned" might be a bit strong :) I was thinking of a number of >> os_posix related cleanups for which issues exist, but also forgot that >> some of our general clean-up RFE's have been closed as WNF :( I may do >> some of them after hours anyway :) >> >> David >> ----- >> >>> Chris >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Although we computed a minimum stack size of 72k, so -Xss72k should be >>>>> fine, pthreads on this platform requires the stack be at least >>>>> 128k, so >>>>> it failed the pthread_attr_setstacksize() call. The end result is >>>>> pthread_attr_setstacksize() had no impact on the thread's stack size, >>>>> and we ended up with the platform default of 8mb. The fix is to >>>>> round up >>>>> the following variables to PTHREAD_STACK_MIN after computing their new >>>>> values: >>>>> >>>>> _java_thread_min_stack_allowed >>>>> _compiler_thread_min_stack_allowed >>>>> _vm_internal_thread_min_stack_allowed >>>>> >>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>> #defined, and this needs to be done before including OS header >>>>> files. I >>>>> noticed that on solaris we were using thr_min_stack() elsewhere >>>>> instead >>>>> of PTHREAD_STACK_MIN, so I decided to do the same with this fix. >>>>> Either >>>>> way is ugly (the #define or using thr_min_stack()). >>>>> >>>>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>>>> was being applied before any adjustments to the stack sizes had been >>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>> the >>>>> stack ended up being larger than necessary. With the above fix in >>>>> place, >>>>> we are now applying thr_min_stack() after recomputing the minimum >>>>> stack >>>>> sizes. If for any reason one of those stack sizes is now too small, >>>>> the >>>>> correct fix is to adjust the initial stack sizes, not apply >>>>> thr_min_stack() to the initial stack sizes. However, it looks like no >>>>> adjustment is needed. I did something close to our nightly testing on >>>>> all affect platforms, and no new problems turned up. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>> >>> > From chris.plummer at oracle.com Thu Mar 16 06:33:41 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 15 Mar 2017 23:33:41 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> Message-ID: <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> On 3/15/17 11:18 PM, David Holmes wrote: > On 16/03/2017 4:14 PM, Chris Plummer wrote: >> On 3/15/17 11:11 PM, David Holmes wrote: >>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>>> On 3/15/17 10:23 PM, David Holmes wrote: >>>>> Hi Chris, >>>>> >>>>> On 16/03/2017 3:03 PM, Chris Plummer wrote: >>>>>> Hello, >>>>>> >>>>>> Please review the following: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>> >>>>> >>>>> Change looks good. >>>>> >>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>> even >>>>>> though I was specifying -Xss72k. It turns out the following code was >>>>>> failing: >>>>>> >>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>> >>>>> So these really should be checking return values, at least in debug >>>>> builds. But we can leave that until we refactor the thread startup >>>>> code into os_posix.cpp. >>>> I considered adding checks. I wasn't sure if we should abort or just >>>> print a warning if it failed. >>> >>> When we check pthread lib routines we use: >>> >>> int status = pthread_mutex_lock(_mutex); >>> assert_status(status == 0, status, "mutex_lock"); >>> >>> This is for things that should only fail if we have a programming >>> error. >> Ok, but this is in the launcher, so I'll need to just use the built-in >> assert(). I'll add that if want. > > Oops! I was forgetting that. Need to be consistent with launcher error > checking or lack thereof. And ignore refactoring comments - not relevant. So don't add the error check? > > David > >> Chris >>> >>>> What refactoring is planned? >>> >>> "Planned" might be a bit strong :) I was thinking of a number of >>> os_posix related cleanups for which issues exist, but also forgot that >>> some of our general clean-up RFE's have been closed as WNF :( I may do >>> some of them after hours anyway :) >>> >>> David >>> ----- >>> >>>> Chris >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>> should be >>>>>> fine, pthreads on this platform requires the stack be at least >>>>>> 128k, so >>>>>> it failed the pthread_attr_setstacksize() call. The end result is >>>>>> pthread_attr_setstacksize() had no impact on the thread's stack >>>>>> size, >>>>>> and we ended up with the platform default of 8mb. The fix is to >>>>>> round up >>>>>> the following variables to PTHREAD_STACK_MIN after computing >>>>>> their new >>>>>> values: >>>>>> >>>>>> _java_thread_min_stack_allowed >>>>>> _compiler_thread_min_stack_allowed >>>>>> _vm_internal_thread_min_stack_allowed >>>>>> >>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>>> #defined, and this needs to be done before including OS header >>>>>> files. I >>>>>> noticed that on solaris we were using thr_min_stack() elsewhere >>>>>> instead >>>>>> of PTHREAD_STACK_MIN, so I decided to do the same with this fix. >>>>>> Either >>>>>> way is ugly (the #define or using thr_min_stack()). >>>>>> >>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>> it. It >>>>>> was being applied before any adjustments to the stack sizes had been >>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>> the >>>>>> stack ended up being larger than necessary. With the above fix in >>>>>> place, >>>>>> we are now applying thr_min_stack() after recomputing the minimum >>>>>> stack >>>>>> sizes. If for any reason one of those stack sizes is now too small, >>>>>> the >>>>>> correct fix is to adjust the initial stack sizes, not apply >>>>>> thr_min_stack() to the initial stack sizes. However, it looks >>>>>> like no >>>>>> adjustment is needed. I did something close to our nightly >>>>>> testing on >>>>>> all affect platforms, and no new problems turned up. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>> >>>> >> From ioi.lam at oracle.com Thu Mar 16 06:48:06 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 16 Mar 2017 14:48:06 +0800 Subject: What's the purpose of hashtable::reverse() Message-ID: <58CA3526.2050600@oracle.com> This seems to be used only by CDS dumping, and if I remove the call nothing bad seems to happen. Does anyone know what hashtable::reverse() could possibly achieve? Also, there's another variant that seems to be dead code that no one calls. template void Hashtable::reverse(void* boundary) - Ioi From david.holmes at oracle.com Thu Mar 16 06:50:21 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Mar 2017 16:50:21 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> Message-ID: <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> On 16/03/2017 4:33 PM, Chris Plummer wrote: > On 3/15/17 11:18 PM, David Holmes wrote: >> On 16/03/2017 4:14 PM, Chris Plummer wrote: >>> On 3/15/17 11:11 PM, David Holmes wrote: >>>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>>>> On 3/15/17 10:23 PM, David Holmes wrote: >>>>>> Hi Chris, >>>>>> >>>>>> On 16/03/2017 3:03 PM, Chris Plummer wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review the following: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>> >>>>>> >>>>>> Change looks good. >>>>>> >>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>> even >>>>>>> though I was specifying -Xss72k. It turns out the following code was >>>>>>> failing: >>>>>>> >>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>> >>>>>> So these really should be checking return values, at least in debug >>>>>> builds. But we can leave that until we refactor the thread startup >>>>>> code into os_posix.cpp. >>>>> I considered adding checks. I wasn't sure if we should abort or just >>>>> print a warning if it failed. >>>> >>>> When we check pthread lib routines we use: >>>> >>>> int status = pthread_mutex_lock(_mutex); >>>> assert_status(status == 0, status, "mutex_lock"); >>>> >>>> This is for things that should only fail if we have a programming >>>> error. >>> Ok, but this is in the launcher, so I'll need to just use the built-in >>> assert(). I'll add that if want. >> >> Oops! I was forgetting that. Need to be consistent with launcher error >> checking or lack thereof. And ignore refactoring comments - not relevant. > So don't add the error check? Given there is no error checking, or assertions, in those files I reluctantly have to say leave it out. Thanks, David ----- >> >> David >> >>> Chris >>>> >>>>> What refactoring is planned? >>>> >>>> "Planned" might be a bit strong :) I was thinking of a number of >>>> os_posix related cleanups for which issues exist, but also forgot that >>>> some of our general clean-up RFE's have been closed as WNF :( I may do >>>> some of them after hours anyway :) >>>> >>>> David >>>> ----- >>>> >>>>> Chris >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>> should be >>>>>>> fine, pthreads on this platform requires the stack be at least >>>>>>> 128k, so >>>>>>> it failed the pthread_attr_setstacksize() call. The end result is >>>>>>> pthread_attr_setstacksize() had no impact on the thread's stack >>>>>>> size, >>>>>>> and we ended up with the platform default of 8mb. The fix is to >>>>>>> round up >>>>>>> the following variables to PTHREAD_STACK_MIN after computing >>>>>>> their new >>>>>>> values: >>>>>>> >>>>>>> _java_thread_min_stack_allowed >>>>>>> _compiler_thread_min_stack_allowed >>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>> >>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>>>> #defined, and this needs to be done before including OS header >>>>>>> files. I >>>>>>> noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>> instead >>>>>>> of PTHREAD_STACK_MIN, so I decided to do the same with this fix. >>>>>>> Either >>>>>>> way is ugly (the #define or using thr_min_stack()). >>>>>>> >>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>> it. It >>>>>>> was being applied before any adjustments to the stack sizes had been >>>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>>> the >>>>>>> stack ended up being larger than necessary. With the above fix in >>>>>>> place, >>>>>>> we are now applying thr_min_stack() after recomputing the minimum >>>>>>> stack >>>>>>> sizes. If for any reason one of those stack sizes is now too small, >>>>>>> the >>>>>>> correct fix is to adjust the initial stack sizes, not apply >>>>>>> thr_min_stack() to the initial stack sizes. However, it looks >>>>>>> like no >>>>>>> adjustment is needed. I did something close to our nightly >>>>>>> testing on >>>>>>> all affect platforms, and no new problems turned up. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>> >>>>> >>> > From thomas.stuefe at gmail.com Thu Mar 16 08:30:38 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 16 Mar 2017 09:30:38 +0100 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> Message-ID: Hi Chris, David, the change looks good. I see that in the launcher we require a minimum stack size across all platforms ("STACK_SIZE_MINIMUM"), should we do the same fix (adjust for PTHREAD_STACK_MIN) there? I do not understand, why does error checking in the hotspot have to be consistent with the launcher? What prevents us from asserting in the hotspot - or at least print a warning? Note that in the hotspot, there is already UL logging ("os", "thread") after pthread_create() in the platform files, so the least we could do is add a warning log output case ppthread_attr_setstacksize fails. If we ever refactor this coding, could we rename the variables holding the base stack size requirement for java frames - in all its incarnations in all the os_cpu files - to be renamed to something different? It is a bit confusing to have a variable which at different times in VM life means different things (before and after the call to os::Posix::set_minimum_stack_sizes()). Or, at least, rename "set_minimum_stack_sizes" to something like "adjust_minimum_stack_sizes" which makes the intent clearer. Kind Regards, Thomas On Thu, Mar 16, 2017 at 7:50 AM, David Holmes wrote: > On 16/03/2017 4:33 PM, Chris Plummer wrote: > >> On 3/15/17 11:18 PM, David Holmes wrote: >> >>> On 16/03/2017 4:14 PM, Chris Plummer wrote: >>> >>>> On 3/15/17 11:11 PM, David Holmes wrote: >>>> >>>>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>>>> >>>>>> On 3/15/17 10:23 PM, David Holmes wrote: >>>>>> >>>>>>> Hi Chris, >>>>>>> >>>>>>> On 16/03/2017 3:03 PM, Chris Plummer wrote: >>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the following: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webr >>>>>>>> ev.hotspot >>>>>>>> >>>>>>>> >>>>>>> Change looks good. >>>>>>> >>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>>> even >>>>>>>> though I was specifying -Xss72k. It turns out the following code was >>>>>>>> failing: >>>>>>>> >>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>> >>>>>>> >>>>>>> So these really should be checking return values, at least in debug >>>>>>> builds. But we can leave that until we refactor the thread startup >>>>>>> code into os_posix.cpp. >>>>>>> >>>>>> I considered adding checks. I wasn't sure if we should abort or just >>>>>> print a warning if it failed. >>>>>> >>>>> >>>>> When we check pthread lib routines we use: >>>>> >>>>> int status = pthread_mutex_lock(_mutex); >>>>> assert_status(status == 0, status, "mutex_lock"); >>>>> >>>>> This is for things that should only fail if we have a programming >>>>> error. >>>>> >>>> Ok, but this is in the launcher, so I'll need to just use the built-in >>>> assert(). I'll add that if want. >>>> >>> >>> Oops! I was forgetting that. Need to be consistent with launcher error >>> checking or lack thereof. And ignore refactoring comments - not relevant. >>> >> So don't add the error check? >> > > Given there is no error checking, or assertions, in those files I > reluctantly have to say leave it out. > > Thanks, > David > ----- > > > >>> David >>> >>> Chris >>>> >>>>> >>>>> What refactoring is planned? >>>>>> >>>>> >>>>> "Planned" might be a bit strong :) I was thinking of a number of >>>>> os_posix related cleanups for which issues exist, but also forgot that >>>>> some of our general clean-up RFE's have been closed as WNF :( I may do >>>>> some of them after hours anyway :) >>>>> >>>>> David >>>>> ----- >>>>> >>>>> Chris >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>> should be >>>>>>>> fine, pthreads on this platform requires the stack be at least >>>>>>>> 128k, so >>>>>>>> it failed the pthread_attr_setstacksize() call. The end result is >>>>>>>> pthread_attr_setstacksize() had no impact on the thread's stack >>>>>>>> size, >>>>>>>> and we ended up with the platform default of 8mb. The fix is to >>>>>>>> round up >>>>>>>> the following variables to PTHREAD_STACK_MIN after computing >>>>>>>> their new >>>>>>>> values: >>>>>>>> >>>>>>>> _java_thread_min_stack_allowed >>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>> >>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>> files. I >>>>>>>> noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>> instead >>>>>>>> of PTHREAD_STACK_MIN, so I decided to do the same with this fix. >>>>>>>> Either >>>>>>>> way is ugly (the #define or using thr_min_stack()). >>>>>>>> >>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>> it. It >>>>>>>> was being applied before any adjustments to the stack sizes had been >>>>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>>>> the >>>>>>>> stack ended up being larger than necessary. With the above fix in >>>>>>>> place, >>>>>>>> we are now applying thr_min_stack() after recomputing the minimum >>>>>>>> stack >>>>>>>> sizes. If for any reason one of those stack sizes is now too small, >>>>>>>> the >>>>>>>> correct fix is to adjust the initial stack sizes, not apply >>>>>>>> thr_min_stack() to the initial stack sizes. However, it looks >>>>>>>> like no >>>>>>>> adjustment is needed. I did something close to our nightly >>>>>>>> testing on >>>>>>>> all affect platforms, and no new problems turned up. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>> >> From david.holmes at oracle.com Thu Mar 16 09:16:53 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Mar 2017 19:16:53 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> Message-ID: <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> On 16/03/2017 6:30 PM, Thomas St?fe wrote: > Hi Chris, David, > > the change looks good. > > I see that in the launcher we require a minimum stack size across all > platforms ("STACK_SIZE_MINIMUM"), should we do the same fix (adjust for > PTHREAD_STACK_MIN) there? > > I do not understand, why does error checking in the hotspot have to be > consistent with the launcher? What prevents us from asserting in the > hotspot - or at least print a warning? Note that in the hotspot, there > is already UL logging ("os", "thread") after pthread_create() in the > platform files, so the least we could do is add a warning log output > case ppthread_attr_setstacksize fails. Sorry I'm getting this group of bugs all muddled up. Chris: this issue does affect hotspot and the launcher (potentially). Ideally both should be checking for failures in the pthread calls but neither do so. Hotspot at least does so in some places but not in a lot of others. pthread_create is different in hotspot because failure can happen easily and we need to detect it and report it (via an exception and also via UL). The other pthread calls are not expected to fail under "normal" conditions but only due to a programming error. Those calls should at least be checked in debug builds as we already do in places with assert_status. The launcher code doesn't do any error checking at all (but again pthread_create is a special case). David ----- > If we ever refactor this coding, could we rename the variables holding > the base stack size requirement for java frames - in all its > incarnations in all the os_cpu files - to be renamed to something > different? It is a bit confusing to have a variable which at different > times in VM life means different things (before and after the call > to os::Posix::set_minimum_stack_sizes()). Or, at least, rename > "set_minimum_stack_sizes" to something like "adjust_minimum_stack_sizes" > which makes the intent clearer. > > Kind Regards, Thomas > > On Thu, Mar 16, 2017 at 7:50 AM, David Holmes > wrote: > > On 16/03/2017 4:33 PM, Chris Plummer wrote: > > On 3/15/17 11:18 PM, David Holmes wrote: > > On 16/03/2017 4:14 PM, Chris Plummer wrote: > > On 3/15/17 11:11 PM, David Holmes wrote: > > On 16/03/2017 3:51 PM, Chris Plummer wrote: > > On 3/15/17 10:23 PM, David Holmes wrote: > > Hi Chris, > > On 16/03/2017 3:03 PM, Chris Plummer wrote: > > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8176768 > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot > > > > Change looks good. > > While working on 8175342 I noticed our > stack size on xgene was 8mb > even > though I was specifying -Xss72k. It > turns out the following code was > failing: > > pthread_attr_setstacksize(&attr, > stack_size); > > > So these really should be checking return > values, at least in debug > builds. But we can leave that until we > refactor the thread startup > code into os_posix.cpp. > > I considered adding checks. I wasn't sure if we > should abort or just > print a warning if it failed. > > > When we check pthread lib routines we use: > > int status = pthread_mutex_lock(_mutex); > assert_status(status == 0, status, "mutex_lock"); > > This is for things that should only fail if we have > a programming > error. > > Ok, but this is in the launcher, so I'll need to just > use the built-in > assert(). I'll add that if want. > > > Oops! I was forgetting that. Need to be consistent with > launcher error > checking or lack thereof. And ignore refactoring comments - > not relevant. > > So don't add the error check? > > > Given there is no error checking, or assertions, in those files I > reluctantly have to say leave it out. > > Thanks, > David > ----- > > > > David > > Chris > > > What refactoring is planned? > > > "Planned" might be a bit strong :) I was thinking of > a number of > os_posix related cleanups for which issues exist, > but also forgot that > some of our general clean-up RFE's have been closed > as WNF :( I may do > some of them after hours anyway :) > > David > ----- > > Chris > > > Thanks, > David > ----- > > Although we computed a minimum stack > size of 72k, so -Xss72k > should be > fine, pthreads on this platform requires > the stack be at least > 128k, so > it failed the > pthread_attr_setstacksize() call. The > end result is > pthread_attr_setstacksize() had no > impact on the thread's stack > size, > and we ended up with the platform > default of 8mb. The fix is to > round up > the following variables to > PTHREAD_STACK_MIN after computing > their new > values: > > _java_thread_min_stack_allowed > _compiler_thread_min_stack_allowed > _vm_internal_thread_min_stack_allowed > > For solaris, there was an issue using > PTHREAD_STACK_MIN. You need to > #define _POSIX_C_SOURCE >= 199506L in > order to get PTHREAD_STACK_MIN > #defined, and this needs to be done > before including OS header > files. I > noticed that on solaris we were using > thr_min_stack() elsewhere > instead > of PTHREAD_STACK_MIN, so I decided to do > the same with this fix. > Either > way is ugly (the #define or using > thr_min_stack()). > > And speaking of the existing use of > thr_min_stack(), I deleted > it. It > was being applied before any adjustments > to the stack sizes had been > made (rounding and adding red, yellow, > and shadow zones). This mean > the > stack ended up being larger than > necessary. With the above fix in > place, > we are now applying thr_min_stack() > after recomputing the minimum > stack > sizes. If for any reason one of those > stack sizes is now too small, > the > correct fix is to adjust the initial > stack sizes, not apply > thr_min_stack() to the initial stack > sizes. However, it looks > like no > adjustment is needed. I did something > close to our nightly > testing on > all affect platforms, and no new > problems turned up. > > thanks, > > Chris > > > > > > From mikael.gerdin at oracle.com Thu Mar 16 09:17:19 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 16 Mar 2017 10:17:19 +0100 Subject: What's the purpose of hashtable::reverse() In-Reply-To: <58CA3526.2050600@oracle.com> References: <58CA3526.2050600@oracle.com> Message-ID: <6b5987a7-e2c4-9990-ba1c-e207393f07b5@oracle.com> Hi Ioi, I found this old bug where I think the reversing was added: https://bugs.openjdk.java.net/browse/JDK-4994483 /Mikael On 2017-03-16 07:48, Ioi Lam wrote: > This seems to be used only by CDS dumping, and if I remove the call > nothing bad seems to happen. Does anyone know what hashtable::reverse() > could possibly achieve? > > Also, there's another variant that seems to be dead code that no one calls. > > template void Hashtable::reverse(void* > boundary) > > - Ioi > > From tobias.hartmann at oracle.com Thu Mar 16 09:52:37 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 16 Mar 2017 10:52:37 +0100 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> Message-ID: <6791984c-2e58-2dd6-5c9d-023f2dad5861@oracle.com> Hi Dean, On 15.03.2017 22:28, dean.long at oracle.com wrote: > http://cr.openjdk.java.net/~dlong/8158168/ > > This crash is caused by missing array bounds checks on compact string intrinsics. It shows up when unsynchronized access to a StringBuilder object causes inconsistent field values. > > To convince myself that all the necessary bounds checks are being done, I put callers into two groups, trusted and untrusted. Untrusted callers are all directed through StringUTF16 methods, so that bounds checks are done in one place and can be tested easily. Trusted callers bypass the bounds checks, so they must do their own checking. > > As a safety net, I added asserts around the intrinsic calls, and a try/catch that so any out of bounds exception turns into an assert error as well. So the assert and try/catch are only necessary to catch invalid offsets passed to the C1 intrinsic, right? Interpreted code is safe and the C2 intrinsics have additional guards in debug builds. I'm fine with that but an alternative would be to also have such guards in the C1 intrinsic. For example, one could enable the normal bound checks and add a simple check to Runtime1::throw_range_check_exception() that fails if the throwing method is the C1 intrinsic. Like this, we could avoid the assert, try-catch and DEBUG_INTRINSICS code. > Finally, I restored some C2 debug code that was previously removed, and I use it to do bounds checking in debug builds. In a product build C2 will remove all of these. To clarify: we introduced those bound checks during Compact Strings development for additional verification and removed them before integration: http://hg.openjdk.java.net/jdk9/sandbox/hotspot/rev/8a7f17b4709f It may make sense re-introduce these checks also for other intrinsics in JDK 10 to catch similar problems. I'll look at it with JDK-8156534. I noticed these unused imports in StringUTF16.java: 38 //import static java.lang.String.checkIndex; 39 //import static java.lang.String.checkOffset; 40 //import static java.lang.String.checkBoundsOffCount; Maybe add a comment to the DEBUG_INTRINSICS declaration. Thanks, Tobias From ioi.lam at oracle.com Thu Mar 16 10:20:06 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 16 Mar 2017 18:20:06 +0800 Subject: What's the purpose of hashtable::reverse() In-Reply-To: <6b5987a7-e2c4-9990-ba1c-e207393f07b5@oracle.com> References: <58CA3526.2050600@oracle.com> <6b5987a7-e2c4-9990-ba1c-e207393f07b5@oracle.com> Message-ID: <58CA66D6.7070504@oracle.com> Thanks Mikael for the detective work. The bug comment says: "the String and symbol tables are hashtables where the hashbuckets are linked lists -- in reverse chronological order of there creation. This means that, in some circumstances, some symbols are unnecessarily touched when looking for older most likely needed symbols. This can be addressed by simply reversing the order of the lists before creating the archive." Since JDK9, CDS no longer stores the string and symbol tables using the "regular" hashtable. The only "regular" hashtable use by CDS is the shared dictionary (for name->class lookup), but this table is rehashed at some point, so calling reverse() will no longer has the desired effect of moving popular Klasses (such as java/lang/Object) to the front of a linked-list. So this optimization is irrelevant now. I've created an RFE to remove it. https://bugs.openjdk.java.net/browse/JDK-8176863 Thanks - Ioi On 3/16/17 5:17 PM, Mikael Gerdin wrote: > Hi Ioi, > > I found this old bug where I think the reversing was added: > https://bugs.openjdk.java.net/browse/JDK-4994483 > > /Mikael > > On 2017-03-16 07:48, Ioi Lam wrote: >> This seems to be used only by CDS dumping, and if I remove the call >> nothing bad seems to happen. Does anyone know what hashtable::reverse() >> could possibly achieve? >> >> Also, there's another variant that seems to be dead code that no one >> calls. >> >> template void Hashtable::reverse(void* >> boundary) >> >> - Ioi >> >> From coleen.phillimore at oracle.com Thu Mar 16 15:52:25 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 16 Mar 2017 11:52:25 -0400 Subject: What's the purpose of hashtable::reverse() In-Reply-To: <58CA66D6.7070504@oracle.com> References: <58CA3526.2050600@oracle.com> <6b5987a7-e2c4-9990-ba1c-e207393f07b5@oracle.com> <58CA66D6.7070504@oracle.com> Message-ID: On 3/16/17 6:20 AM, Ioi Lam wrote: > Thanks Mikael for the detective work. The bug comment says: > > "the String and symbol tables are hashtables where the hashbuckets > are linked lists -- in reverse chronological order of there creation. > This > means that, in some circumstances, some symbols are unnecessarily touched > when looking for older most likely needed symbols. This can be > addressed by > simply reversing the order of the lists before creating the archive." > > Since JDK9, CDS no longer stores the string and symbol tables using > the "regular" hashtable. > > The only "regular" hashtable use by CDS is the shared dictionary (for > name->class lookup), but this table is rehashed at some point, so > calling reverse() will no longer has the desired effect of moving > popular Klasses (such as java/lang/Object) to the front of a linked-list. > > So this optimization is irrelevant now. I've created an RFE to remove it. Great, thank you! More optimizations that we have no means of measurement. Coleen > > https://bugs.openjdk.java.net/browse/JDK-8176863 > > Thanks > - Ioi > > On 3/16/17 5:17 PM, Mikael Gerdin wrote: >> Hi Ioi, >> >> I found this old bug where I think the reversing was added: >> https://bugs.openjdk.java.net/browse/JDK-4994483 >> >> /Mikael >> >> On 2017-03-16 07:48, Ioi Lam wrote: >>> This seems to be used only by CDS dumping, and if I remove the call >>> nothing bad seems to happen. Does anyone know what hashtable::reverse() >>> could possibly achieve? >>> >>> Also, there's another variant that seems to be dead code that no one >>> calls. >>> >>> template void Hashtable::reverse(void* >>> boundary) >>> >>> - Ioi >>> >>> > From mikael.gerdin at oracle.com Thu Mar 16 16:13:31 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Thu, 16 Mar 2017 17:13:31 +0100 Subject: What's the purpose of hashtable::reverse() In-Reply-To: References: <58CA3526.2050600@oracle.com> <6b5987a7-e2c4-9990-ba1c-e207393f07b5@oracle.com> <58CA66D6.7070504@oracle.com> Message-ID: Hi Coleen, On 2017-03-16 16:52, coleen.phillimore at oracle.com wrote: > > > On 3/16/17 6:20 AM, Ioi Lam wrote: >> Thanks Mikael for the detective work. The bug comment says: >> >> "the String and symbol tables are hashtables where the hashbuckets >> are linked lists -- in reverse chronological order of there creation. >> This >> means that, in some circumstances, some symbols are unnecessarily touched >> when looking for older most likely needed symbols. This can be >> addressed by >> simply reversing the order of the lists before creating the archive." >> >> Since JDK9, CDS no longer stores the string and symbol tables using >> the "regular" hashtable. >> >> The only "regular" hashtable use by CDS is the shared dictionary (for >> name->class lookup), but this table is rehashed at some point, so >> calling reverse() will no longer has the desired effect of moving >> popular Klasses (such as java/lang/Object) to the front of a linked-list. >> >> So this optimization is irrelevant now. I've created an RFE to remove it. > > Great, thank you! > > More optimizations that we have no means of measurement. That was stated in the evaluation of this issue some 13 years ago (time flies ;) > ###@###.### 2004-02-13 > The measured performance benefit of these changes is less than expected and > it will be difficult to justify changes to tiger this late without better > numbers. /Mikael > > Coleen >> >> https://bugs.openjdk.java.net/browse/JDK-8176863 >> >> Thanks >> - Ioi >> >> On 3/16/17 5:17 PM, Mikael Gerdin wrote: >>> Hi Ioi, >>> >>> I found this old bug where I think the reversing was added: >>> https://bugs.openjdk.java.net/browse/JDK-4994483 >>> >>> /Mikael >>> >>> On 2017-03-16 07:48, Ioi Lam wrote: >>>> This seems to be used only by CDS dumping, and if I remove the call >>>> nothing bad seems to happen. Does anyone know what hashtable::reverse() >>>> could possibly achieve? >>>> >>>> Also, there's another variant that seems to be dead code that no one >>>> calls. >>>> >>>> template void Hashtable::reverse(void* >>>> boundary) >>>> >>>> - Ioi >>>> >>>> >> > From aph at redhat.com Thu Mar 16 16:50:10 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 16 Mar 2017 16:50:10 +0000 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> Message-ID: <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> I've just noticed a nasty problem. "java -version" on AArch64 gives no clue about which version of HotSpot, Oracle's or the aarch64-port project, is running. I hadn't realized that the version wasn't going to appear anywhere. And all that a crash says is # SIGSEGV (0xb) at pc=0x0000007fb7a0c2e4, pid=44736, tid=44737 # # JRE version: (9.0) (build ) # Java VM: OpenJDK 64-Bit Server VM (9-internal+0-adhoc.aph.hs, mixed mode, tiered, compressed oops, g1 gc, linux-aarch64) There is a bit more in the log: --------------- S U M M A R Y ------------ Command Line: Host: AArch64 Processor rev 0 (aarch64), 48 cores, 62G, Random Linux Distro I guess the Oracle proprietary release will have the Oracle copyright etc., so that one wil be clear enough, and if it's from a Linux distro we'll know which port they use, unless some distro (Gentoo? Debian?) is crazy enough to package both versions, in which case it's their problem. I had assumed that Oracle's port would call itself "arm64" or somesuch, to distinguish it. Even the bug database only has "aarch32" and "aarch64", so it's going to be crazy hard to distinguish which port has a bug. We should really get that fixed. It's funny how this stuff comes out of the woodwork. Andrew. From chris.plummer at oracle.com Thu Mar 16 17:49:49 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 16 Mar 2017 10:49:49 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> Message-ID: <2ba3095a-0bff-68af-2d77-299b7edf35b1@oracle.com> On 3/16/17 2:16 AM, David Holmes wrote: > On 16/03/2017 6:30 PM, Thomas St?fe wrote: >> Hi Chris, David, >> >> the change looks good. >> >> I see that in the launcher we require a minimum stack size across all >> platforms ("STACK_SIZE_MINIMUM"), should we do the same fix (adjust for >> PTHREAD_STACK_MIN) there? >> >> I do not understand, why does error checking in the hotspot have to be >> consistent with the launcher? What prevents us from asserting in the >> hotspot - or at least print a warning? Note that in the hotspot, there >> is already UL logging ("os", "thread") after pthread_create() in the >> platform files, so the least we could do is add a warning log output >> case ppthread_attr_setstacksize fails. > > Sorry I'm getting this group of bugs all muddled up. > > Chris: this issue does affect hotspot and the launcher (potentially). > > Ideally both should be checking for failures in the pthread calls but > neither do so. Hotspot at least does so in some places but not in a > lot of others. > > pthread_create is different in hotspot because failure can happen > easily and we need to detect it and report it (via an exception and > also via UL). The other pthread calls are not expected to fail under > "normal" conditions but only due to a programming error. Those calls > should at least be checked in debug builds as we already do in places > with assert_status. > > The launcher code doesn't do any error checking at all (but again > pthread_create is a special case). Are you just referring to the pthread related error checking? It does do other error checking. Chris > > David > ----- > >> If we ever refactor this coding, could we rename the variables holding >> the base stack size requirement for java frames - in all its >> incarnations in all the os_cpu files - to be renamed to something >> different? It is a bit confusing to have a variable which at different >> times in VM life means different things (before and after the call >> to os::Posix::set_minimum_stack_sizes()). Or, at least, rename >> "set_minimum_stack_sizes" to something like "adjust_minimum_stack_sizes" >> which makes the intent clearer. >> >> Kind Regards, Thomas >> >> On Thu, Mar 16, 2017 at 7:50 AM, David Holmes > > wrote: >> >> On 16/03/2017 4:33 PM, Chris Plummer wrote: >> >> On 3/15/17 11:18 PM, David Holmes wrote: >> >> On 16/03/2017 4:14 PM, Chris Plummer wrote: >> >> On 3/15/17 11:11 PM, David Holmes wrote: >> >> On 16/03/2017 3:51 PM, Chris Plummer wrote: >> >> On 3/15/17 10:23 PM, David Holmes wrote: >> >> Hi Chris, >> >> On 16/03/2017 3:03 PM, Chris Plummer wrote: >> >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8176768 >> >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >> >> >> >> Change looks good. >> >> While working on 8175342 I noticed our >> stack size on xgene was 8mb >> even >> though I was specifying -Xss72k. It >> turns out the following code was >> failing: >> >> pthread_attr_setstacksize(&attr, >> stack_size); >> >> >> So these really should be checking return >> values, at least in debug >> builds. But we can leave that until we >> refactor the thread startup >> code into os_posix.cpp. >> >> I considered adding checks. I wasn't sure if we >> should abort or just >> print a warning if it failed. >> >> >> When we check pthread lib routines we use: >> >> int status = pthread_mutex_lock(_mutex); >> assert_status(status == 0, status, "mutex_lock"); >> >> This is for things that should only fail if we have >> a programming >> error. >> >> Ok, but this is in the launcher, so I'll need to just >> use the built-in >> assert(). I'll add that if want. >> >> >> Oops! I was forgetting that. Need to be consistent with >> launcher error >> checking or lack thereof. And ignore refactoring comments - >> not relevant. >> >> So don't add the error check? >> >> >> Given there is no error checking, or assertions, in those files I >> reluctantly have to say leave it out. >> >> Thanks, >> David >> ----- >> >> >> >> David >> >> Chris >> >> >> What refactoring is planned? >> >> >> "Planned" might be a bit strong :) I was thinking of >> a number of >> os_posix related cleanups for which issues exist, >> but also forgot that >> some of our general clean-up RFE's have been closed >> as WNF :( I may do >> some of them after hours anyway :) >> >> David >> ----- >> >> Chris >> >> >> Thanks, >> David >> ----- >> >> Although we computed a minimum stack >> size of 72k, so -Xss72k >> should be >> fine, pthreads on this platform requires >> the stack be at least >> 128k, so >> it failed the >> pthread_attr_setstacksize() call. The >> end result is >> pthread_attr_setstacksize() had no >> impact on the thread's stack >> size, >> and we ended up with the platform >> default of 8mb. The fix is to >> round up >> the following variables to >> PTHREAD_STACK_MIN after computing >> their new >> values: >> >> _java_thread_min_stack_allowed >> _compiler_thread_min_stack_allowed >> _vm_internal_thread_min_stack_allowed >> >> For solaris, there was an issue using >> PTHREAD_STACK_MIN. You need to >> #define _POSIX_C_SOURCE >= 199506L in >> order to get PTHREAD_STACK_MIN >> #defined, and this needs to be done >> before including OS header >> files. I >> noticed that on solaris we were using >> thr_min_stack() elsewhere >> instead >> of PTHREAD_STACK_MIN, so I decided to do >> the same with this fix. >> Either >> way is ugly (the #define or using >> thr_min_stack()). >> >> And speaking of the existing use of >> thr_min_stack(), I deleted >> it. It >> was being applied before any adjustments >> to the stack sizes had been >> made (rounding and adding red, yellow, >> and shadow zones). This mean >> the >> stack ended up being larger than >> necessary. With the above fix in >> place, >> we are now applying thr_min_stack() >> after recomputing the minimum >> stack >> sizes. If for any reason one of those >> stack sizes is now too small, >> the >> correct fix is to adjust the initial >> stack sizes, not apply >> thr_min_stack() to the initial stack >> sizes. However, it looks >> like no >> adjustment is needed. I did something >> close to our nightly >> testing on >> all affect platforms, and no new >> problems turned up. >> >> thanks, >> >> Chris >> >> >> >> >> >> From bob.vandette at oracle.com Thu Mar 16 18:03:55 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 16 Mar 2017 14:03:55 -0400 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> Message-ID: I agree that this is an issue but I?m not sure that it?s a show stopper. The Oracle build will not have OpenJDK in the version string which will help to differentiate our binaries from OpenJDK builds. The bug database field that I think you are describing is only an indication of the architecture that a bug can be reproduced on. It is not meant to describe the sources that were used to produce the binaries or where the binaries came from. That should to be specified elsewhere in the bug report. I don?t like the idea of listing arm64 in the version string since we are only using arm64 internally to trigger the use of the hotspot ?arm? directory. We?d also end up with lots of incorrect bug entries since folks will fail to use arm64 to report a bug in the Oracle 64-bit ARM port running on an aarch64 based system. If we start putting build configuration information in the version string, then where do we stop. Bob. > On Mar 16, 2017, at 12:50 PM, Andrew Haley wrote: > > I've just noticed a nasty problem. "java -version" on AArch64 gives > no clue about which version of HotSpot, Oracle's or the aarch64-port > project, is running. I hadn't realized that the version wasn't going > to appear anywhere. > > And all that a crash says is > > # SIGSEGV (0xb) at pc=0x0000007fb7a0c2e4, pid=44736, tid=44737 > # > # JRE version: (9.0) (build ) > # Java VM: OpenJDK 64-Bit Server VM (9-internal+0-adhoc.aph.hs, mixed mode, tiered, compressed oops, g1 gc, linux-aarch64) > > There is a bit more in the log: > > --------------- S U M M A R Y ------------ > Command Line: > Host: AArch64 Processor rev 0 (aarch64), 48 cores, 62G, Random Linux Distro > > I guess the Oracle proprietary release will have the Oracle copyright > etc., so that one wil be clear enough, and if it's from a Linux distro > we'll know which port they use, unless some distro (Gentoo? Debian?) > is crazy enough to package both versions, in which case it's their > problem. > > I had assumed that Oracle's port would call itself "arm64" or > somesuch, to distinguish it. > > Even the bug database only has "aarch32" and "aarch64", so it's going > to be crazy hard to distinguish which port has a bug. We should > really get that fixed. > > It's funny how this stuff comes out of the woodwork. > > Andrew. From vladimir.x.ivanov at oracle.com Thu Mar 16 18:09:01 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 16 Mar 2017 21:09:01 +0300 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> Message-ID: <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> > The changes to the JDK core classes are quite extensive. This will need > rigorous functional and performance testing and it is very late in the > release cycle to make these kinds of changes. But I'll leave that to the > core-libs folk to comment on. I have the same concern. Can we fix the immediate problem in 9 and integrate verification logic in 10? Best regards, Vladimir Ivanov From aph at redhat.com Thu Mar 16 18:27:49 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 16 Mar 2017 18:27:49 +0000 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> Message-ID: <642003bd-a5dc-d1ae-87ff-aca50388cb9c@redhat.com> On 16/03/17 18:03, Bob Vandette wrote: > I agree that this is an issue but I?m not sure that it?s a show > stopper. > > The Oracle build will not have OpenJDK in the version string which > will help to differentiate our binaries from OpenJDK builds. Right, like I said. > The bug database field that I think you are describing is only an > indication of the architecture that a bug can be reproduced on. It > is not meant to describe the sources that were used to produce the > binaries or where the binaries came from. That should to be > specified elsewhere in the bug report. OK. I would surely have tried to insist that the version strings were different for our two ports at the time your port was committed, but I blew my chance. > I don?t like the idea of listing arm64 in the version string since > we are only using arm64 internally to trigger the use of the hotspot > ?arm? directory. We?d also end up with lots of incorrect bug > entries since folks will fail to use arm64 to report a bug in the > Oracle 64-bit ARM port running on an aarch64 based system. > > If we start putting build configuration information in the version > string, then where do we stop. It's going to be rather horrible, though. How do we reproduce a bug if we don't know what port is causing the bug? How do we even ask the question if we don't know what the ports are called? I always assumed we were "aarch64" and you were "arm64". How are we to ask a user if we can't tell them what to look for? Even if we don't change anything in OpenJDK itself, we'll still have to agree on a label to use in the bug database. I don't know what labels we should use, but we should agree on them now. Do you have any preferences? Andrew. From bob.vandette at oracle.com Thu Mar 16 18:40:00 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 16 Mar 2017 14:40:00 -0400 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: <642003bd-a5dc-d1ae-87ff-aca50388cb9c@redhat.com> References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <642003bd-a5dc-d1ae-87ff-aca50388cb9c@redhat.com> Message-ID: <4E700D62-8D04-47F7-9298-D1ED776BE376@oracle.com> > On Mar 16, 2017, at 2:27 PM, Andrew Haley wrote: > > On 16/03/17 18:03, Bob Vandette wrote: > >> I agree that this is an issue but I?m not sure that it?s a show >> stopper. >> >> The Oracle build will not have OpenJDK in the version string which >> will help to differentiate our binaries from OpenJDK builds. > > Right, like I said. > >> The bug database field that I think you are describing is only an >> indication of the architecture that a bug can be reproduced on. It >> is not meant to describe the sources that were used to produce the >> binaries or where the binaries came from. That should to be >> specified elsewhere in the bug report. > > OK. I would surely have tried to insist that the version strings were > different for our two ports at the time your port was committed, but I > blew my chance. > >> I don?t like the idea of listing arm64 in the version string since >> we are only using arm64 internally to trigger the use of the hotspot >> ?arm? directory. We?d also end up with lots of incorrect bug >> entries since folks will fail to use arm64 to report a bug in the >> Oracle 64-bit ARM port running on an aarch64 based system. >> >> If we start putting build configuration information in the version >> string, then where do we stop. > > It's going to be rather horrible, though. How do we reproduce a bug > if we don't know what port is causing the bug? How do we even ask the > question if we don't know what the ports are called? I always assumed > we were "aarch64" and you were "arm64". How are we to ask a user if > we can't tell them what to look for? > > Even if we don't change anything in OpenJDK itself, we'll still have > to agree on a label to use in the bug database. I don't know what > labels we should use, but we should agree on them now. Do you have > any preferences? I agree that a label would be very useful. For this purpose, I?m not opposed to using the arm64 versus aarch64 names. Let me check around to see if anyone has a better suggestion. Bob. > > Andrew. From dean.long at oracle.com Thu Mar 16 19:41:35 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 16 Mar 2017 12:41:35 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> Message-ID: On 3/15/17 6:19 PM, David Holmes wrote: > src/share/vm/classfile/javaClasses.hpp > > Given the the only call to java_lang_String::set_debug_intrinsics is > within an ifdef, shouldn't the declaration and definition of the > method also be guarded the same way? OK I'll change it. dl From dean.long at oracle.com Thu Mar 16 20:01:43 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 16 Mar 2017 13:01:43 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <6791984c-2e58-2dd6-5c9d-023f2dad5861@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <6791984c-2e58-2dd6-5c9d-023f2dad5861@oracle.com> Message-ID: <51a494e9-fe2d-4622-a1a0-1c7e69287ace@oracle.com> On 3/16/17 2:52 AM, Tobias Hartmann wrote: >> As a safety net, I added asserts around the intrinsic calls, and a try/catch that so any out of bounds exception turns into an assert error as well. > So the assert and try/catch are only necessary to catch invalid offsets passed to the C1 intrinsic, right? Interpreted code is safe and the C2 intrinsics have additional guards in debug builds. The try/catch turns an exception into an assert for interpreted and C1 debug, and the assert can be used in product builds. dl From dean.long at oracle.com Thu Mar 16 20:08:10 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 16 Mar 2017 13:08:10 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> Message-ID: On 3/16/17 2:52 AM, Tobias Hartmann wrote: >> As a safety net, I added asserts around the intrinsic calls, and a try/catch that so any out of bounds exception turns into an assert error as well. > So the assert and try/catch are only necessary to catch invalid offsets passed to the C1 intrinsic, right? Interpreted code is safe and the C2 intrinsics have additional guards in debug builds. > > > I'm fine with that but an alternative would be to also have such guards in the C1 intrinsic. For example, one could enable the normal bound checks and add a simple check to Runtime1::throw_range_check_exception() that fails if the throwing method is the C1 intrinsic. Like this, we could avoid the assert, try-catch and DEBUG_INTRINSICS code. > On 3/16/17 11:09 AM, Vladimir Ivanov wrote: > >> The changes to the JDK core classes are quite extensive. This will need >> rigorous functional and performance testing and it is very late in the >> release cycle to make these kinds of changes. But I'll leave that to the >> core-libs folk to comment on. > > I have the same concern. Can we fix the immediate problem in 9 and > integrate verification logic in 10? > OK, Tobias is suggesting having verification logic only inside the intrinsics. Are you suggesting removing that as well? I'm OK with removing all the verification, but that won't reduce the library changes much. I could undo the renaming to Trusted.getChar, but we would still have the bounds checks moved into StringUTF16. dl > Best regards, > Vladimir Ivanov From david.holmes at oracle.com Thu Mar 16 21:35:56 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Mar 2017 07:35:56 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <2ba3095a-0bff-68af-2d77-299b7edf35b1@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> <2ba3095a-0bff-68af-2d77-299b7edf35b1@oracle.com> Message-ID: <050560cc-aa81-9da4-1eae-1dab105d3926@oracle.com> On 17/03/2017 3:49 AM, Chris Plummer wrote: > On 3/16/17 2:16 AM, David Holmes wrote: >> On 16/03/2017 6:30 PM, Thomas St?fe wrote: >>> Hi Chris, David, >>> >>> the change looks good. >>> >>> I see that in the launcher we require a minimum stack size across all >>> platforms ("STACK_SIZE_MINIMUM"), should we do the same fix (adjust for >>> PTHREAD_STACK_MIN) there? >>> >>> I do not understand, why does error checking in the hotspot have to be >>> consistent with the launcher? What prevents us from asserting in the >>> hotspot - or at least print a warning? Note that in the hotspot, there >>> is already UL logging ("os", "thread") after pthread_create() in the >>> platform files, so the least we could do is add a warning log output >>> case ppthread_attr_setstacksize fails. >> >> Sorry I'm getting this group of bugs all muddled up. >> >> Chris: this issue does affect hotspot and the launcher (potentially). >> >> Ideally both should be checking for failures in the pthread calls but >> neither do so. Hotspot at least does so in some places but not in a >> lot of others. >> >> pthread_create is different in hotspot because failure can happen >> easily and we need to detect it and report it (via an exception and >> also via UL). The other pthread calls are not expected to fail under >> "normal" conditions but only due to a programming error. Those calls >> should at least be checked in debug builds as we already do in places >> with assert_status. >> >> The launcher code doesn't do any error checking at all (but again >> pthread_create is a special case). > Are you just referring to the pthread related error checking? It does do > other error checking. pthread error checking. So trying to think this through ... If the user specifies a too small, or unaligned-to-page-size, -Xss value the pthread_setstacksize() in the launcher will silently fail and the main thread will get the default stack of 8M. It will then load the VM which will then check the -Xss value, which will do its own validity checking. That seems like quite a reasonable position for the launcher to take. David ----- > > Chris >> >> David >> ----- >> >>> If we ever refactor this coding, could we rename the variables holding >>> the base stack size requirement for java frames - in all its >>> incarnations in all the os_cpu files - to be renamed to something >>> different? It is a bit confusing to have a variable which at different >>> times in VM life means different things (before and after the call >>> to os::Posix::set_minimum_stack_sizes()). Or, at least, rename >>> "set_minimum_stack_sizes" to something like "adjust_minimum_stack_sizes" >>> which makes the intent clearer. >>> >>> Kind Regards, Thomas >>> >>> On Thu, Mar 16, 2017 at 7:50 AM, David Holmes >> > wrote: >>> >>> On 16/03/2017 4:33 PM, Chris Plummer wrote: >>> >>> On 3/15/17 11:18 PM, David Holmes wrote: >>> >>> On 16/03/2017 4:14 PM, Chris Plummer wrote: >>> >>> On 3/15/17 11:11 PM, David Holmes wrote: >>> >>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>> >>> On 3/15/17 10:23 PM, David Holmes wrote: >>> >>> Hi Chris, >>> >>> On 16/03/2017 3:03 PM, Chris Plummer wrote: >>> >>> Hello, >>> >>> Please review the following: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>> >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>> >>> >>> >>> Change looks good. >>> >>> While working on 8175342 I noticed our >>> stack size on xgene was 8mb >>> even >>> though I was specifying -Xss72k. It >>> turns out the following code was >>> failing: >>> >>> pthread_attr_setstacksize(&attr, >>> stack_size); >>> >>> >>> So these really should be checking return >>> values, at least in debug >>> builds. But we can leave that until we >>> refactor the thread startup >>> code into os_posix.cpp. >>> >>> I considered adding checks. I wasn't sure if we >>> should abort or just >>> print a warning if it failed. >>> >>> >>> When we check pthread lib routines we use: >>> >>> int status = pthread_mutex_lock(_mutex); >>> assert_status(status == 0, status, "mutex_lock"); >>> >>> This is for things that should only fail if we have >>> a programming >>> error. >>> >>> Ok, but this is in the launcher, so I'll need to just >>> use the built-in >>> assert(). I'll add that if want. >>> >>> >>> Oops! I was forgetting that. Need to be consistent with >>> launcher error >>> checking or lack thereof. And ignore refactoring comments - >>> not relevant. >>> >>> So don't add the error check? >>> >>> >>> Given there is no error checking, or assertions, in those files I >>> reluctantly have to say leave it out. >>> >>> Thanks, >>> David >>> ----- >>> >>> >>> >>> David >>> >>> Chris >>> >>> >>> What refactoring is planned? >>> >>> >>> "Planned" might be a bit strong :) I was thinking of >>> a number of >>> os_posix related cleanups for which issues exist, >>> but also forgot that >>> some of our general clean-up RFE's have been closed >>> as WNF :( I may do >>> some of them after hours anyway :) >>> >>> David >>> ----- >>> >>> Chris >>> >>> >>> Thanks, >>> David >>> ----- >>> >>> Although we computed a minimum stack >>> size of 72k, so -Xss72k >>> should be >>> fine, pthreads on this platform requires >>> the stack be at least >>> 128k, so >>> it failed the >>> pthread_attr_setstacksize() call. The >>> end result is >>> pthread_attr_setstacksize() had no >>> impact on the thread's stack >>> size, >>> and we ended up with the platform >>> default of 8mb. The fix is to >>> round up >>> the following variables to >>> PTHREAD_STACK_MIN after computing >>> their new >>> values: >>> >>> _java_thread_min_stack_allowed >>> _compiler_thread_min_stack_allowed >>> _vm_internal_thread_min_stack_allowed >>> >>> For solaris, there was an issue using >>> PTHREAD_STACK_MIN. You need to >>> #define _POSIX_C_SOURCE >= 199506L in >>> order to get PTHREAD_STACK_MIN >>> #defined, and this needs to be done >>> before including OS header >>> files. I >>> noticed that on solaris we were using >>> thr_min_stack() elsewhere >>> instead >>> of PTHREAD_STACK_MIN, so I decided to do >>> the same with this fix. >>> Either >>> way is ugly (the #define or using >>> thr_min_stack()). >>> >>> And speaking of the existing use of >>> thr_min_stack(), I deleted >>> it. It >>> was being applied before any adjustments >>> to the stack sizes had been >>> made (rounding and adding red, yellow, >>> and shadow zones). This mean >>> the >>> stack ended up being larger than >>> necessary. With the above fix in >>> place, >>> we are now applying thr_min_stack() >>> after recomputing the minimum >>> stack >>> sizes. If for any reason one of those >>> stack sizes is now too small, >>> the >>> correct fix is to adjust the initial >>> stack sizes, not apply >>> thr_min_stack() to the initial stack >>> sizes. However, it looks >>> like no >>> adjustment is needed. I did something >>> close to our nightly >>> testing on >>> all affect platforms, and no new >>> problems turned up. >>> >>> thanks, >>> >>> Chris >>> >>> >>> >>> >>> >>> > > From chris.plummer at oracle.com Thu Mar 16 21:43:58 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 16 Mar 2017 14:43:58 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <050560cc-aa81-9da4-1eae-1dab105d3926@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> <2ba3095a-0bff-68af-2d77-299b7edf35b1@oracle.com> <050560cc-aa81-9da4-1eae-1dab105d3926@oracle.com> Message-ID: On 3/16/17 2:35 PM, David Holmes wrote: > On 17/03/2017 3:49 AM, Chris Plummer wrote: >> On 3/16/17 2:16 AM, David Holmes wrote: >>> On 16/03/2017 6:30 PM, Thomas St?fe wrote: >>>> Hi Chris, David, >>>> >>>> the change looks good. >>>> >>>> I see that in the launcher we require a minimum stack size across all >>>> platforms ("STACK_SIZE_MINIMUM"), should we do the same fix (adjust >>>> for >>>> PTHREAD_STACK_MIN) there? >>>> >>>> I do not understand, why does error checking in the hotspot have to be >>>> consistent with the launcher? What prevents us from asserting in the >>>> hotspot - or at least print a warning? Note that in the hotspot, there >>>> is already UL logging ("os", "thread") after pthread_create() in the >>>> platform files, so the least we could do is add a warning log output >>>> case ppthread_attr_setstacksize fails. >>> >>> Sorry I'm getting this group of bugs all muddled up. >>> >>> Chris: this issue does affect hotspot and the launcher (potentially). >>> >>> Ideally both should be checking for failures in the pthread calls but >>> neither do so. Hotspot at least does so in some places but not in a >>> lot of others. >>> >>> pthread_create is different in hotspot because failure can happen >>> easily and we need to detect it and report it (via an exception and >>> also via UL). The other pthread calls are not expected to fail under >>> "normal" conditions but only due to a programming error. Those calls >>> should at least be checked in debug builds as we already do in places >>> with assert_status. >>> >>> The launcher code doesn't do any error checking at all (but again >>> pthread_create is a special case). >> Are you just referring to the pthread related error checking? It does do >> other error checking. > > pthread error checking. > > So trying to think this through ... > > If the user specifies a too small, or unaligned-to-page-size, -Xss > value the pthread_setstacksize() in the launcher will silently fail > and the main thread will get the default stack of 8M. It will then > load the VM which will then check the -Xss value, which will do its > own validity checking. > Close, except there is still a potential issue if the size is bigger than the minimum hotspot requires, but is not page size aligned. pthread_setstacksize *could* fail in this case, and there would be no "stack size too small" rejection from the hotspot. However, pthread_setstacksize did not fail on the two platforms I tried unaligned stack sizes on. Chris > That seems like quite a reasonable position for the launcher to take. > > David > ----- > >> >> Chris >>> >>> David >>> ----- >>> >>>> If we ever refactor this coding, could we rename the variables holding >>>> the base stack size requirement for java frames - in all its >>>> incarnations in all the os_cpu files - to be renamed to something >>>> different? It is a bit confusing to have a variable which at different >>>> times in VM life means different things (before and after the call >>>> to os::Posix::set_minimum_stack_sizes()). Or, at least, rename >>>> "set_minimum_stack_sizes" to something like >>>> "adjust_minimum_stack_sizes" >>>> which makes the intent clearer. >>>> >>>> Kind Regards, Thomas >>>> >>>> On Thu, Mar 16, 2017 at 7:50 AM, David Holmes >>> > wrote: >>>> >>>> On 16/03/2017 4:33 PM, Chris Plummer wrote: >>>> >>>> On 3/15/17 11:18 PM, David Holmes wrote: >>>> >>>> On 16/03/2017 4:14 PM, Chris Plummer wrote: >>>> >>>> On 3/15/17 11:11 PM, David Holmes wrote: >>>> >>>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>>> >>>> On 3/15/17 10:23 PM, David Holmes wrote: >>>> >>>> Hi Chris, >>>> >>>> On 16/03/2017 3:03 PM, Chris Plummer >>>> wrote: >>>> >>>> Hello, >>>> >>>> Please review the following: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>> >>>> >>>> >>>> >>>> Change looks good. >>>> >>>> While working on 8175342 I noticed our >>>> stack size on xgene was 8mb >>>> even >>>> though I was specifying -Xss72k. It >>>> turns out the following code was >>>> failing: >>>> >>>> pthread_attr_setstacksize(&attr, >>>> stack_size); >>>> >>>> >>>> So these really should be checking return >>>> values, at least in debug >>>> builds. But we can leave that until we >>>> refactor the thread startup >>>> code into os_posix.cpp. >>>> >>>> I considered adding checks. I wasn't sure >>>> if we >>>> should abort or just >>>> print a warning if it failed. >>>> >>>> >>>> When we check pthread lib routines we use: >>>> >>>> int status = pthread_mutex_lock(_mutex); >>>> assert_status(status == 0, status, >>>> "mutex_lock"); >>>> >>>> This is for things that should only fail if we >>>> have >>>> a programming >>>> error. >>>> >>>> Ok, but this is in the launcher, so I'll need to just >>>> use the built-in >>>> assert(). I'll add that if want. >>>> >>>> >>>> Oops! I was forgetting that. Need to be consistent with >>>> launcher error >>>> checking or lack thereof. And ignore refactoring >>>> comments - >>>> not relevant. >>>> >>>> So don't add the error check? >>>> >>>> >>>> Given there is no error checking, or assertions, in those files I >>>> reluctantly have to say leave it out. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>> >>>> David >>>> >>>> Chris >>>> >>>> >>>> What refactoring is planned? >>>> >>>> >>>> "Planned" might be a bit strong :) I was >>>> thinking of >>>> a number of >>>> os_posix related cleanups for which issues exist, >>>> but also forgot that >>>> some of our general clean-up RFE's have been >>>> closed >>>> as WNF :( I may do >>>> some of them after hours anyway :) >>>> >>>> David >>>> ----- >>>> >>>> Chris >>>> >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> Although we computed a minimum stack >>>> size of 72k, so -Xss72k >>>> should be >>>> fine, pthreads on this platform >>>> requires >>>> the stack be at least >>>> 128k, so >>>> it failed the >>>> pthread_attr_setstacksize() call. The >>>> end result is >>>> pthread_attr_setstacksize() had no >>>> impact on the thread's stack >>>> size, >>>> and we ended up with the platform >>>> default of 8mb. The fix is to >>>> round up >>>> the following variables to >>>> PTHREAD_STACK_MIN after computing >>>> their new >>>> values: >>>> >>>> _java_thread_min_stack_allowed >>>> _compiler_thread_min_stack_allowed >>>> _vm_internal_thread_min_stack_allowed >>>> >>>> For solaris, there was an issue using >>>> PTHREAD_STACK_MIN. You need to >>>> #define _POSIX_C_SOURCE >= 199506L in >>>> order to get PTHREAD_STACK_MIN >>>> #defined, and this needs to be done >>>> before including OS header >>>> files. I >>>> noticed that on solaris we were using >>>> thr_min_stack() elsewhere >>>> instead >>>> of PTHREAD_STACK_MIN, so I decided >>>> to do >>>> the same with this fix. >>>> Either >>>> way is ugly (the #define or using >>>> thr_min_stack()). >>>> >>>> And speaking of the existing use of >>>> thr_min_stack(), I deleted >>>> it. It >>>> was being applied before any >>>> adjustments >>>> to the stack sizes had been >>>> made (rounding and adding red, yellow, >>>> and shadow zones). This mean >>>> the >>>> stack ended up being larger than >>>> necessary. With the above fix in >>>> place, >>>> we are now applying thr_min_stack() >>>> after recomputing the minimum >>>> stack >>>> sizes. If for any reason one of those >>>> stack sizes is now too small, >>>> the >>>> correct fix is to adjust the initial >>>> stack sizes, not apply >>>> thr_min_stack() to the initial stack >>>> sizes. However, it looks >>>> like no >>>> adjustment is needed. I did something >>>> close to our nightly >>>> testing on >>>> all affect platforms, and no new >>>> problems turned up. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> >>>> >>>> >>>> >>>> >> >> From david.holmes at oracle.com Thu Mar 16 22:01:42 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Mar 2017 08:01:42 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> <2ba3095a-0bff-68af-2d77-299b7edf35b1@oracle.com> <050560cc-aa81-9da4-1eae-1dab105d3926@oracle.com> Message-ID: <129b18f4-ea7a-915f-e17c-76b7e96214e8@oracle.com> On 17/03/2017 7:43 AM, Chris Plummer wrote: > On 3/16/17 2:35 PM, David Holmes wrote: >> On 17/03/2017 3:49 AM, Chris Plummer wrote: >>> On 3/16/17 2:16 AM, David Holmes wrote: >>>> On 16/03/2017 6:30 PM, Thomas St?fe wrote: >>>>> Hi Chris, David, >>>>> >>>>> the change looks good. >>>>> >>>>> I see that in the launcher we require a minimum stack size across all >>>>> platforms ("STACK_SIZE_MINIMUM"), should we do the same fix (adjust >>>>> for >>>>> PTHREAD_STACK_MIN) there? >>>>> >>>>> I do not understand, why does error checking in the hotspot have to be >>>>> consistent with the launcher? What prevents us from asserting in the >>>>> hotspot - or at least print a warning? Note that in the hotspot, there >>>>> is already UL logging ("os", "thread") after pthread_create() in the >>>>> platform files, so the least we could do is add a warning log output >>>>> case ppthread_attr_setstacksize fails. >>>> >>>> Sorry I'm getting this group of bugs all muddled up. >>>> >>>> Chris: this issue does affect hotspot and the launcher (potentially). >>>> >>>> Ideally both should be checking for failures in the pthread calls but >>>> neither do so. Hotspot at least does so in some places but not in a >>>> lot of others. >>>> >>>> pthread_create is different in hotspot because failure can happen >>>> easily and we need to detect it and report it (via an exception and >>>> also via UL). The other pthread calls are not expected to fail under >>>> "normal" conditions but only due to a programming error. Those calls >>>> should at least be checked in debug builds as we already do in places >>>> with assert_status. >>>> >>>> The launcher code doesn't do any error checking at all (but again >>>> pthread_create is a special case). >>> Are you just referring to the pthread related error checking? It does do >>> other error checking. >> >> pthread error checking. >> >> So trying to think this through ... >> >> If the user specifies a too small, or unaligned-to-page-size, -Xss >> value the pthread_setstacksize() in the launcher will silently fail >> and the main thread will get the default stack of 8M. It will then >> load the VM which will then check the -Xss value, which will do its >> own validity checking. >> > Close, except there is still a potential issue if the size is bigger > than the minimum hotspot requires, but is not page size aligned. > pthread_setstacksize *could* fail in this case, and there would be no > "stack size too small" rejection from the hotspot. However, > pthread_setstacksize did not fail on the two platforms I tried unaligned > stack sizes on. Perhaps because that is not specified by POSIX. For POSIX we only have: [EINVAL] The value of stacksize is less than {PTHREAD_STACK_MIN} or exceeds a system-imposed limit. Anyway that is a check that hotspot could perform if pthread_attr_setstacksize fails. Though that then makes me wonder if we do any rounding when the stack size set on a per thread basis via the java.lang.Thread constructor? I think imposing the PTHREAD_STACK_MIN in hotspot, with an assert checking pthread_attr_setstacksize succeeded (in hotspot) would suffice here. David ----- > Chris >> That seems like quite a reasonable position for the launcher to take. >> >> David >> ----- >> >>> >>> Chris >>>> >>>> David >>>> ----- >>>> >>>>> If we ever refactor this coding, could we rename the variables holding >>>>> the base stack size requirement for java frames - in all its >>>>> incarnations in all the os_cpu files - to be renamed to something >>>>> different? It is a bit confusing to have a variable which at different >>>>> times in VM life means different things (before and after the call >>>>> to os::Posix::set_minimum_stack_sizes()). Or, at least, rename >>>>> "set_minimum_stack_sizes" to something like >>>>> "adjust_minimum_stack_sizes" >>>>> which makes the intent clearer. >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> On Thu, Mar 16, 2017 at 7:50 AM, David Holmes >>>> > wrote: >>>>> >>>>> On 16/03/2017 4:33 PM, Chris Plummer wrote: >>>>> >>>>> On 3/15/17 11:18 PM, David Holmes wrote: >>>>> >>>>> On 16/03/2017 4:14 PM, Chris Plummer wrote: >>>>> >>>>> On 3/15/17 11:11 PM, David Holmes wrote: >>>>> >>>>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>>>> >>>>> On 3/15/17 10:23 PM, David Holmes wrote: >>>>> >>>>> Hi Chris, >>>>> >>>>> On 16/03/2017 3:03 PM, Chris Plummer >>>>> wrote: >>>>> >>>>> Hello, >>>>> >>>>> Please review the following: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>> >>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>> >>>>> >>>>> >>>>> >>>>> Change looks good. >>>>> >>>>> While working on 8175342 I noticed our >>>>> stack size on xgene was 8mb >>>>> even >>>>> though I was specifying -Xss72k. It >>>>> turns out the following code was >>>>> failing: >>>>> >>>>> pthread_attr_setstacksize(&attr, >>>>> stack_size); >>>>> >>>>> >>>>> So these really should be checking return >>>>> values, at least in debug >>>>> builds. But we can leave that until we >>>>> refactor the thread startup >>>>> code into os_posix.cpp. >>>>> >>>>> I considered adding checks. I wasn't sure >>>>> if we >>>>> should abort or just >>>>> print a warning if it failed. >>>>> >>>>> >>>>> When we check pthread lib routines we use: >>>>> >>>>> int status = pthread_mutex_lock(_mutex); >>>>> assert_status(status == 0, status, >>>>> "mutex_lock"); >>>>> >>>>> This is for things that should only fail if we >>>>> have >>>>> a programming >>>>> error. >>>>> >>>>> Ok, but this is in the launcher, so I'll need to just >>>>> use the built-in >>>>> assert(). I'll add that if want. >>>>> >>>>> >>>>> Oops! I was forgetting that. Need to be consistent with >>>>> launcher error >>>>> checking or lack thereof. And ignore refactoring >>>>> comments - >>>>> not relevant. >>>>> >>>>> So don't add the error check? >>>>> >>>>> >>>>> Given there is no error checking, or assertions, in those files I >>>>> reluctantly have to say leave it out. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> >>>>> >>>>> David >>>>> >>>>> Chris >>>>> >>>>> >>>>> What refactoring is planned? >>>>> >>>>> >>>>> "Planned" might be a bit strong :) I was >>>>> thinking of >>>>> a number of >>>>> os_posix related cleanups for which issues exist, >>>>> but also forgot that >>>>> some of our general clean-up RFE's have been >>>>> closed >>>>> as WNF :( I may do >>>>> some of them after hours anyway :) >>>>> >>>>> David >>>>> ----- >>>>> >>>>> Chris >>>>> >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> Although we computed a minimum stack >>>>> size of 72k, so -Xss72k >>>>> should be >>>>> fine, pthreads on this platform >>>>> requires >>>>> the stack be at least >>>>> 128k, so >>>>> it failed the >>>>> pthread_attr_setstacksize() call. The >>>>> end result is >>>>> pthread_attr_setstacksize() had no >>>>> impact on the thread's stack >>>>> size, >>>>> and we ended up with the platform >>>>> default of 8mb. The fix is to >>>>> round up >>>>> the following variables to >>>>> PTHREAD_STACK_MIN after computing >>>>> their new >>>>> values: >>>>> >>>>> _java_thread_min_stack_allowed >>>>> _compiler_thread_min_stack_allowed >>>>> _vm_internal_thread_min_stack_allowed >>>>> >>>>> For solaris, there was an issue using >>>>> PTHREAD_STACK_MIN. You need to >>>>> #define _POSIX_C_SOURCE >= 199506L in >>>>> order to get PTHREAD_STACK_MIN >>>>> #defined, and this needs to be done >>>>> before including OS header >>>>> files. I >>>>> noticed that on solaris we were using >>>>> thr_min_stack() elsewhere >>>>> instead >>>>> of PTHREAD_STACK_MIN, so I decided >>>>> to do >>>>> the same with this fix. >>>>> Either >>>>> way is ugly (the #define or using >>>>> thr_min_stack()). >>>>> >>>>> And speaking of the existing use of >>>>> thr_min_stack(), I deleted >>>>> it. It >>>>> was being applied before any >>>>> adjustments >>>>> to the stack sizes had been >>>>> made (rounding and adding red, yellow, >>>>> and shadow zones). This mean >>>>> the >>>>> stack ended up being larger than >>>>> necessary. With the above fix in >>>>> place, >>>>> we are now applying thr_min_stack() >>>>> after recomputing the minimum >>>>> stack >>>>> sizes. If for any reason one of those >>>>> stack sizes is now too small, >>>>> the >>>>> correct fix is to adjust the initial >>>>> stack sizes, not apply >>>>> thr_min_stack() to the initial stack >>>>> sizes. However, it looks >>>>> like no >>>>> adjustment is needed. I did something >>>>> close to our nightly >>>>> testing on >>>>> all affect platforms, and no new >>>>> problems turned up. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>> >>> > > From chris.plummer at oracle.com Thu Mar 16 22:14:35 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 16 Mar 2017 15:14:35 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <129b18f4-ea7a-915f-e17c-76b7e96214e8@oracle.com> References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> <2ba3095a-0bff-68af-2d77-299b7edf35b1@oracle.com> <050560cc-aa81-9da4-1eae-1dab105d3926@oracle.com> <129b18f4-ea7a-915f-e17c-76b7e96214e8@oracle.com> Message-ID: On 3/16/17 3:01 PM, David Holmes wrote: > On 17/03/2017 7:43 AM, Chris Plummer wrote: >> On 3/16/17 2:35 PM, David Holmes wrote: >>> On 17/03/2017 3:49 AM, Chris Plummer wrote: >>>> On 3/16/17 2:16 AM, David Holmes wrote: >>>>> On 16/03/2017 6:30 PM, Thomas St?fe wrote: >>>>>> Hi Chris, David, >>>>>> >>>>>> the change looks good. >>>>>> >>>>>> I see that in the launcher we require a minimum stack size across >>>>>> all >>>>>> platforms ("STACK_SIZE_MINIMUM"), should we do the same fix (adjust >>>>>> for >>>>>> PTHREAD_STACK_MIN) there? >>>>>> >>>>>> I do not understand, why does error checking in the hotspot have >>>>>> to be >>>>>> consistent with the launcher? What prevents us from asserting in the >>>>>> hotspot - or at least print a warning? Note that in the hotspot, >>>>>> there >>>>>> is already UL logging ("os", "thread") after pthread_create() in the >>>>>> platform files, so the least we could do is add a warning log output >>>>>> case ppthread_attr_setstacksize fails. >>>>> >>>>> Sorry I'm getting this group of bugs all muddled up. >>>>> >>>>> Chris: this issue does affect hotspot and the launcher (potentially). >>>>> >>>>> Ideally both should be checking for failures in the pthread calls but >>>>> neither do so. Hotspot at least does so in some places but not in a >>>>> lot of others. >>>>> >>>>> pthread_create is different in hotspot because failure can happen >>>>> easily and we need to detect it and report it (via an exception and >>>>> also via UL). The other pthread calls are not expected to fail under >>>>> "normal" conditions but only due to a programming error. Those calls >>>>> should at least be checked in debug builds as we already do in places >>>>> with assert_status. >>>>> >>>>> The launcher code doesn't do any error checking at all (but again >>>>> pthread_create is a special case). >>>> Are you just referring to the pthread related error checking? It >>>> does do >>>> other error checking. >>> >>> pthread error checking. >>> >>> So trying to think this through ... >>> >>> If the user specifies a too small, or unaligned-to-page-size, -Xss >>> value the pthread_setstacksize() in the launcher will silently fail >>> and the main thread will get the default stack of 8M. It will then >>> load the VM which will then check the -Xss value, which will do its >>> own validity checking. >>> >> Close, except there is still a potential issue if the size is bigger >> than the minimum hotspot requires, but is not page size aligned. >> pthread_setstacksize *could* fail in this case, and there would be no >> "stack size too small" rejection from the hotspot. However, >> pthread_setstacksize did not fail on the two platforms I tried unaligned >> stack sizes on. > > Perhaps because that is not specified by POSIX. For POSIX we only have: > > [EINVAL] > The value of stacksize is less than {PTHREAD_STACK_MIN} or exceeds > a system-imposed limit. The man page on my linux host also adds the warning about "some hosts can fail if the stack size is not a multiple of the system page size." Is the man page documenting something different? > > Anyway that is a check that hotspot could perform if > pthread_attr_setstacksize fails. Though that then makes me wonder if > we do any rounding when the stack size set on a per thread basis via > the java.lang.Thread constructor? > > I think imposing the PTHREAD_STACK_MIN in hotspot, with an assert > checking pthread_attr_setstacksize succeeded (in hotspot) would > suffice here. If you are certain that the assert would be a programming error and not a user error, then I can see doing that. However, shouldn't we be consistent in the launcher and do the same there also? We can skip imposing PTHREAD_STACK_MIN since hotspot will already do this, but unless the user creates another java thread there will be no hotspot assert for pthread_attr_setstacksize failing. Chris > > David > ----- > >> Chris >>> That seems like quite a reasonable position for the launcher to take. >>> >>> David >>> ----- >>> >>>> >>>> Chris >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> If we ever refactor this coding, could we rename the variables >>>>>> holding >>>>>> the base stack size requirement for java frames - in all its >>>>>> incarnations in all the os_cpu files - to be renamed to something >>>>>> different? It is a bit confusing to have a variable which at >>>>>> different >>>>>> times in VM life means different things (before and after the call >>>>>> to os::Posix::set_minimum_stack_sizes()). Or, at least, rename >>>>>> "set_minimum_stack_sizes" to something like >>>>>> "adjust_minimum_stack_sizes" >>>>>> which makes the intent clearer. >>>>>> >>>>>> Kind Regards, Thomas >>>>>> >>>>>> On Thu, Mar 16, 2017 at 7:50 AM, David Holmes >>>>>> >>>>> > wrote: >>>>>> >>>>>> On 16/03/2017 4:33 PM, Chris Plummer wrote: >>>>>> >>>>>> On 3/15/17 11:18 PM, David Holmes wrote: >>>>>> >>>>>> On 16/03/2017 4:14 PM, Chris Plummer wrote: >>>>>> >>>>>> On 3/15/17 11:11 PM, David Holmes wrote: >>>>>> >>>>>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>>>>> >>>>>> On 3/15/17 10:23 PM, David Holmes wrote: >>>>>> >>>>>> Hi Chris, >>>>>> >>>>>> On 16/03/2017 3:03 PM, Chris Plummer >>>>>> wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>> Please review the following: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>> >>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Change looks good. >>>>>> >>>>>> While working on 8175342 I >>>>>> noticed our >>>>>> stack size on xgene was 8mb >>>>>> even >>>>>> though I was specifying -Xss72k. It >>>>>> turns out the following code was >>>>>> failing: >>>>>> >>>>>> pthread_attr_setstacksize(&attr, >>>>>> stack_size); >>>>>> >>>>>> >>>>>> So these really should be checking >>>>>> return >>>>>> values, at least in debug >>>>>> builds. But we can leave that until we >>>>>> refactor the thread startup >>>>>> code into os_posix.cpp. >>>>>> >>>>>> I considered adding checks. I wasn't sure >>>>>> if we >>>>>> should abort or just >>>>>> print a warning if it failed. >>>>>> >>>>>> >>>>>> When we check pthread lib routines we use: >>>>>> >>>>>> int status = pthread_mutex_lock(_mutex); >>>>>> assert_status(status == 0, status, >>>>>> "mutex_lock"); >>>>>> >>>>>> This is for things that should only fail if we >>>>>> have >>>>>> a programming >>>>>> error. >>>>>> >>>>>> Ok, but this is in the launcher, so I'll need to >>>>>> just >>>>>> use the built-in >>>>>> assert(). I'll add that if want. >>>>>> >>>>>> >>>>>> Oops! I was forgetting that. Need to be consistent with >>>>>> launcher error >>>>>> checking or lack thereof. And ignore refactoring >>>>>> comments - >>>>>> not relevant. >>>>>> >>>>>> So don't add the error check? >>>>>> >>>>>> >>>>>> Given there is no error checking, or assertions, in those >>>>>> files I >>>>>> reluctantly have to say leave it out. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> >>>>>> >>>>>> David >>>>>> >>>>>> Chris >>>>>> >>>>>> >>>>>> What refactoring is planned? >>>>>> >>>>>> >>>>>> "Planned" might be a bit strong :) I was >>>>>> thinking of >>>>>> a number of >>>>>> os_posix related cleanups for which issues >>>>>> exist, >>>>>> but also forgot that >>>>>> some of our general clean-up RFE's have been >>>>>> closed >>>>>> as WNF :( I may do >>>>>> some of them after hours anyway :) >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>> Chris >>>>>> >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> Although we computed a minimum stack >>>>>> size of 72k, so -Xss72k >>>>>> should be >>>>>> fine, pthreads on this platform >>>>>> requires >>>>>> the stack be at least >>>>>> 128k, so >>>>>> it failed the >>>>>> pthread_attr_setstacksize() call. The >>>>>> end result is >>>>>> pthread_attr_setstacksize() had no >>>>>> impact on the thread's stack >>>>>> size, >>>>>> and we ended up with the platform >>>>>> default of 8mb. The fix is to >>>>>> round up >>>>>> the following variables to >>>>>> PTHREAD_STACK_MIN after computing >>>>>> their new >>>>>> values: >>>>>> >>>>>> _java_thread_min_stack_allowed >>>>>> _compiler_thread_min_stack_allowed >>>>>> _vm_internal_thread_min_stack_allowed >>>>>> >>>>>> For solaris, there was an issue >>>>>> using >>>>>> PTHREAD_STACK_MIN. You need to >>>>>> #define _POSIX_C_SOURCE >= >>>>>> 199506L in >>>>>> order to get PTHREAD_STACK_MIN >>>>>> #defined, and this needs to be done >>>>>> before including OS header >>>>>> files. I >>>>>> noticed that on solaris we were >>>>>> using >>>>>> thr_min_stack() elsewhere >>>>>> instead >>>>>> of PTHREAD_STACK_MIN, so I decided >>>>>> to do >>>>>> the same with this fix. >>>>>> Either >>>>>> way is ugly (the #define or using >>>>>> thr_min_stack()). >>>>>> >>>>>> And speaking of the existing use of >>>>>> thr_min_stack(), I deleted >>>>>> it. It >>>>>> was being applied before any >>>>>> adjustments >>>>>> to the stack sizes had been >>>>>> made (rounding and adding red, >>>>>> yellow, >>>>>> and shadow zones). This mean >>>>>> the >>>>>> stack ended up being larger than >>>>>> necessary. With the above fix in >>>>>> place, >>>>>> we are now applying thr_min_stack() >>>>>> after recomputing the minimum >>>>>> stack >>>>>> sizes. If for any reason one of >>>>>> those >>>>>> stack sizes is now too small, >>>>>> the >>>>>> correct fix is to adjust the initial >>>>>> stack sizes, not apply >>>>>> thr_min_stack() to the initial stack >>>>>> sizes. However, it looks >>>>>> like no >>>>>> adjustment is needed. I did >>>>>> something >>>>>> close to our nightly >>>>>> testing on >>>>>> all affect platforms, and no new >>>>>> problems turned up. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> >> >> From david.holmes at oracle.com Fri Mar 17 00:04:29 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Mar 2017 10:04:29 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> <2ba3095a-0bff-68af-2d77-299b7edf35b1@oracle.com> <050560cc-aa81-9da4-1eae-1dab105d3926@oracle.com> <129b18f4-ea7a-915f-e17c-76b7e96214e8@oracle.com> Message-ID: Hi Chris, On 17/03/2017 8:14 AM, Chris Plummer wrote: > On 3/16/17 3:01 PM, David Holmes wrote: >> On 17/03/2017 7:43 AM, Chris Plummer wrote: >>> On 3/16/17 2:35 PM, David Holmes wrote: >>>> On 17/03/2017 3:49 AM, Chris Plummer wrote: >>>>> On 3/16/17 2:16 AM, David Holmes wrote: >>>>>> On 16/03/2017 6:30 PM, Thomas St?fe wrote: >>>>>>> Hi Chris, David, >>>>>>> >>>>>>> the change looks good. >>>>>>> >>>>>>> I see that in the launcher we require a minimum stack size across >>>>>>> all >>>>>>> platforms ("STACK_SIZE_MINIMUM"), should we do the same fix (adjust >>>>>>> for >>>>>>> PTHREAD_STACK_MIN) there? >>>>>>> >>>>>>> I do not understand, why does error checking in the hotspot have >>>>>>> to be >>>>>>> consistent with the launcher? What prevents us from asserting in the >>>>>>> hotspot - or at least print a warning? Note that in the hotspot, >>>>>>> there >>>>>>> is already UL logging ("os", "thread") after pthread_create() in the >>>>>>> platform files, so the least we could do is add a warning log output >>>>>>> case ppthread_attr_setstacksize fails. >>>>>> >>>>>> Sorry I'm getting this group of bugs all muddled up. >>>>>> >>>>>> Chris: this issue does affect hotspot and the launcher (potentially). >>>>>> >>>>>> Ideally both should be checking for failures in the pthread calls but >>>>>> neither do so. Hotspot at least does so in some places but not in a >>>>>> lot of others. >>>>>> >>>>>> pthread_create is different in hotspot because failure can happen >>>>>> easily and we need to detect it and report it (via an exception and >>>>>> also via UL). The other pthread calls are not expected to fail under >>>>>> "normal" conditions but only due to a programming error. Those calls >>>>>> should at least be checked in debug builds as we already do in places >>>>>> with assert_status. >>>>>> >>>>>> The launcher code doesn't do any error checking at all (but again >>>>>> pthread_create is a special case). >>>>> Are you just referring to the pthread related error checking? It >>>>> does do >>>>> other error checking. >>>> >>>> pthread error checking. >>>> >>>> So trying to think this through ... >>>> >>>> If the user specifies a too small, or unaligned-to-page-size, -Xss >>>> value the pthread_setstacksize() in the launcher will silently fail >>>> and the main thread will get the default stack of 8M. It will then >>>> load the VM which will then check the -Xss value, which will do its >>>> own validity checking. >>>> >>> Close, except there is still a potential issue if the size is bigger >>> than the minimum hotspot requires, but is not page size aligned. >>> pthread_setstacksize *could* fail in this case, and there would be no >>> "stack size too small" rejection from the hotspot. However, >>> pthread_setstacksize did not fail on the two platforms I tried unaligned >>> stack sizes on. >> >> Perhaps because that is not specified by POSIX. For POSIX we only have: >> >> [EINVAL] >> The value of stacksize is less than {PTHREAD_STACK_MIN} or exceeds >> a system-imposed limit. > The man page on my linux host also adds the warning about "some hosts > can fail if the stack size is not a multiple of the system page size." > Is the man page documenting something different? Yes it's documenting that Linux doesn't follow POSIX very well a lot of the time. :( However the source code seems different: http://code.metager.de/source/xref/gnu/glibc/nptl/pthreadP.h 628 static inline int 629 check_stacksize_attr (size_t st) 630 { 631 if (st >= PTHREAD_STACK_MIN) 632 return 0; 633 634 return EINVAL; 635 } That said, it seems that OSX will return EINVAL: https://opensource.apple.com/source/libpthread/libpthread-218.1.3/src/pthread.c.auto.html int ret = EINVAL; if (attr->sig == _PTHREAD_ATTR_SIG && (stacksize % vm_page_size) == 0 && stacksize >= PTHREAD_STACK_MIN) { attr->stacksize = stacksize; ret = 0; } return ret; >> >> Anyway that is a check that hotspot could perform if >> pthread_attr_setstacksize fails. Though that then makes me wonder if >> we do any rounding when the stack size set on a per thread basis via >> the java.lang.Thread constructor? >> >> I think imposing the PTHREAD_STACK_MIN in hotspot, with an assert >> checking pthread_attr_setstacksize succeeded (in hotspot) would >> suffice here. > If you are certain that the assert would be a programming error and not > a user error, then I can see doing that. However, shouldn't we be > consistent in the launcher and do the same there also? We can skip > imposing PTHREAD_STACK_MIN since hotspot will already do this, but > unless the user creates another java thread there will be no hotspot > assert for pthread_attr_setstacksize failing. The launcher has its own policy regarding errors here - it ignores them. As I said before if you pass an invalid -Xss value the launcher will create the main thread with the default stack size. That is not an unreasonable position to take. If you feel strongly about it you could file a bug against the launcher, but I would not try to fix it in this CR. Inside hotspot I think we already do various roundings on the value eventually passed to pthread_attr_setstacksize, don't we? So any EINVAL after that should be effectively impossible when combined with the PTHREAD_STACK_MIN check. Thanks, David > Chris >> >> David >> ----- >> >>> Chris >>>> That seems like quite a reasonable position for the launcher to take. >>>> >>>> David >>>> ----- >>>> >>>>> >>>>> Chris >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> If we ever refactor this coding, could we rename the variables >>>>>>> holding >>>>>>> the base stack size requirement for java frames - in all its >>>>>>> incarnations in all the os_cpu files - to be renamed to something >>>>>>> different? It is a bit confusing to have a variable which at >>>>>>> different >>>>>>> times in VM life means different things (before and after the call >>>>>>> to os::Posix::set_minimum_stack_sizes()). Or, at least, rename >>>>>>> "set_minimum_stack_sizes" to something like >>>>>>> "adjust_minimum_stack_sizes" >>>>>>> which makes the intent clearer. >>>>>>> >>>>>>> Kind Regards, Thomas >>>>>>> >>>>>>> On Thu, Mar 16, 2017 at 7:50 AM, David Holmes >>>>>>> >>>>>> > wrote: >>>>>>> >>>>>>> On 16/03/2017 4:33 PM, Chris Plummer wrote: >>>>>>> >>>>>>> On 3/15/17 11:18 PM, David Holmes wrote: >>>>>>> >>>>>>> On 16/03/2017 4:14 PM, Chris Plummer wrote: >>>>>>> >>>>>>> On 3/15/17 11:11 PM, David Holmes wrote: >>>>>>> >>>>>>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>>>>>> >>>>>>> On 3/15/17 10:23 PM, David Holmes wrote: >>>>>>> >>>>>>> Hi Chris, >>>>>>> >>>>>>> On 16/03/2017 3:03 PM, Chris Plummer >>>>>>> wrote: >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Please review the following: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Change looks good. >>>>>>> >>>>>>> While working on 8175342 I >>>>>>> noticed our >>>>>>> stack size on xgene was 8mb >>>>>>> even >>>>>>> though I was specifying -Xss72k. It >>>>>>> turns out the following code was >>>>>>> failing: >>>>>>> >>>>>>> pthread_attr_setstacksize(&attr, >>>>>>> stack_size); >>>>>>> >>>>>>> >>>>>>> So these really should be checking >>>>>>> return >>>>>>> values, at least in debug >>>>>>> builds. But we can leave that until we >>>>>>> refactor the thread startup >>>>>>> code into os_posix.cpp. >>>>>>> >>>>>>> I considered adding checks. I wasn't sure >>>>>>> if we >>>>>>> should abort or just >>>>>>> print a warning if it failed. >>>>>>> >>>>>>> >>>>>>> When we check pthread lib routines we use: >>>>>>> >>>>>>> int status = pthread_mutex_lock(_mutex); >>>>>>> assert_status(status == 0, status, >>>>>>> "mutex_lock"); >>>>>>> >>>>>>> This is for things that should only fail if we >>>>>>> have >>>>>>> a programming >>>>>>> error. >>>>>>> >>>>>>> Ok, but this is in the launcher, so I'll need to >>>>>>> just >>>>>>> use the built-in >>>>>>> assert(). I'll add that if want. >>>>>>> >>>>>>> >>>>>>> Oops! I was forgetting that. Need to be consistent with >>>>>>> launcher error >>>>>>> checking or lack thereof. And ignore refactoring >>>>>>> comments - >>>>>>> not relevant. >>>>>>> >>>>>>> So don't add the error check? >>>>>>> >>>>>>> >>>>>>> Given there is no error checking, or assertions, in those >>>>>>> files I >>>>>>> reluctantly have to say leave it out. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> >>>>>>> >>>>>>> David >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> >>>>>>> What refactoring is planned? >>>>>>> >>>>>>> >>>>>>> "Planned" might be a bit strong :) I was >>>>>>> thinking of >>>>>>> a number of >>>>>>> os_posix related cleanups for which issues >>>>>>> exist, >>>>>>> but also forgot that >>>>>>> some of our general clean-up RFE's have been >>>>>>> closed >>>>>>> as WNF :( I may do >>>>>>> some of them after hours anyway :) >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> Although we computed a minimum stack >>>>>>> size of 72k, so -Xss72k >>>>>>> should be >>>>>>> fine, pthreads on this platform >>>>>>> requires >>>>>>> the stack be at least >>>>>>> 128k, so >>>>>>> it failed the >>>>>>> pthread_attr_setstacksize() call. The >>>>>>> end result is >>>>>>> pthread_attr_setstacksize() had no >>>>>>> impact on the thread's stack >>>>>>> size, >>>>>>> and we ended up with the platform >>>>>>> default of 8mb. The fix is to >>>>>>> round up >>>>>>> the following variables to >>>>>>> PTHREAD_STACK_MIN after computing >>>>>>> their new >>>>>>> values: >>>>>>> >>>>>>> _java_thread_min_stack_allowed >>>>>>> _compiler_thread_min_stack_allowed >>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>> >>>>>>> For solaris, there was an issue >>>>>>> using >>>>>>> PTHREAD_STACK_MIN. You need to >>>>>>> #define _POSIX_C_SOURCE >= >>>>>>> 199506L in >>>>>>> order to get PTHREAD_STACK_MIN >>>>>>> #defined, and this needs to be done >>>>>>> before including OS header >>>>>>> files. I >>>>>>> noticed that on solaris we were >>>>>>> using >>>>>>> thr_min_stack() elsewhere >>>>>>> instead >>>>>>> of PTHREAD_STACK_MIN, so I decided >>>>>>> to do >>>>>>> the same with this fix. >>>>>>> Either >>>>>>> way is ugly (the #define or using >>>>>>> thr_min_stack()). >>>>>>> >>>>>>> And speaking of the existing use of >>>>>>> thr_min_stack(), I deleted >>>>>>> it. It >>>>>>> was being applied before any >>>>>>> adjustments >>>>>>> to the stack sizes had been >>>>>>> made (rounding and adding red, >>>>>>> yellow, >>>>>>> and shadow zones). This mean >>>>>>> the >>>>>>> stack ended up being larger than >>>>>>> necessary. With the above fix in >>>>>>> place, >>>>>>> we are now applying thr_min_stack() >>>>>>> after recomputing the minimum >>>>>>> stack >>>>>>> sizes. If for any reason one of >>>>>>> those >>>>>>> stack sizes is now too small, >>>>>>> the >>>>>>> correct fix is to adjust the initial >>>>>>> stack sizes, not apply >>>>>>> thr_min_stack() to the initial stack >>>>>>> sizes. However, it looks >>>>>>> like no >>>>>>> adjustment is needed. I did >>>>>>> something >>>>>>> close to our nightly >>>>>>> testing on >>>>>>> all affect platforms, and no new >>>>>>> problems turned up. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>> >>> > > From chris.plummer at oracle.com Fri Mar 17 00:18:11 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 16 Mar 2017 17:18:11 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <37203fa8-54c1-91b8-1c8c-7103b1c7add5@oracle.com> <600c0e89-65a4-a12e-992a-f29dcad1a6f9@oracle.com> <36fe5bce-8a1c-39f5-c8ae-f5c8ed04b745@oracle.com> <9ed9898e-ea43-96e6-d0fa-607751929711@oracle.com> <046b9a56-18ff-a78f-599d-63cfcfa333e0@oracle.com> <51d56ab6-7f00-6ef0-30fd-82590480cabe@oracle.com> <2ba3095a-0bff-68af-2d77-299b7edf35b1@oracle.com> <050560cc-aa81-9da4-1eae-1dab105d3926@oracle.com> <129b18f4-ea7a-915f-e17c-76b7e96214e8@oracle.com> Message-ID: <488d69ff-88a8-5cc1-b132-99da0868b1c9@oracle.com> On 3/16/17 5:04 PM, David Holmes wrote: > Hi Chris, > > On 17/03/2017 8:14 AM, Chris Plummer wrote: >> On 3/16/17 3:01 PM, David Holmes wrote: >>> On 17/03/2017 7:43 AM, Chris Plummer wrote: >>>> On 3/16/17 2:35 PM, David Holmes wrote: >>>>> On 17/03/2017 3:49 AM, Chris Plummer wrote: >>>>>> On 3/16/17 2:16 AM, David Holmes wrote: >>>>>>> On 16/03/2017 6:30 PM, Thomas St?fe wrote: >>>>>>>> Hi Chris, David, >>>>>>>> >>>>>>>> the change looks good. >>>>>>>> >>>>>>>> I see that in the launcher we require a minimum stack size across >>>>>>>> all >>>>>>>> platforms ("STACK_SIZE_MINIMUM"), should we do the same fix >>>>>>>> (adjust >>>>>>>> for >>>>>>>> PTHREAD_STACK_MIN) there? >>>>>>>> >>>>>>>> I do not understand, why does error checking in the hotspot have >>>>>>>> to be >>>>>>>> consistent with the launcher? What prevents us from asserting >>>>>>>> in the >>>>>>>> hotspot - or at least print a warning? Note that in the hotspot, >>>>>>>> there >>>>>>>> is already UL logging ("os", "thread") after pthread_create() >>>>>>>> in the >>>>>>>> platform files, so the least we could do is add a warning log >>>>>>>> output >>>>>>>> case ppthread_attr_setstacksize fails. >>>>>>> >>>>>>> Sorry I'm getting this group of bugs all muddled up. >>>>>>> >>>>>>> Chris: this issue does affect hotspot and the launcher >>>>>>> (potentially). >>>>>>> >>>>>>> Ideally both should be checking for failures in the pthread >>>>>>> calls but >>>>>>> neither do so. Hotspot at least does so in some places but not in a >>>>>>> lot of others. >>>>>>> >>>>>>> pthread_create is different in hotspot because failure can happen >>>>>>> easily and we need to detect it and report it (via an exception and >>>>>>> also via UL). The other pthread calls are not expected to fail >>>>>>> under >>>>>>> "normal" conditions but only due to a programming error. Those >>>>>>> calls >>>>>>> should at least be checked in debug builds as we already do in >>>>>>> places >>>>>>> with assert_status. >>>>>>> >>>>>>> The launcher code doesn't do any error checking at all (but again >>>>>>> pthread_create is a special case). >>>>>> Are you just referring to the pthread related error checking? It >>>>>> does do >>>>>> other error checking. >>>>> >>>>> pthread error checking. >>>>> >>>>> So trying to think this through ... >>>>> >>>>> If the user specifies a too small, or unaligned-to-page-size, -Xss >>>>> value the pthread_setstacksize() in the launcher will silently fail >>>>> and the main thread will get the default stack of 8M. It will then >>>>> load the VM which will then check the -Xss value, which will do its >>>>> own validity checking. >>>>> >>>> Close, except there is still a potential issue if the size is bigger >>>> than the minimum hotspot requires, but is not page size aligned. >>>> pthread_setstacksize *could* fail in this case, and there would be no >>>> "stack size too small" rejection from the hotspot. However, >>>> pthread_setstacksize did not fail on the two platforms I tried >>>> unaligned >>>> stack sizes on. >>> >>> Perhaps because that is not specified by POSIX. For POSIX we only have: >>> >>> [EINVAL] >>> The value of stacksize is less than {PTHREAD_STACK_MIN} or exceeds >>> a system-imposed limit. >> The man page on my linux host also adds the warning about "some hosts >> can fail if the stack size is not a multiple of the system page size." >> Is the man page documenting something different? > > Yes it's documenting that Linux doesn't follow POSIX very well a lot > of the time. :( However the source code seems different: > > http://code.metager.de/source/xref/gnu/glibc/nptl/pthreadP.h > > 628 static inline int > 629 check_stacksize_attr (size_t st) > 630 { > 631 if (st >= PTHREAD_STACK_MIN) > 632 return 0; > 633 > 634 return EINVAL; > 635 } > > That said, it seems that OSX will return EINVAL: > > https://opensource.apple.com/source/libpthread/libpthread-218.1.3/src/pthread.c.auto.html > > > int ret = EINVAL; > if (attr->sig == _PTHREAD_ATTR_SIG && > (stacksize % vm_page_size) == 0 && > stacksize >= PTHREAD_STACK_MIN) { > attr->stacksize = stacksize; > ret = 0; > } > return ret; Ok. Thanks for finding this. I had found the Mac OS X man page and it too mentioned that unaligned sizes might cause EINVAL. > >>> >>> Anyway that is a check that hotspot could perform if >>> pthread_attr_setstacksize fails. Though that then makes me wonder if >>> we do any rounding when the stack size set on a per thread basis via >>> the java.lang.Thread constructor? >>> >>> I think imposing the PTHREAD_STACK_MIN in hotspot, with an assert >>> checking pthread_attr_setstacksize succeeded (in hotspot) would >>> suffice here. >> If you are certain that the assert would be a programming error and not >> a user error, then I can see doing that. However, shouldn't we be >> consistent in the launcher and do the same there also? We can skip >> imposing PTHREAD_STACK_MIN since hotspot will already do this, but >> unless the user creates another java thread there will be no hotspot >> assert for pthread_attr_setstacksize failing. > > The launcher has its own policy regarding errors here - it ignores > them. As I said before if you pass an invalid -Xss value the launcher > will create the main thread with the default stack size. That is not > an unreasonable position to take. If you feel strongly about it you > could file a bug against the launcher, but I would not try to fix it > in this CR. Ok. At least it's not fatal. > > > Inside hotspot I think we already do various roundings on the value > eventually passed to pthread_attr_setstacksize, don't we? So any > EINVAL after that should be effectively impossible when combined with > the PTHREAD_STACK_MIN check. > Yes: size_t stack_size_in_bytes = ThreadStackSize * K; ... // Make the stack size a multiple of the page size so that // the yellow/red zones can be guarded. JavaThread::set_stack_size_at_create(round_to(stack_size_in_bytes, vm_page_size())); Maybe I should add to the comment "...and also so pthread_attr_setstacksize won't fail". Chris > Thanks, > David > >> Chris >>> >>> David >>> ----- >>> >>>> Chris >>>>> That seems like quite a reasonable position for the launcher to take. >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Chris >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> If we ever refactor this coding, could we rename the variables >>>>>>>> holding >>>>>>>> the base stack size requirement for java frames - in all its >>>>>>>> incarnations in all the os_cpu files - to be renamed to something >>>>>>>> different? It is a bit confusing to have a variable which at >>>>>>>> different >>>>>>>> times in VM life means different things (before and after the call >>>>>>>> to os::Posix::set_minimum_stack_sizes()). Or, at least, rename >>>>>>>> "set_minimum_stack_sizes" to something like >>>>>>>> "adjust_minimum_stack_sizes" >>>>>>>> which makes the intent clearer. >>>>>>>> >>>>>>>> Kind Regards, Thomas >>>>>>>> >>>>>>>> On Thu, Mar 16, 2017 at 7:50 AM, David Holmes >>>>>>>> >>>>>>> > wrote: >>>>>>>> >>>>>>>> On 16/03/2017 4:33 PM, Chris Plummer wrote: >>>>>>>> >>>>>>>> On 3/15/17 11:18 PM, David Holmes wrote: >>>>>>>> >>>>>>>> On 16/03/2017 4:14 PM, Chris Plummer wrote: >>>>>>>> >>>>>>>> On 3/15/17 11:11 PM, David Holmes wrote: >>>>>>>> >>>>>>>> On 16/03/2017 3:51 PM, Chris Plummer wrote: >>>>>>>> >>>>>>>> On 3/15/17 10:23 PM, David Holmes wrote: >>>>>>>> >>>>>>>> Hi Chris, >>>>>>>> >>>>>>>> On 16/03/2017 3:03 PM, Chris Plummer >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the following: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Change looks good. >>>>>>>> >>>>>>>> While working on 8175342 I >>>>>>>> noticed our >>>>>>>> stack size on xgene was 8mb >>>>>>>> even >>>>>>>> though I was specifying >>>>>>>> -Xss72k. It >>>>>>>> turns out the following code was >>>>>>>> failing: >>>>>>>> >>>>>>>> pthread_attr_setstacksize(&attr, >>>>>>>> stack_size); >>>>>>>> >>>>>>>> >>>>>>>> So these really should be checking >>>>>>>> return >>>>>>>> values, at least in debug >>>>>>>> builds. But we can leave that until we >>>>>>>> refactor the thread startup >>>>>>>> code into os_posix.cpp. >>>>>>>> >>>>>>>> I considered adding checks. I wasn't sure >>>>>>>> if we >>>>>>>> should abort or just >>>>>>>> print a warning if it failed. >>>>>>>> >>>>>>>> >>>>>>>> When we check pthread lib routines we use: >>>>>>>> >>>>>>>> int status = pthread_mutex_lock(_mutex); >>>>>>>> assert_status(status == 0, status, >>>>>>>> "mutex_lock"); >>>>>>>> >>>>>>>> This is for things that should only fail if we >>>>>>>> have >>>>>>>> a programming >>>>>>>> error. >>>>>>>> >>>>>>>> Ok, but this is in the launcher, so I'll need to >>>>>>>> just >>>>>>>> use the built-in >>>>>>>> assert(). I'll add that if want. >>>>>>>> >>>>>>>> >>>>>>>> Oops! I was forgetting that. Need to be consistent >>>>>>>> with >>>>>>>> launcher error >>>>>>>> checking or lack thereof. And ignore refactoring >>>>>>>> comments - >>>>>>>> not relevant. >>>>>>>> >>>>>>>> So don't add the error check? >>>>>>>> >>>>>>>> >>>>>>>> Given there is no error checking, or assertions, in those >>>>>>>> files I >>>>>>>> reluctantly have to say leave it out. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>>> What refactoring is planned? >>>>>>>> >>>>>>>> >>>>>>>> "Planned" might be a bit strong :) I was >>>>>>>> thinking of >>>>>>>> a number of >>>>>>>> os_posix related cleanups for which issues >>>>>>>> exist, >>>>>>>> but also forgot that >>>>>>>> some of our general clean-up RFE's have been >>>>>>>> closed >>>>>>>> as WNF :( I may do >>>>>>>> some of them after hours anyway :) >>>>>>>> >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>> Although we computed a minimum >>>>>>>> stack >>>>>>>> size of 72k, so -Xss72k >>>>>>>> should be >>>>>>>> fine, pthreads on this platform >>>>>>>> requires >>>>>>>> the stack be at least >>>>>>>> 128k, so >>>>>>>> it failed the >>>>>>>> pthread_attr_setstacksize() call. The >>>>>>>> end result is >>>>>>>> pthread_attr_setstacksize() had no >>>>>>>> impact on the thread's stack >>>>>>>> size, >>>>>>>> and we ended up with the platform >>>>>>>> default of 8mb. The fix is to >>>>>>>> round up >>>>>>>> the following variables to >>>>>>>> PTHREAD_STACK_MIN after computing >>>>>>>> their new >>>>>>>> values: >>>>>>>> >>>>>>>> _java_thread_min_stack_allowed >>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>> >>>>>>>> For solaris, there was an issue >>>>>>>> using >>>>>>>> PTHREAD_STACK_MIN. You need to >>>>>>>> #define _POSIX_C_SOURCE >= >>>>>>>> 199506L in >>>>>>>> order to get PTHREAD_STACK_MIN >>>>>>>> #defined, and this needs to be >>>>>>>> done >>>>>>>> before including OS header >>>>>>>> files. I >>>>>>>> noticed that on solaris we were >>>>>>>> using >>>>>>>> thr_min_stack() elsewhere >>>>>>>> instead >>>>>>>> of PTHREAD_STACK_MIN, so I decided >>>>>>>> to do >>>>>>>> the same with this fix. >>>>>>>> Either >>>>>>>> way is ugly (the #define or using >>>>>>>> thr_min_stack()). >>>>>>>> >>>>>>>> And speaking of the existing >>>>>>>> use of >>>>>>>> thr_min_stack(), I deleted >>>>>>>> it. It >>>>>>>> was being applied before any >>>>>>>> adjustments >>>>>>>> to the stack sizes had been >>>>>>>> made (rounding and adding red, >>>>>>>> yellow, >>>>>>>> and shadow zones). This mean >>>>>>>> the >>>>>>>> stack ended up being larger than >>>>>>>> necessary. With the above fix in >>>>>>>> place, >>>>>>>> we are now applying >>>>>>>> thr_min_stack() >>>>>>>> after recomputing the minimum >>>>>>>> stack >>>>>>>> sizes. If for any reason one of >>>>>>>> those >>>>>>>> stack sizes is now too small, >>>>>>>> the >>>>>>>> correct fix is to adjust the >>>>>>>> initial >>>>>>>> stack sizes, not apply >>>>>>>> thr_min_stack() to the initial >>>>>>>> stack >>>>>>>> sizes. However, it looks >>>>>>>> like no >>>>>>>> adjustment is needed. I did >>>>>>>> something >>>>>>>> close to our nightly >>>>>>>> testing on >>>>>>>> all affect platforms, and no new >>>>>>>> problems turned up. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>> >>>> >> >> From serguei.spitsyn at oracle.com Fri Mar 17 00:59:40 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 16 Mar 2017 17:59:40 -0700 Subject: RFR (S): 8176831: Dead code: function jmm_GetLoadedClasses is not used in jmm_interface Message-ID: <84f65d92-d937-deef-827a-cbc30a03f3e1@oracle.com> Please, review the jdk 10 fix for: https://bugs.openjdk.java.net/browse/JDK-8176831 Webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8176831-jmm-dead.1/ Summary: It was found at the review of the 8155672 that the function jmm_GetLoadedClasses() defined in services/management.cpp is not really used in the jmm_interface. This function and dead code associated with its implementation is removed. Testing: The nsk.monitoring and jtreg jdk_management tests are in progress. Thanks, Serguei From david.holmes at oracle.com Fri Mar 17 01:15:21 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Mar 2017 11:15:21 +1000 Subject: RFR (S): 8176831: Dead code: function jmm_GetLoadedClasses is not used in jmm_interface In-Reply-To: <84f65d92-d937-deef-827a-cbc30a03f3e1@oracle.com> References: <84f65d92-d937-deef-827a-cbc30a03f3e1@oracle.com> Message-ID: Looks good! Don't forget to update copyright years. Thanks, David On 17/03/2017 10:59 AM, serguei.spitsyn at oracle.com wrote: > Please, review the jdk 10 fix for: > https://bugs.openjdk.java.net/browse/JDK-8176831 > > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8176831-jmm-dead.1/ > > > > Summary: > > It was found at the review of the 8155672 that the function > jmm_GetLoadedClasses() defined in services/management.cpp > is not really used in the jmm_interface. This function and > dead code associated with its implementation is removed. > > Testing: > The nsk.monitoring and jtreg jdk_management tests are in progress. > > > Thanks, > Serguei > > > > From serguei.spitsyn at oracle.com Fri Mar 17 01:20:42 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 16 Mar 2017 18:20:42 -0700 Subject: RFR (S): 8176831: Dead code: function jmm_GetLoadedClasses is not used in jmm_interface In-Reply-To: References: <84f65d92-d937-deef-827a-cbc30a03f3e1@oracle.com> Message-ID: <480b2c46-c094-de6e-2b90-05b90b3ea783@oracle.com> David, Thank you for the review! On 3/16/17 18:15, David Holmes wrote: > Looks good! > > Don't forget to update copyright years. Yes, of course. Thanks, Serguei > > Thanks, > David > > On 17/03/2017 10:59 AM, serguei.spitsyn at oracle.com wrote: >> Please, review the jdk 10 fix for: >> https://bugs.openjdk.java.net/browse/JDK-8176831 >> >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8176831-jmm-dead.1/ >> >> >> >> >> Summary: >> >> It was found at the review of the 8155672 that the function >> jmm_GetLoadedClasses() defined in services/management.cpp >> is not really used in the jmm_interface. This function and >> dead code associated with its implementation is removed. >> >> Testing: >> The nsk.monitoring and jtreg jdk_management tests are in progress. >> >> >> Thanks, >> Serguei >> >> >> >> From serguei.spitsyn at oracle.com Fri Mar 17 01:38:14 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 16 Mar 2017 18:38:14 -0700 Subject: RFR (S): 8176831: Dead code: function jmm_GetLoadedClasses is not used in jmm_interface In-Reply-To: <480b2c46-c094-de6e-2b90-05b90b3ea783@oracle.com> References: <84f65d92-d937-deef-827a-cbc30a03f3e1@oracle.com> <480b2c46-c094-de6e-2b90-05b90b3ea783@oracle.com> Message-ID: <0b347619-f5f7-4804-ab3a-1cbb7871be0e@oracle.com> I've reloaded the webrev with pulled latest jdk10 changes. It looks almost the same but has the recent update from Coleen. Thanks, Serguei On 3/16/17 18:20, serguei.spitsyn at oracle.com wrote: > David, > > Thank you for the review! > > > On 3/16/17 18:15, David Holmes wrote: >> Looks good! >> >> Don't forget to update copyright years. > > Yes, of course. > > > Thanks, > Serguei >> >> Thanks, >> David >> >> On 17/03/2017 10:59 AM, serguei.spitsyn at oracle.com wrote: >>> Please, review the jdk 10 fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8176831 >>> >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8176831-jmm-dead.1/ >>> >>> >>> >>> >>> Summary: >>> >>> It was found at the review of the 8155672 that the function >>> jmm_GetLoadedClasses() defined in services/management.cpp >>> is not really used in the jmm_interface. This function and >>> dead code associated with its implementation is removed. >>> >>> Testing: >>> The nsk.monitoring and jtreg jdk_management tests are in progress. >>> >>> >>> Thanks, >>> Serguei >>> >>> >>> >>> > From chris.plummer at oracle.com Fri Mar 17 04:27:23 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 16 Mar 2017 21:27:23 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: Message-ID: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> Ok, time for a new webrev: http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot The only thing that has changed since the first webrev are the asserts added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss already: - The assert should never happen due to the stack size being too small since it will be at least PTHREAD_STACK_MIN. - The assert should never happen due to an unaligned stack size because we always align it to the page size. - Any assert would therefore be a VM bug and not due to user error. - No fixing the java launcher. If the user specifies a stack that is too small, hotspot will already detect this. If the user specifies a stack size that is large enough but not page aligned, then we just ignore any error (if the platform doth protest) and the user gets a main thread with a stack size set to whatever the OS default is. I still need to retest (I only ran TooSmallStackSize.java), but figure getting agreement on the changes first would be best before I bog down our testing resources. thanks, Chris On 3/15/17 10:03 PM, Chris Plummer wrote: > Hello, > > Please review the following: > > https://bugs.openjdk.java.net/browse/JDK-8176768 > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot > > While working on 8175342 I noticed our stack size on xgene was 8mb > even though I was specifying -Xss72k. It turns out the following code > was failing: > > pthread_attr_setstacksize(&attr, stack_size); > > Although we computed a minimum stack size of 72k, so -Xss72k should be > fine, pthreads on this platform requires the stack be at least 128k, > so it failed the pthread_attr_setstacksize() call. The end result is > pthread_attr_setstacksize() had no impact on the thread's stack size, > and we ended up with the platform default of 8mb. The fix is to round > up the following variables to PTHREAD_STACK_MIN after computing their > new values: > > _java_thread_min_stack_allowed > _compiler_thread_min_stack_allowed > _vm_internal_thread_min_stack_allowed > > For solaris, there was an issue using PTHREAD_STACK_MIN. You need to > #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN > #defined, and this needs to be done before including OS header files. > I noticed that on solaris we were using thr_min_stack() elsewhere > instead of PTHREAD_STACK_MIN, so I decided to do the same with this > fix. Either way is ugly (the #define or using thr_min_stack()). > > And speaking of the existing use of thr_min_stack(), I deleted it. It > was being applied before any adjustments to the stack sizes had been > made (rounding and adding red, yellow, and shadow zones). This mean > the stack ended up being larger than necessary. With the above fix in > place, we are now applying thr_min_stack() after recomputing the > minimum stack sizes. If for any reason one of those stack sizes is now > too small, the correct fix is to adjust the initial stack sizes, not > apply thr_min_stack() to the initial stack sizes. However, it looks > like no adjustment is needed. I did something close to our nightly > testing on all affect platforms, and no new problems turned up. > > thanks, > > Chris From david.holmes at oracle.com Fri Mar 17 06:26:37 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Mar 2017 16:26:37 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> Message-ID: <22513c52-0150-3b85-acfc-85e4c161e549@oracle.com> On 17/03/2017 2:27 PM, Chris Plummer wrote: > Ok, time for a new webrev: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot Looks good. Full agreement from me on all the below. Thanks, David > The only thing that has changed since the first webrev are the asserts > added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss > already: > > - The assert should never happen due to the stack size being too small > since it will be at least PTHREAD_STACK_MIN. > - The assert should never happen due to an unaligned stack size because > we always align it to the page size. > - Any assert would therefore be a VM bug and not due to user error. > - No fixing the java launcher. If the user specifies a stack that is > too small, hotspot will already detect this. If the user specifies a > stack size that is large enough but not page aligned, then we just > ignore any error (if the platform doth protest) and the user gets a main > thread with a stack size set to whatever the OS default is. > > I still need to retest (I only ran TooSmallStackSize.java), but figure > getting agreement on the changes first would be best before I bog down > our testing resources. > > thanks, > > Chris > > On 3/15/17 10:03 PM, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8176768 >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >> >> While working on 8175342 I noticed our stack size on xgene was 8mb >> even though I was specifying -Xss72k. It turns out the following code >> was failing: >> >> pthread_attr_setstacksize(&attr, stack_size); >> >> Although we computed a minimum stack size of 72k, so -Xss72k should be >> fine, pthreads on this platform requires the stack be at least 128k, >> so it failed the pthread_attr_setstacksize() call. The end result is >> pthread_attr_setstacksize() had no impact on the thread's stack size, >> and we ended up with the platform default of 8mb. The fix is to round >> up the following variables to PTHREAD_STACK_MIN after computing their >> new values: >> >> _java_thread_min_stack_allowed >> _compiler_thread_min_stack_allowed >> _vm_internal_thread_min_stack_allowed >> >> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >> #defined, and this needs to be done before including OS header files. >> I noticed that on solaris we were using thr_min_stack() elsewhere >> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >> fix. Either way is ugly (the #define or using thr_min_stack()). >> >> And speaking of the existing use of thr_min_stack(), I deleted it. It >> was being applied before any adjustments to the stack sizes had been >> made (rounding and adding red, yellow, and shadow zones). This mean >> the stack ended up being larger than necessary. With the above fix in >> place, we are now applying thr_min_stack() after recomputing the >> minimum stack sizes. If for any reason one of those stack sizes is now >> too small, the correct fix is to adjust the initial stack sizes, not >> apply thr_min_stack() to the initial stack sizes. However, it looks >> like no adjustment is needed. I did something close to our nightly >> testing on all affect platforms, and no new problems turned up. >> >> thanks, >> >> Chris > > From thomas.stuefe at gmail.com Fri Mar 17 06:48:49 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 17 Mar 2017 07:48:49 +0100 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> Message-ID: Hi Chris, please find my answers inline. On Fri, Mar 17, 2017 at 5:27 AM, Chris Plummer wrote: > Ok, time for a new webrev: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot > > The changes look good. > The only thing that has changed since the first webrev are the asserts > added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss > already: > > - The assert should never happen due to the stack size being too small > since it will be at least PTHREAD_STACK_MIN. > - The assert should never happen due to an unaligned stack size because > we always align it to the page size. > As a side note, on AIX we have a complicated page scheme where thread stacks may have a different page size from the java heap or from the primordial thread. But using os::vm_page_size() should be ok, we ensure that os::vm_page_size is always the same or a multiple of the thread stack page size. Would you mind adding the assert to os_aix.cpp too? > - Any assert would therefore be a VM bug and not due to user error. > - No fixing the java launcher. If the user specifies a stack that is too > small, hotspot will already detect this. If the user specifies a stack size > that is large enough but not page aligned, then we just ignore any error > (if the platform doth protest) and the user gets a main thread with a stack > size set to whatever the OS default is. > > I still need to retest (I only ran TooSmallStackSize.java), but figure > getting agreement on the changes first would be best before I bog down our > testing resources. > > thanks, > > Chris > > I am all fine with the changes. Kind Regards, Thomas > > On 3/15/17 10:03 PM, Chris Plummer wrote: > >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8176768 >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >> >> While working on 8175342 I noticed our stack size on xgene was 8mb even >> though I was specifying -Xss72k. It turns out the following code was >> failing: >> >> pthread_attr_setstacksize(&attr, stack_size); >> >> Although we computed a minimum stack size of 72k, so -Xss72k should be >> fine, pthreads on this platform requires the stack be at least 128k, so it >> failed the pthread_attr_setstacksize() call. The end result is >> pthread_attr_setstacksize() had no impact on the thread's stack size, and >> we ended up with the platform default of 8mb. The fix is to round up the >> following variables to PTHREAD_STACK_MIN after computing their new values: >> >> _java_thread_min_stack_allowed >> _compiler_thread_min_stack_allowed >> _vm_internal_thread_min_stack_allowed >> >> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >> #defined, and this needs to be done before including OS header files. I >> noticed that on solaris we were using thr_min_stack() elsewhere instead of >> PTHREAD_STACK_MIN, so I decided to do the same with this fix. Either way is >> ugly (the #define or using thr_min_stack()). >> >> And speaking of the existing use of thr_min_stack(), I deleted it. It was >> being applied before any adjustments to the stack sizes had been made >> (rounding and adding red, yellow, and shadow zones). This mean the stack >> ended up being larger than necessary. With the above fix in place, we are >> now applying thr_min_stack() after recomputing the minimum stack sizes. If >> for any reason one of those stack sizes is now too small, the correct fix >> is to adjust the initial stack sizes, not apply thr_min_stack() to the >> initial stack sizes. However, it looks like no adjustment is needed. I did >> something close to our nightly testing on all affect platforms, and no new >> problems turned up. >> >> thanks, >> >> Chris >> > > > From chris.plummer at oracle.com Fri Mar 17 06:54:03 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 16 Mar 2017 23:54:03 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <22513c52-0150-3b85-acfc-85e4c161e549@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <22513c52-0150-3b85-acfc-85e4c161e549@oracle.com> Message-ID: On 3/16/17 11:26 PM, David Holmes wrote: > On 17/03/2017 2:27 PM, Chris Plummer wrote: >> Ok, time for a new webrev: >> >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot > > Looks good. Full agreement from me on all the below. Hi David, Thanks for your review(s) and all your help and suggestions. Much appreciated. Chris > > Thanks, > David > >> The only thing that has changed since the first webrev are the asserts >> added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss >> already: >> >> - The assert should never happen due to the stack size being too small >> since it will be at least PTHREAD_STACK_MIN. >> - The assert should never happen due to an unaligned stack size because >> we always align it to the page size. >> - Any assert would therefore be a VM bug and not due to user error. >> - No fixing the java launcher. If the user specifies a stack that is >> too small, hotspot will already detect this. If the user specifies a >> stack size that is large enough but not page aligned, then we just >> ignore any error (if the platform doth protest) and the user gets a main >> thread with a stack size set to whatever the OS default is. >> >> I still need to retest (I only ran TooSmallStackSize.java), but figure >> getting agreement on the changes first would be best before I bog down >> our testing resources. >> >> thanks, >> >> Chris >> >> On 3/15/17 10:03 PM, Chris Plummer wrote: >>> Hello, >>> >>> Please review the following: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>> >>> While working on 8175342 I noticed our stack size on xgene was 8mb >>> even though I was specifying -Xss72k. It turns out the following code >>> was failing: >>> >>> pthread_attr_setstacksize(&attr, stack_size); >>> >>> Although we computed a minimum stack size of 72k, so -Xss72k should be >>> fine, pthreads on this platform requires the stack be at least 128k, >>> so it failed the pthread_attr_setstacksize() call. The end result is >>> pthread_attr_setstacksize() had no impact on the thread's stack size, >>> and we ended up with the platform default of 8mb. The fix is to round >>> up the following variables to PTHREAD_STACK_MIN after computing their >>> new values: >>> >>> _java_thread_min_stack_allowed >>> _compiler_thread_min_stack_allowed >>> _vm_internal_thread_min_stack_allowed >>> >>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>> #defined, and this needs to be done before including OS header files. >>> I noticed that on solaris we were using thr_min_stack() elsewhere >>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>> fix. Either way is ugly (the #define or using thr_min_stack()). >>> >>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>> was being applied before any adjustments to the stack sizes had been >>> made (rounding and adding red, yellow, and shadow zones). This mean >>> the stack ended up being larger than necessary. With the above fix in >>> place, we are now applying thr_min_stack() after recomputing the >>> minimum stack sizes. If for any reason one of those stack sizes is now >>> too small, the correct fix is to adjust the initial stack sizes, not >>> apply thr_min_stack() to the initial stack sizes. However, it looks >>> like no adjustment is needed. I did something close to our nightly >>> testing on all affect platforms, and no new problems turned up. >>> >>> thanks, >>> >>> Chris >> >> From volker.simonis at gmail.com Fri Mar 17 08:15:40 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 17 Mar 2017 09:15:40 +0100 Subject: CFW: New hotspot Group Member: Andrew Haley Message-ID: I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. Andrew leads the Java team at Read Hat and is a long standing member of and contributor to the OpenJDK. He's currently an At-Large member of the OpenJDK Governing Board and member of the OpenJDK Porters and Members groups. He's also the Project Lead of the aarch64-port and jdk7u projects and a Committer/Reviewer to various projects ranging from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. As one of the main authors of the aarch64-port he has gained a deep knowledge of the HotSpot internals and has contributed more than 60 changes to the project [4]. Votes are due by March 31, 18:00 CET. Only current Members of the hotspot Group [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list For Lazy Consensus voting instructions, see [2]. Kind Regards, Volker Simonis [1] http://openjdk.java.net/census [2] http://openjdk.java.net/groups/#member-vote [3] http://openjdk.java.net/census#aph [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 From mikael.gerdin at oracle.com Fri Mar 17 08:18:49 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 17 Mar 2017 09:18:49 +0100 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: <1e362b80-6cf1-0ec6-0974-f830c07f44fd@oracle.com> Vote: yes /Mikael On 2017-03-17 09:15, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 > From rwestrel at redhat.com Fri Mar 17 08:32:11 2017 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 17 Mar 2017 09:32:11 +0100 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: Vote: yes Roland. From thomas.schatzl at oracle.com Fri Mar 17 08:36:00 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 17 Mar 2017 09:36:00 +0100 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: <1489739760.4416.2.camel@oracle.com> Vote: yes Thanks, ? Thomas From zoltan.majo at oracle.com Fri Mar 17 08:55:52 2017 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Fri, 17 Mar 2017 09:55:52 +0100 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: Vote: yes. Best regards, Zoltan On 03/17/2017 09:15 AM, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 From tobias.hartmann at oracle.com Fri Mar 17 10:42:50 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 17 Mar 2017 11:42:50 +0100 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: Vote: yes Best regards, Tobias On 17.03.2017 09:15, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 > From shade at redhat.com Fri Mar 17 10:55:40 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 17 Mar 2017 11:55:40 +0100 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: <7a8c0273-4413-6217-6a6c-221b9f1f8a51@redhat.com> Vote: yes -Aleksey On 03/17/2017 09:15 AM, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 > From david.holmes at oracle.com Fri Mar 17 11:54:04 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Mar 2017 21:54:04 +1000 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: <3fdccd28-d4a3-7b04-c23c-e13873f3ec6f@oracle.com> Vote: yes David On 17/03/2017 6:15 PM, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 > From ChrisPhi at LGonQn.Org Fri Mar 17 12:18:05 2017 From: ChrisPhi at LGonQn.Org (Chris Phillips) Date: Fri, 17 Mar 2017 08:18:05 -0400 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: <331313c6-6332-9670-cc45-8e4820d3a065@LGonQn.Org> Vote: yes Chris On 17/03/17 04:15 AM, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 > > From coleen.phillimore at oracle.com Fri Mar 17 12:21:43 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 17 Mar 2017 08:21:43 -0400 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: Vote: yes On 3/17/17 4:15 AM, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 From coleen.phillimore at oracle.com Fri Mar 17 12:29:53 2017 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 17 Mar 2017 08:29:53 -0400 Subject: RFR (S): 8176831: Dead code: function jmm_GetLoadedClasses is not used in jmm_interface In-Reply-To: <0b347619-f5f7-4804-ab3a-1cbb7871be0e@oracle.com> References: <84f65d92-d937-deef-827a-cbc30a03f3e1@oracle.com> <480b2c46-c094-de6e-2b90-05b90b3ea783@oracle.com> <0b347619-f5f7-4804-ab3a-1cbb7871be0e@oracle.com> Message-ID: Serguei, This change looks great! This was one of the SystemDictionary::classes_do function calls that I could not resolve why it didn't want the array classes or anonymous classes, loaded classes in error or redefined scratch classes, or what the purpose of the function was in general. Are uncalled function removals "trivial fixes" only needing one code review? You have two now. Coleen On 3/16/17 9:38 PM, serguei.spitsyn at oracle.com wrote: > I've reloaded the webrev with pulled latest jdk10 changes. > It looks almost the same but has the recent update from Coleen. > > Thanks, > Serguei > > > On 3/16/17 18:20, serguei.spitsyn at oracle.com wrote: >> David, >> >> Thank you for the review! >> >> >> On 3/16/17 18:15, David Holmes wrote: >>> Looks good! >>> >>> Don't forget to update copyright years. >> >> Yes, of course. >> >> >> Thanks, >> Serguei >>> >>> Thanks, >>> David >>> >>> On 17/03/2017 10:59 AM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the jdk 10 fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8176831 >>>> >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8176831-jmm-dead.1/ >>>> >>>> >>>> >>>> >>>> Summary: >>>> >>>> It was found at the review of the 8155672 that the function >>>> jmm_GetLoadedClasses() defined in services/management.cpp >>>> is not really used in the jmm_interface. This function and >>>> dead code associated with its implementation is removed. >>>> >>>> Testing: >>>> The nsk.monitoring and jtreg jdk_management tests are in progress. >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> >>>> >> > From vladimir.x.ivanov at oracle.com Fri Mar 17 12:58:30 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 17 Mar 2017 15:58:30 +0300 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> Message-ID: <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> >> I have the same concern. Can we fix the immediate problem in 9 and >> integrate verification logic in 10? >> > > OK, Tobias is suggesting having verification logic only inside the > intrinsics. Are you suggesting removing that as well? Yes and put them back in 10. > I'm OK with removing all the verification, but that won't reduce the > library changes much. I could undo the renaming to Trusted.getChar, but > we would still have the bounds checks moved into StringUTF16. I suggest to go with a point fix for 9: just add missing range checks. Is AbstractStringBuilder.append() the only affected method? (Sorry, it's hard to say exactly where the problem is by looking at the diff.) I really like the refactoring you propose on jdk side, but there are pieces I'm not sure about. For example, I spotted a repeated range check: jdk/src/java.base/share/classes/java/lang/AbstractStringBuilder.java: public void setCharAt(int index, char ch) { checkIndex(index, count); if (isLatin1() && StringLatin1.canEncode(ch)) { value[index] = (byte)ch; } else { if (isLatin1()) { inflate(); } StringUTF16.putCharSB(value, index, ch); } } Best regards, Vladimir Ivanov From serguei.spitsyn at oracle.com Fri Mar 17 15:15:01 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 17 Mar 2017 08:15:01 -0700 Subject: RFR (S): 8176831: Dead code: function jmm_GetLoadedClasses is not used in jmm_interface In-Reply-To: References: <84f65d92-d937-deef-827a-cbc30a03f3e1@oracle.com> <480b2c46-c094-de6e-2b90-05b90b3ea783@oracle.com> <0b347619-f5f7-4804-ab3a-1cbb7871be0e@oracle.com> Message-ID: Hi Coleen, On 3/17/17 05:29, coleen.phillimore at oracle.com wrote: > Serguei, This change looks great! This was one of the > SystemDictionary::classes_do function calls that I could not resolve > why it didn't want the array classes or anonymous classes, loaded > classes in error or redefined scratch classes, or what the purpose of > the function was in general. Yes, it is nice one of the problematic spots is gone. > > Are uncalled function removals "trivial fixes" only needing one code > review? You have two now. Thank you for the review! It is better to have two anyway. :) Thanks, Serguei > Coleen > > On 3/16/17 9:38 PM, serguei.spitsyn at oracle.com wrote: >> I've reloaded the webrev with pulled latest jdk10 changes. >> It looks almost the same but has the recent update from Coleen. >> >> Thanks, >> Serguei >> >> >> On 3/16/17 18:20, serguei.spitsyn at oracle.com wrote: >>> David, >>> >>> Thank you for the review! >>> >>> >>> On 3/16/17 18:15, David Holmes wrote: >>>> Looks good! >>>> >>>> Don't forget to update copyright years. >>> >>> Yes, of course. >>> >>> >>> Thanks, >>> Serguei >>>> >>>> Thanks, >>>> David >>>> >>>> On 17/03/2017 10:59 AM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the jdk 10 fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8176831 >>>>> >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2017/hotspot/8176831-jmm-dead.1/ >>>>> >>>>> >>>>> >>>>> >>>>> Summary: >>>>> >>>>> It was found at the review of the 8155672 that the function >>>>> jmm_GetLoadedClasses() defined in services/management.cpp >>>>> is not really used in the jmm_interface. This function and >>>>> dead code associated with its implementation is removed. >>>>> >>>>> Testing: >>>>> The nsk.monitoring and jtreg jdk_management tests are in progress. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> >>>>> >>> >> > From jesper.wilhelmsson at oracle.com Fri Mar 17 16:50:44 2017 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Fri, 17 Mar 2017 09:50:44 -0700 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: <553A66FB-2ABC-4A8C-BA69-C5863E34AEAD@oracle.com> Vote: Yes /Jesper > On 17 Mar 2017, at 01:15, Volker Simonis wrote: > > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 From vladimir.kozlov at oracle.com Fri Mar 17 17:30:08 2017 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 17 Mar 2017 10:30:08 -0700 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: <7fda23e2-dbe4-0ec5-1392-fa5990cf6b35@oracle.com> Vote: yes On 3/17/17 1:15 AM, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 > From bob.vandette at oracle.com Fri Mar 17 20:14:45 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 17 Mar 2017 16:14:45 -0400 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: <4E700D62-8D04-47F7-9298-D1ED776BE376@oracle.com> References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <642003bd-a5dc-d1ae-87ff-aca50388cb9c@redhat.com> <4E700D62-8D04-47F7-9298-D1ED776BE376@oracle.com> Message-ID: I checked with the hotspot compiler team and their manager and they are ok with using the labels ?arm64? and ?aarch64? to mark bugs that are specific to 64-bit aarch64 builds that are done using hotspot/src/cpu/arm versus hotspot/src/cpu/aarch64 sources. Please publicize this label?s use throughout interested OpenJDK developers. Bob. > On Mar 16, 2017, at 2:40 PM, Bob Vandette wrote: > > >> On Mar 16, 2017, at 2:27 PM, Andrew Haley wrote: >> >> On 16/03/17 18:03, Bob Vandette wrote: >> >>> I agree that this is an issue but I?m not sure that it?s a show >>> stopper. >>> >>> The Oracle build will not have OpenJDK in the version string which >>> will help to differentiate our binaries from OpenJDK builds. >> >> Right, like I said. >> >>> The bug database field that I think you are describing is only an >>> indication of the architecture that a bug can be reproduced on. It >>> is not meant to describe the sources that were used to produce the >>> binaries or where the binaries came from. That should to be >>> specified elsewhere in the bug report. >> >> OK. I would surely have tried to insist that the version strings were >> different for our two ports at the time your port was committed, but I >> blew my chance. >> >>> I don?t like the idea of listing arm64 in the version string since >>> we are only using arm64 internally to trigger the use of the hotspot >>> ?arm? directory. We?d also end up with lots of incorrect bug >>> entries since folks will fail to use arm64 to report a bug in the >>> Oracle 64-bit ARM port running on an aarch64 based system. >>> >>> If we start putting build configuration information in the version >>> string, then where do we stop. >> >> It's going to be rather horrible, though. How do we reproduce a bug >> if we don't know what port is causing the bug? How do we even ask the >> question if we don't know what the ports are called? I always assumed >> we were "aarch64" and you were "arm64". How are we to ask a user if >> we can't tell them what to look for? >> >> Even if we don't change anything in OpenJDK itself, we'll still have >> to agree on a label to use in the bug database. I don't know what >> labels we should use, but we should agree on them now. Do you have >> any preferences? > > I agree that a label would be very useful. For this purpose, I?m not opposed > to using the arm64 versus aarch64 names. Let me check around to see > if anyone has a better suggestion. > > Bob. > > > >> >> Andrew. > From dean.long at oracle.com Fri Mar 17 20:18:49 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 17 Mar 2017 13:18:49 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> Message-ID: <81d81016-c9df-efbe-6d11-d2ecd9c02090@oracle.com> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: > >>> I have the same concern. Can we fix the immediate problem in 9 and >>> integrate verification logic in 10? >>> >> >> OK, Tobias is suggesting having verification logic only inside the >> intrinsics. Are you suggesting removing that as well? > > Yes and put them back in 10. OK. > >> I'm OK with removing all the verification, but that won't reduce the >> library changes much. I could undo the renaming to Trusted.getChar, but >> we would still have the bounds checks moved into StringUTF16. > > I suggest to go with a point fix for 9: just add missing range checks. > > Is AbstractStringBuilder.append() the only affected method? (Sorry, > it's hard to say exactly where the problem is by looking at the diff.) > In the failing test, yes, it was append, but when I went to fix the problem I found that it was much more wide spread, and there were several methods that were affected. > I really like the refactoring you propose on jdk side, but there are > pieces I'm not sure about. For example, I spotted a repeated range check: > > jdk/src/java.base/share/classes/java/lang/AbstractStringBuilder.java: > public void setCharAt(int index, char ch) { > checkIndex(index, count); > if (isLatin1() && StringLatin1.canEncode(ch)) { > value[index] = (byte)ch; > } else { > if (isLatin1()) { > inflate(); > } > StringUTF16.putCharSB(value, index, ch); > } > } > OK, I did not look for redundant checks. This check is actually not redundant. The "value" array may be oversized, so "count" is supposed to contain the current maximum. For the safety of the intrinsic array access, we check against the array length, but for the API we need to be stricter and check against the character count. However, the checkIndex() here is a good example of what is wrong. Let's say we were checking against value.length instead of "count". Even if checkIndex() succeeds here, based on the current length of "value", we can't trust it because the object is mutable and "value" can change between the checkIndex() and putCharSB(). dl > Best regards, > Vladimir Ivanov From dean.long at oracle.com Fri Mar 17 22:07:02 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 17 Mar 2017 15:07:02 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> Message-ID: <3553280e-6955-a574-8154-19cb52409300@oracle.com> I posted two new versions, webrev.1 keeping the Trusted inner class: http://cr.openjdk.java.net/~dlong/8158168/webrev.1/ and webrev.2 with it removed: http://cr.openjdk.java.net/~dlong/8158168/webrev.2/ dl On 3/17/17 5:58 AM, Vladimir Ivanov wrote: > >>> I have the same concern. Can we fix the immediate problem in 9 and >>> integrate verification logic in 10? >>> >> >> OK, Tobias is suggesting having verification logic only inside the >> intrinsics. Are you suggesting removing that as well? > > Yes and put them back in 10. > >> I'm OK with removing all the verification, but that won't reduce the >> library changes much. I could undo the renaming to Trusted.getChar, but >> we would still have the bounds checks moved into StringUTF16. > > I suggest to go with a point fix for 9: just add missing range checks. From chris.plummer at oracle.com Fri Mar 17 23:11:33 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 17 Mar 2017 16:11:33 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> Message-ID: Looks like this will need some more work since the added asserts are triggering on mac os x (which is the only place we'd currently expect them to assert). The problem is that the code I found that rounds up to the page size is only applied to java threads created by the VM for which the java user specified no stack size. The VM and Compiler thread sizes are not rounded. The failure I saw was with runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java when is specified -XX:CompilerThreadStackSize=9007199254740991. I hit the assert with an EINVAL. The size is not aligned, but it could also be complaining because it is too big. I haven't tried aligning it yet to see. On Linux we do the following: stack_size = align_size_up(stack_size + os::Linux::default_guard_size(thr_type), vm_page_size()); We don't do this on BSD. I think the same on BSD would solve this problem. I'm not sure about adding the guard size. I'll need to see if mac os x has the same pthread bug as linux does. BTW, did you know java users can specify the size of the a new thread's stack: public Thread(ThreadGroup group, Runnable target, String name, long stackSize) { init(group, target, name, stackSize); } Fortunately we already force the stackSize to be at least _java_thread_min_stack_allowed. However, we don't do any OS page rounding on Mac OS X as noted above, and I was able to trigger the assert by creating a thread with size 257k. I'll get another webrev out once I've made the needed fixes. I also have a new test I'd like to add. Chris On 3/16/17 9:27 PM, Chris Plummer wrote: > Ok, time for a new webrev: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot > > The only thing that has changed since the first webrev are the asserts > added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss > already: > > - The assert should never happen due to the stack size being too > small since it will be at least PTHREAD_STACK_MIN. > - The assert should never happen due to an unaligned stack size > because we always align it to the page size. > - Any assert would therefore be a VM bug and not due to user error. > - No fixing the java launcher. If the user specifies a stack that is > too small, hotspot will already detect this. If the user specifies a > stack size that is large enough but not page aligned, then we just > ignore any error (if the platform doth protest) and the user gets a > main thread with a stack size set to whatever the OS default is. > > I still need to retest (I only ran TooSmallStackSize.java), but figure > getting agreement on the changes first would be best before I bog down > our testing resources. > > thanks, > > Chris > > On 3/15/17 10:03 PM, Chris Plummer wrote: >> Hello, >> >> Please review the following: >> >> https://bugs.openjdk.java.net/browse/JDK-8176768 >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >> >> While working on 8175342 I noticed our stack size on xgene was 8mb >> even though I was specifying -Xss72k. It turns out the following code >> was failing: >> >> pthread_attr_setstacksize(&attr, stack_size); >> >> Although we computed a minimum stack size of 72k, so -Xss72k should >> be fine, pthreads on this platform requires the stack be at least >> 128k, so it failed the pthread_attr_setstacksize() call. The end >> result is pthread_attr_setstacksize() had no impact on the thread's >> stack size, and we ended up with the platform default of 8mb. The fix >> is to round up the following variables to PTHREAD_STACK_MIN after >> computing their new values: >> >> _java_thread_min_stack_allowed >> _compiler_thread_min_stack_allowed >> _vm_internal_thread_min_stack_allowed >> >> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >> #defined, and this needs to be done before including OS header files. >> I noticed that on solaris we were using thr_min_stack() elsewhere >> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >> fix. Either way is ugly (the #define or using thr_min_stack()). >> >> And speaking of the existing use of thr_min_stack(), I deleted it. It >> was being applied before any adjustments to the stack sizes had been >> made (rounding and adding red, yellow, and shadow zones). This mean >> the stack ended up being larger than necessary. With the above fix in >> place, we are now applying thr_min_stack() after recomputing the >> minimum stack sizes. If for any reason one of those stack sizes is >> now too small, the correct fix is to adjust the initial stack sizes, >> not apply thr_min_stack() to the initial stack sizes. However, it >> looks like no adjustment is needed. I did something close to our >> nightly testing on all affect platforms, and no new problems turned up. >> >> thanks, >> >> Chris > > From david.holmes at oracle.com Sat Mar 18 02:01:17 2017 From: david.holmes at oracle.com (David Holmes) Date: Sat, 18 Mar 2017 12:01:17 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> Message-ID: On 18/03/2017 9:11 AM, Chris Plummer wrote: > Looks like this will need some more work since the added asserts are > triggering on mac os x (which is the only place we'd currently expect > them to assert). > > The problem is that the code I found that rounds up to the page size is > only applied to java threads created by the VM for which the java user > specified no stack size. The VM and Compiler thread sizes are not > rounded. The failure I saw was with > runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java when is > specified -XX:CompilerThreadStackSize=9007199254740991. I hit the assert > with an EINVAL. The size is not aligned, but it could also be > complaining because it is too big. I haven't tried aligning it yet to see. > > On Linux we do the following: > > stack_size = align_size_up(stack_size + > os::Linux::default_guard_size(thr_type), vm_page_size()); > > We don't do this on BSD. I think the same on BSD would solve this > problem. I'm not sure about adding the guard size. I'll need to see if > mac os x has the same pthread bug as linux does. At this stage I would only deal with alignment issues. IIRC the guard issue only affected Linux. > BTW, did you know java users can specify the size of the a new thread's > stack: Yes I mentioned that in another reply - wondering whether we suitably check and aligned such requests. > public Thread(ThreadGroup group, Runnable target, String name, > long stackSize) { > init(group, target, name, stackSize); > } > > Fortunately we already force the stackSize to be at least > _java_thread_min_stack_allowed. However, we don't do any OS page > rounding on Mac OS X as noted above, and I was able to trigger the > assert by creating a thread with size 257k. Note this means that OSX stack logic is broken because it will currently be silently failing due to EINVAL! > I'll get another webrev out once I've made the needed fixes. I also have > a new test I'd like to add. Ok. Thanks, David > Chris > > On 3/16/17 9:27 PM, Chris Plummer wrote: >> Ok, time for a new webrev: >> >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >> >> The only thing that has changed since the first webrev are the asserts >> added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss >> already: >> >> - The assert should never happen due to the stack size being too >> small since it will be at least PTHREAD_STACK_MIN. >> - The assert should never happen due to an unaligned stack size >> because we always align it to the page size. >> - Any assert would therefore be a VM bug and not due to user error. >> - No fixing the java launcher. If the user specifies a stack that is >> too small, hotspot will already detect this. If the user specifies a >> stack size that is large enough but not page aligned, then we just >> ignore any error (if the platform doth protest) and the user gets a >> main thread with a stack size set to whatever the OS default is. >> >> I still need to retest (I only ran TooSmallStackSize.java), but figure >> getting agreement on the changes first would be best before I bog down >> our testing resources. >> >> thanks, >> >> Chris >> >> On 3/15/17 10:03 PM, Chris Plummer wrote: >>> Hello, >>> >>> Please review the following: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>> >>> While working on 8175342 I noticed our stack size on xgene was 8mb >>> even though I was specifying -Xss72k. It turns out the following code >>> was failing: >>> >>> pthread_attr_setstacksize(&attr, stack_size); >>> >>> Although we computed a minimum stack size of 72k, so -Xss72k should >>> be fine, pthreads on this platform requires the stack be at least >>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>> result is pthread_attr_setstacksize() had no impact on the thread's >>> stack size, and we ended up with the platform default of 8mb. The fix >>> is to round up the following variables to PTHREAD_STACK_MIN after >>> computing their new values: >>> >>> _java_thread_min_stack_allowed >>> _compiler_thread_min_stack_allowed >>> _vm_internal_thread_min_stack_allowed >>> >>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>> #defined, and this needs to be done before including OS header files. >>> I noticed that on solaris we were using thr_min_stack() elsewhere >>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>> fix. Either way is ugly (the #define or using thr_min_stack()). >>> >>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>> was being applied before any adjustments to the stack sizes had been >>> made (rounding and adding red, yellow, and shadow zones). This mean >>> the stack ended up being larger than necessary. With the above fix in >>> place, we are now applying thr_min_stack() after recomputing the >>> minimum stack sizes. If for any reason one of those stack sizes is >>> now too small, the correct fix is to adjust the initial stack sizes, >>> not apply thr_min_stack() to the initial stack sizes. However, it >>> looks like no adjustment is needed. I did something close to our >>> nightly testing on all affect platforms, and no new problems turned up. >>> >>> thanks, >>> >>> Chris >> >> > From chris.plummer at oracle.com Sat Mar 18 03:17:48 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 17 Mar 2017 20:17:48 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> Message-ID: <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> On 3/17/17 7:01 PM, David Holmes wrote: > On 18/03/2017 9:11 AM, Chris Plummer wrote: >> Looks like this will need some more work since the added asserts are >> triggering on mac os x (which is the only place we'd currently expect >> them to assert). >> >> The problem is that the code I found that rounds up to the page size is >> only applied to java threads created by the VM for which the java user >> specified no stack size. The VM and Compiler thread sizes are not >> rounded. The failure I saw was with >> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java when is >> specified -XX:CompilerThreadStackSize=9007199254740991. I hit the assert >> with an EINVAL. The size is not aligned, but it could also be >> complaining because it is too big. I haven't tried aligning it yet to >> see. >> >> On Linux we do the following: >> >> stack_size = align_size_up(stack_size + >> os::Linux::default_guard_size(thr_type), vm_page_size()); >> >> We don't do this on BSD. I think the same on BSD would solve this >> problem. I'm not sure about adding the guard size. I'll need to see if >> mac os x has the same pthread bug as linux does. > > At this stage I would only deal with alignment issues. IIRC the guard > issue only affected Linux. Yes, that's what I eventually concluded. I put the fix in os::Posix::get_initial_stack_size() in os_posix.cpp, and only did the page aligning, not add the guard page. That way all Posix ports are fixed in one place. It seems to be working. > >> BTW, did you know java users can specify the size of the a new thread's >> stack: > > Yes I mentioned that in another reply - wondering whether we suitably > check and aligned such requests. No we don't. Below I mentioned I was able to trigger the assert with a 257k stack size. I guess I wasn't clear that I did that from Java. I have a new test to add that will be testing this, plus the 9007199254740991 stack size (which fails to create the thread with an OOME, but that's acceptable). The fix I mention above in os::Posix::get_initial_stack_size() takes care of this issue also. > >> public Thread(ThreadGroup group, Runnable target, String name, >> long stackSize) { >> init(group, target, name, stackSize); >> } >> >> Fortunately we already force the stackSize to be at least >> _java_thread_min_stack_allowed. However, we don't do any OS page >> rounding on Mac OS X as noted above, and I was able to trigger the >> assert by creating a thread with size 257k. > > Note this means that OSX stack logic is broken because it will > currently be silently failing due to EINVAL! Yes, that is correct. Chris > >> I'll get another webrev out once I've made the needed fixes. I also have >> a new test I'd like to add. > > Ok. > > Thanks, > David > >> Chris >> >> On 3/16/17 9:27 PM, Chris Plummer wrote: >>> Ok, time for a new webrev: >>> >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>> >>> The only thing that has changed since the first webrev are the asserts >>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss >>> already: >>> >>> - The assert should never happen due to the stack size being too >>> small since it will be at least PTHREAD_STACK_MIN. >>> - The assert should never happen due to an unaligned stack size >>> because we always align it to the page size. >>> - Any assert would therefore be a VM bug and not due to user error. >>> - No fixing the java launcher. If the user specifies a stack that is >>> too small, hotspot will already detect this. If the user specifies a >>> stack size that is large enough but not page aligned, then we just >>> ignore any error (if the platform doth protest) and the user gets a >>> main thread with a stack size set to whatever the OS default is. >>> >>> I still need to retest (I only ran TooSmallStackSize.java), but figure >>> getting agreement on the changes first would be best before I bog down >>> our testing resources. >>> >>> thanks, >>> >>> Chris >>> >>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>> Hello, >>>> >>>> Please review the following: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>> >>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>> even though I was specifying -Xss72k. It turns out the following code >>>> was failing: >>>> >>>> pthread_attr_setstacksize(&attr, stack_size); >>>> >>>> Although we computed a minimum stack size of 72k, so -Xss72k should >>>> be fine, pthreads on this platform requires the stack be at least >>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>> result is pthread_attr_setstacksize() had no impact on the thread's >>>> stack size, and we ended up with the platform default of 8mb. The fix >>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>> computing their new values: >>>> >>>> _java_thread_min_stack_allowed >>>> _compiler_thread_min_stack_allowed >>>> _vm_internal_thread_min_stack_allowed >>>> >>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>> #defined, and this needs to be done before including OS header files. >>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>> >>>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>>> was being applied before any adjustments to the stack sizes had been >>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>> the stack ended up being larger than necessary. With the above fix in >>>> place, we are now applying thr_min_stack() after recomputing the >>>> minimum stack sizes. If for any reason one of those stack sizes is >>>> now too small, the correct fix is to adjust the initial stack sizes, >>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>> looks like no adjustment is needed. I did something close to our >>>> nightly testing on all affect platforms, and no new problems turned >>>> up. >>>> >>>> thanks, >>>> >>>> Chris >>> >>> >> From jesper.wilhelmsson at oracle.com Sat Mar 18 04:26:49 2017 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Fri, 17 Mar 2017 21:26:49 -0700 Subject: RFR (XS): JDK-8177070 - Quarantine tests due to JDK-8177069 Message-ID: <26031984-4D21-4D0A-B874-64224FA52953@oracle.com> Hi, Four AOT tests was not prepared to run on windows and are now failing in JPRT when I merge recent JDK 9 changes into JDK 10. I'll quarantine the tests for now to be able to integrate 10/10 to 10/hs. Bug: https://bugs.openjdk.java.net/browse/JDK-8177070 Patch for jdk10/hs/hotspot: diff --git a/test/ProblemList.txt b/test/ProblemList.txt --- a/test/ProblemList.txt +++ b/test/ProblemList.txt @@ -106,6 +106,11 @@ compiler/aot/calls/fromCompiled/CompiledInvokeDynamic2AotTest.java 8175791 windows-all compiler/aot/DeoptimizationTest.java 8175791 windows-all +# aot tests failing due to JDK-8177069 +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/SearchPathTest.java +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/jar/JarSourceProviderTest.java +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/module/ModuleSourceProviderTest.java ############################################################################# From igor.veresov at oracle.com Sat Mar 18 04:59:49 2017 From: igor.veresov at oracle.com (Igor Veresov) Date: Fri, 17 Mar 2017 21:59:49 -0700 Subject: RFR (XS): JDK-8177070 - Quarantine tests due to JDK-8177069 In-Reply-To: <26031984-4D21-4D0A-B874-64224FA52953@oracle.com> References: <26031984-4D21-4D0A-B874-64224FA52953@oracle.com> Message-ID: <2C642D36-B78D-4B0B-A226-CA0517152DE0@oracle.com> Looks good! igor > On Mar 17, 2017, at 9:26 PM, jesper.wilhelmsson at oracle.com wrote: > > Hi, > > Four AOT tests was not prepared to run on windows and are now failing in JPRT when I merge recent JDK 9 changes into JDK 10. > I'll quarantine the tests for now to be able to integrate 10/10 to 10/hs. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8177070 > Patch for jdk10/hs/hotspot: > > diff --git a/test/ProblemList.txt b/test/ProblemList.txt > --- a/test/ProblemList.txt > +++ b/test/ProblemList.txt > @@ -106,6 +106,11 @@ > compiler/aot/calls/fromCompiled/CompiledInvokeDynamic2AotTest.java 8175791 windows-all > compiler/aot/DeoptimizationTest.java 8175791 windows-all > > +# aot tests failing due to JDK-8177069 > +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java > +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/SearchPathTest.java > +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/jar/JarSourceProviderTest.java > +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/module/ModuleSourceProviderTest.java > > ############################################################################# > From chris.plummer at oracle.com Sat Mar 18 06:37:59 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 17 Mar 2017 23:37:59 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: On 3/17/17 8:17 PM, Chris Plummer wrote: > On 3/17/17 7:01 PM, David Holmes wrote: >> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>> Looks like this will need some more work since the added asserts are >>> triggering on mac os x (which is the only place we'd currently expect >>> them to assert). >>> >>> The problem is that the code I found that rounds up to the page size is >>> only applied to java threads created by the VM for which the java user >>> specified no stack size. The VM and Compiler thread sizes are not >>> rounded. The failure I saw was with >>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>> when is >>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit the >>> assert >>> with an EINVAL. The size is not aligned, but it could also be >>> complaining because it is too big. I haven't tried aligning it yet >>> to see. >>> >>> On Linux we do the following: >>> >>> stack_size = align_size_up(stack_size + >>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>> >>> We don't do this on BSD. I think the same on BSD would solve this >>> problem. I'm not sure about adding the guard size. I'll need to see if >>> mac os x has the same pthread bug as linux does. >> >> At this stage I would only deal with alignment issues. IIRC the guard >> issue only affected Linux. > Yes, that's what I eventually concluded. I put the fix in > os::Posix::get_initial_stack_size() in os_posix.cpp, and only did the > page aligning, not add the guard page. That way all Posix ports are > fixed in one place. It seems to be working. >> >>> BTW, did you know java users can specify the size of the a new thread's >>> stack: >> >> Yes I mentioned that in another reply - wondering whether we suitably >> check and aligned such requests. > No we don't. Below I mentioned I was able to trigger the assert with a > 257k stack size. I guess I wasn't clear that I did that from Java. I > have a new test to add that will be testing this, plus the > 9007199254740991 stack size (which fails to create the thread with an > OOME, but that's acceptable). The fix I mention above in > os::Posix::get_initial_stack_size() takes care of this issue also. Rounding up triggers a new assert, this time on 32-bit x86 and arm. I should have clarified it's 9007199254740991 "K", which is 9223372036854774784. Unfortunately on 32bit systems that is asserting with EINVAL. I think we need to do a better job of dealing with 32-bit size_t values: jlong java_lang_Thread::stackSize(oop java_thread) { if (_stackSize_offset > 0) { return java_thread->long_field(_stackSize_offset); } else { return 0; } } jlong size = java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); // Allocate the C++ Thread structure and create the native thread. The // stack size retrieved from java is signed, but the constructor takes // size_t (an unsigned type), so avoid passing negative values which would // result in really large stacks. size_t sz = size > 0 ? (size_t) size : 0; native_thread = new JavaThread(&thread_entry, sz); 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit MAX_INT), which is 0xfffffc00 when cast to a size_t on a 32-bit system (close to 32-bit MAX_UINT). Round it up to the 4k page size and you get 0, which I guess pthread_attr_setstacksize() doesn't like. So I think more processing of the size is needed here. Maybe in os::create_thread() we should check for 0 after rounding up, and subtract the os page size if it is 0. However, I think we should also avoid truncating on 32-bit to what is basically some random number. Maybe if "size" (a jlong) is greater than UINT_MAX, then just set "sz" (a size_t) it to UINT_MAX. Chris >> >>> public Thread(ThreadGroup group, Runnable target, String name, >>> long stackSize) { >>> init(group, target, name, stackSize); >>> } >>> >>> Fortunately we already force the stackSize to be at least >>> _java_thread_min_stack_allowed. However, we don't do any OS page >>> rounding on Mac OS X as noted above, and I was able to trigger the >>> assert by creating a thread with size 257k. >> >> Note this means that OSX stack logic is broken because it will >> currently be silently failing due to EINVAL! > Yes, that is correct. > > Chris >> >>> I'll get another webrev out once I've made the needed fixes. I also >>> have >>> a new test I'd like to add. >> >> Ok. >> >> Thanks, >> David >> >>> Chris >>> >>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>> Ok, time for a new webrev: >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>> >>>> The only thing that has changed since the first webrev are the asserts >>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss >>>> already: >>>> >>>> - The assert should never happen due to the stack size being too >>>> small since it will be at least PTHREAD_STACK_MIN. >>>> - The assert should never happen due to an unaligned stack size >>>> because we always align it to the page size. >>>> - Any assert would therefore be a VM bug and not due to user error. >>>> - No fixing the java launcher. If the user specifies a stack that is >>>> too small, hotspot will already detect this. If the user specifies a >>>> stack size that is large enough but not page aligned, then we just >>>> ignore any error (if the platform doth protest) and the user gets a >>>> main thread with a stack size set to whatever the OS default is. >>>> >>>> I still need to retest (I only ran TooSmallStackSize.java), but figure >>>> getting agreement on the changes first would be best before I bog down >>>> our testing resources. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>> Hello, >>>>> >>>>> Please review the following: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>> >>>>> >>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>> even though I was specifying -Xss72k. It turns out the following code >>>>> was failing: >>>>> >>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>> >>>>> Although we computed a minimum stack size of 72k, so -Xss72k should >>>>> be fine, pthreads on this platform requires the stack be at least >>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>> result is pthread_attr_setstacksize() had no impact on the thread's >>>>> stack size, and we ended up with the platform default of 8mb. The fix >>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>> computing their new values: >>>>> >>>>> _java_thread_min_stack_allowed >>>>> _compiler_thread_min_stack_allowed >>>>> _vm_internal_thread_min_stack_allowed >>>>> >>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>> #defined, and this needs to be done before including OS header files. >>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>> >>>>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>>>> was being applied before any adjustments to the stack sizes had been >>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>> the stack ended up being larger than necessary. With the above fix in >>>>> place, we are now applying thr_min_stack() after recomputing the >>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>> now too small, the correct fix is to adjust the initial stack sizes, >>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>> looks like no adjustment is needed. I did something close to our >>>>> nightly testing on all affect platforms, and no new problems >>>>> turned up. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>> >>>> >>> > From jesper.wilhelmsson at oracle.com Sat Mar 18 07:06:49 2017 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Sat, 18 Mar 2017 00:06:49 -0700 Subject: RFR (XS): JDK-8177070 - Quarantine tests due to JDK-8177069 In-Reply-To: <2C642D36-B78D-4B0B-A226-CA0517152DE0@oracle.com> References: <26031984-4D21-4D0A-B874-64224FA52953@oracle.com> <2C642D36-B78D-4B0B-A226-CA0517152DE0@oracle.com> Message-ID: <7BA4A7F4-A5BB-4EF8-8CC2-5B9AAC070F96@oracle.com> Thanks! /Jesper > On 17 Mar 2017, at 21:59, Igor Veresov wrote: > > Looks good! > > igor > >> On Mar 17, 2017, at 9:26 PM, jesper.wilhelmsson at oracle.com wrote: >> >> Hi, >> >> Four AOT tests was not prepared to run on windows and are now failing in JPRT when I merge recent JDK 9 changes into JDK 10. >> I'll quarantine the tests for now to be able to integrate 10/10 to 10/hs. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8177070 >> Patch for jdk10/hs/hotspot: >> >> diff --git a/test/ProblemList.txt b/test/ProblemList.txt >> --- a/test/ProblemList.txt >> +++ b/test/ProblemList.txt >> @@ -106,6 +106,11 @@ >> compiler/aot/calls/fromCompiled/CompiledInvokeDynamic2AotTest.java 8175791 windows-all >> compiler/aot/DeoptimizationTest.java 8175791 windows-all >> >> +# aot tests failing due to JDK-8177069 >> +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java >> +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/SearchPathTest.java >> +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/jar/JarSourceProviderTest.java >> +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/module/ModuleSourceProviderTest.java >> >> ############################################################################# >> > From thomas.stuefe at gmail.com Sat Mar 18 07:23:17 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 18 Mar 2017 08:23:17 +0100 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: Hi Chris, On Sat, Mar 18, 2017 at 7:37 AM, Chris Plummer wrote: > On 3/17/17 8:17 PM, Chris Plummer wrote: > >> On 3/17/17 7:01 PM, David Holmes wrote: >> >>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>> >>>> Looks like this will need some more work since the added asserts are >>>> triggering on mac os x (which is the only place we'd currently expect >>>> them to assert). >>>> >>>> The problem is that the code I found that rounds up to the page size is >>>> only applied to java threads created by the VM for which the java user >>>> specified no stack size. The VM and Compiler thread sizes are not >>>> rounded. The failure I saw was with >>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java when >>>> is >>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit the >>>> assert >>>> with an EINVAL. The size is not aligned, but it could also be >>>> complaining because it is too big. I haven't tried aligning it yet to >>>> see. >>>> >>>> On Linux we do the following: >>>> >>>> stack_size = align_size_up(stack_size + >>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>> >>>> We don't do this on BSD. I think the same on BSD would solve this >>>> problem. I'm not sure about adding the guard size. I'll need to see if >>>> mac os x has the same pthread bug as linux does. >>>> >>> >>> At this stage I would only deal with alignment issues. IIRC the guard >>> issue only affected Linux. >>> >> Yes, that's what I eventually concluded. I put the fix in >> os::Posix::get_initial_stack_size() in os_posix.cpp, and only did the >> page aligning, not add the guard page. That way all Posix ports are fixed >> in one place. It seems to be working. >> >>> >>> BTW, did you know java users can specify the size of the a new thread's >>>> stack: >>>> >>> >>> Yes I mentioned that in another reply - wondering whether we suitably >>> check and aligned such requests. >>> >> No we don't. Below I mentioned I was able to trigger the assert with a >> 257k stack size. I guess I wasn't clear that I did that from Java. I have a >> new test to add that will be testing this, plus the 9007199254740991 stack >> size (which fails to create the thread with an OOME, but that's >> acceptable). The fix I mention above in os::Posix::get_initial_stack_size() >> takes care of this issue also. >> > Rounding up triggers a new assert, this time on 32-bit x86 and arm. > > I should have clarified it's 9007199254740991 "K", which is > 9223372036854774784. Unfortunately on 32bit systems that is asserting with > EINVAL. I think we need to do a better job of dealing with 32-bit size_t > values: > > jlong java_lang_Thread::stackSize(oop java_thread) { > if (_stackSize_offset > 0) { > return java_thread->long_field(_stackSize_offset); > } else { > return 0; > } > } > > jlong size = > java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); > // Allocate the C++ Thread structure and create the native thread. > The > // stack size retrieved from java is signed, but the constructor > takes > // size_t (an unsigned type), so avoid passing negative values which > would > // result in really large stacks. > size_t sz = size > 0 ? (size_t) size : 0; > native_thread = new JavaThread(&thread_entry, sz); > > 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit MAX_INT), which > is 0xfffffc00 when cast to a size_t on a 32-bit system (close to 32-bit > MAX_UINT). Round it up to the 4k page size and you get 0, which I guess > pthread_attr_setstacksize() doesn't like. So I think more processing of the > size is needed here. Maybe in os::create_thread() we should check for 0 > after rounding up, and subtract the os page size if it is 0. However, I > think we should also avoid truncating on 32-bit to what is basically some > random number. Maybe if "size" (a jlong) is greater than UINT_MAX, then > just set "sz" (a size_t) it to UINT_MAX. > > Chris > > IMHO both solutions are fine: if cast to size_t would truncate the value, either to cap the value to SIZE_MAX (to be portable, instead of UINT_MAX), maybe minus one page size to forestall any rounding up. Then let the OS fail with ENOMEM and handle that as usual. Alternatively fail immediately with OOME, which will happen anyway, so we save some cycles. Funny, I never get suspicious of (size_t) casts, because I am too used to 64bit. I did a quick grep for (size_t) casts in the hotspot sources, there are quite a few. Probably overwhelmingly harmless, but it may be worth checking them nevertheless. ... (took a cursory glance at the grep results, found one example of casting jlong to size_t, see JVM_ENTRY (jlong , jmm_SetPoolThreshold (JNIEnv * env , jobject obj , jmmThresholdType type , jlong threshold )) in management.cpp). So, its an issue. Kind Regards, Thomas > > >>> public Thread(ThreadGroup group, Runnable target, String name, >>>> long stackSize) { >>>> init(group, target, name, stackSize); >>>> } >>>> >>>> Fortunately we already force the stackSize to be at least >>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>> assert by creating a thread with size 257k. >>>> >>> >>> Note this means that OSX stack logic is broken because it will currently >>> be silently failing due to EINVAL! >>> >> Yes, that is correct. >> >> Chris >> >>> >>> I'll get another webrev out once I've made the needed fixes. I also have >>>> a new test I'd like to add. >>>> >>> >>> Ok. >>> >>> Thanks, >>> David >>> >>> Chris >>>> >>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>> >>>>> Ok, time for a new webrev: >>>>> >>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>> >>>>> The only thing that has changed since the first webrev are the asserts >>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss >>>>> already: >>>>> >>>>> - The assert should never happen due to the stack size being too >>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>> - The assert should never happen due to an unaligned stack size >>>>> because we always align it to the page size. >>>>> - Any assert would therefore be a VM bug and not due to user error. >>>>> - No fixing the java launcher. If the user specifies a stack that is >>>>> too small, hotspot will already detect this. If the user specifies a >>>>> stack size that is large enough but not page aligned, then we just >>>>> ignore any error (if the platform doth protest) and the user gets a >>>>> main thread with a stack size set to whatever the OS default is. >>>>> >>>>> I still need to retest (I only ran TooSmallStackSize.java), but figure >>>>> getting agreement on the changes first would be best before I bog down >>>>> our testing resources. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> Please review the following: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webr >>>>>> ev.hotspot >>>>>> >>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>> even though I was specifying -Xss72k. It turns out the following code >>>>>> was failing: >>>>>> >>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>> >>>>>> Although we computed a minimum stack size of 72k, so -Xss72k should >>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>> result is pthread_attr_setstacksize() had no impact on the thread's >>>>>> stack size, and we ended up with the platform default of 8mb. The fix >>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>> computing their new values: >>>>>> >>>>>> _java_thread_min_stack_allowed >>>>>> _compiler_thread_min_stack_allowed >>>>>> _vm_internal_thread_min_stack_allowed >>>>>> >>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>>> #defined, and this needs to be done before including OS header files. >>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>> >>>>>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>>>>> was being applied before any adjustments to the stack sizes had been >>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>> the stack ended up being larger than necessary. With the above fix in >>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>> now too small, the correct fix is to adjust the initial stack sizes, >>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>> looks like no adjustment is needed. I did something close to our >>>>>> nightly testing on all affect platforms, and no new problems turned >>>>>> up. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>> >>>>> >>>>> >>>> >> > From igor.ignatyev at oracle.com Sat Mar 18 16:54:46 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sat, 18 Mar 2017 09:54:46 -0700 Subject: RFR (XS): JDK-8177070 - Quarantine tests due to JDK-8177069 In-Reply-To: <26031984-4D21-4D0A-B874-64224FA52953@oracle.com> References: <26031984-4D21-4D0A-B874-64224FA52953@oracle.com> Message-ID: <36AD999D-9568-44CA-90E9-B767A6A55B4D@oracle.com> Hi Jesper, The right syntax for quarantining tests in ProblemList.txt is , so it will be > compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java 8177069 generic-all > ? And since they fail only on windows, it should be > compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java 8177069 windows-all > compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/SearchPathTest.java 8177069 windows-all > compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/jar/JarSourceProviderTest.java 8177069 windows-all > compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/module/ModuleSourceProviderTest.java 8177069 windows-all Thanks, ? Igor > On Mar 17, 2017, at 9:26 PM, jesper.wilhelmsson at oracle.com wrote: > > Hi, > > Four AOT tests was not prepared to run on windows and are now failing in JPRT when I merge recent JDK 9 changes into JDK 10. > I'll quarantine the tests for now to be able to integrate 10/10 to 10/hs. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8177070 > Patch for jdk10/hs/hotspot: > > diff --git a/test/ProblemList.txt b/test/ProblemList.txt > --- a/test/ProblemList.txt > +++ b/test/ProblemList.txt > @@ -106,6 +106,11 @@ > compiler/aot/calls/fromCompiled/CompiledInvokeDynamic2AotTest.java 8175791 windows-all > compiler/aot/DeoptimizationTest.java 8175791 windows-all > > +# aot tests failing due to JDK-8177069 > +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java > +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/SearchPathTest.java > +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/jar/JarSourceProviderTest.java > +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/module/ModuleSourceProviderTest.java > > ############################################################################# > From jesper.wilhelmsson at oracle.com Mon Mar 20 06:55:17 2017 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Mon, 20 Mar 2017 07:55:17 +0100 Subject: RFR (XS): JDK-8177070 - Quarantine tests due to JDK-8177069 In-Reply-To: <36AD999D-9568-44CA-90E9-B767A6A55B4D@oracle.com> References: <26031984-4D21-4D0A-B874-64224FA52953@oracle.com> <36AD999D-9568-44CA-90E9-B767A6A55B4D@oracle.com> Message-ID: <27C3BCA2-07B0-4809-9BF6-062BC4B29011@oracle.com> Thanks for catching that Igor! /Jesper > On 18 Mar 2017, at 17:54, Igor Ignatyev wrote: > > Hi Jesper, > > The right syntax for quarantining tests in ProblemList.txt is , so it will be > >> compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java 8177069 generic-all >> ? > And since they fail only on windows, it should be >> compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java 8177069 windows-all >> compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/SearchPathTest.java 8177069 windows-all >> compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/jar/JarSourceProviderTest.java 8177069 windows-all >> compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/module/ModuleSourceProviderTest.java 8177069 windows-all > > Thanks, > ? Igor > >> On Mar 17, 2017, at 9:26 PM, jesper.wilhelmsson at oracle.com wrote: >> >> Hi, >> >> Four AOT tests was not prepared to run on windows and are now failing in JPRT when I merge recent JDK 9 changes into JDK 10. >> I'll quarantine the tests for now to be able to integrate 10/10 to 10/hs. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8177070 >> Patch for jdk10/hs/hotspot: >> >> diff --git a/test/ProblemList.txt b/test/ProblemList.txt >> --- a/test/ProblemList.txt >> +++ b/test/ProblemList.txt >> @@ -106,6 +106,11 @@ >> compiler/aot/calls/fromCompiled/CompiledInvokeDynamic2AotTest.java 8175791 windows-all >> compiler/aot/DeoptimizationTest.java 8175791 windows-all >> >> +# aot tests failing due to JDK-8177069 >> +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/ClassSourceTest.java >> +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/SearchPathTest.java >> +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/jar/JarSourceProviderTest.java >> +compiler/aot/jdk.tools.jaotc.test/src/jdk/tools/jaotc/test/collect/module/ModuleSourceProviderTest.java >> >> ############################################################################# >> > From stefan.karlsson at oracle.com Mon Mar 20 14:28:55 2017 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 20 Mar 2017 15:28:55 +0100 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: <4af76518-7800-baa4-c048-056c056fda17@oracle.com> Vote: yes StefanK On 2017-03-17 09:15, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 > From daniel.daugherty at oracle.com Mon Mar 20 22:27:21 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 20 Mar 2017 16:27:21 -0600 Subject: CFW: New hotspot Group Member: Andrew Haley In-Reply-To: References: Message-ID: Vote: yes Dan On 3/17/17 2:15 AM, Volker Simonis wrote: > I hereby nominate Andrew Haley (aph [3]) to Membership in the hotspot Group. > > Andrew leads the Java team at Read Hat and is a long standing member > of and contributor to the OpenJDK. He's currently an At-Large member > of the OpenJDK Governing Board and member of the OpenJDK Porters and > Members groups. He's also the Project Lead of the aarch64-port and > jdk7u projects and a Committer/Reviewer to various projects ranging > from jdk6 to jdk10 and including aarch32-port, icedtea and shenandoah. > As one of the main authors of the aarch64-port he has gained a deep > knowledge of the HotSpot internals and has contributed more than 60 > changes to the project [4]. > > Votes are due by March 31, 18:00 CET. > > Only current Members of the hotspot Group [1] are eligible to vote on > this nomination. Votes must be cast in the open by replying to this > mailing list > > For Lazy Consensus voting instructions, see [2]. > > Kind Regards, > Volker Simonis > > [1] http://openjdk.java.net/census > [2] http://openjdk.java.net/groups/#member-vote > [3] http://openjdk.java.net/census#aph > [4] http://hg.openjdk.java.net/jdk9/hs/hotspot/log?revcount=125&rev=%28author%28%22aph%22%29%29 From chris.plummer at oracle.com Mon Mar 20 23:29:13 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 20 Mar 2017 16:29:13 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: On 3/17/17 11:37 PM, Chris Plummer wrote: > On 3/17/17 8:17 PM, Chris Plummer wrote: >> On 3/17/17 7:01 PM, David Holmes wrote: >>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>> Looks like this will need some more work since the added asserts are >>>> triggering on mac os x (which is the only place we'd currently expect >>>> them to assert). >>>> >>>> The problem is that the code I found that rounds up to the page >>>> size is >>>> only applied to java threads created by the VM for which the java user >>>> specified no stack size. The VM and Compiler thread sizes are not >>>> rounded. The failure I saw was with >>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>> when is >>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit the >>>> assert >>>> with an EINVAL. The size is not aligned, but it could also be >>>> complaining because it is too big. I haven't tried aligning it yet >>>> to see. >>>> >>>> On Linux we do the following: >>>> >>>> stack_size = align_size_up(stack_size + >>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>> >>>> We don't do this on BSD. I think the same on BSD would solve this >>>> problem. I'm not sure about adding the guard size. I'll need to see if >>>> mac os x has the same pthread bug as linux does. >>> >>> At this stage I would only deal with alignment issues. IIRC the >>> guard issue only affected Linux. >> Yes, that's what I eventually concluded. I put the fix in >> os::Posix::get_initial_stack_size() in os_posix.cpp, and only did the >> page aligning, not add the guard page. That way all Posix ports are >> fixed in one place. It seems to be working. >>> >>>> BTW, did you know java users can specify the size of the a new >>>> thread's >>>> stack: >>> >>> Yes I mentioned that in another reply - wondering whether we >>> suitably check and aligned such requests. >> No we don't. Below I mentioned I was able to trigger the assert with >> a 257k stack size. I guess I wasn't clear that I did that from Java. >> I have a new test to add that will be testing this, plus the >> 9007199254740991 stack size (which fails to create the thread with an >> OOME, but that's acceptable). The fix I mention above in >> os::Posix::get_initial_stack_size() takes care of this issue also. > Rounding up triggers a new assert, this time on 32-bit x86 and arm. > > I should have clarified it's 9007199254740991 "K", which is > 9223372036854774784. Unfortunately on 32bit systems that is asserting > with EINVAL. I think we need to do a better job of dealing with 32-bit > size_t values: > > jlong java_lang_Thread::stackSize(oop java_thread) { > if (_stackSize_offset > 0) { > return java_thread->long_field(_stackSize_offset); > } else { > return 0; > } > } > > jlong size = > java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); > // Allocate the C++ Thread structure and create the native > thread. The > // stack size retrieved from java is signed, but the constructor > takes > // size_t (an unsigned type), so avoid passing negative values > which would > // result in really large stacks. > size_t sz = size > 0 ? (size_t) size : 0; > native_thread = new JavaThread(&thread_entry, sz); > > 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit MAX_INT), > which is 0xfffffc00 when cast to a size_t on a 32-bit system (close to > 32-bit MAX_UINT). Round it up to the 4k page size and you get 0, which > I guess pthread_attr_setstacksize() doesn't like. So I think more > processing of the size is needed here. Maybe in os::create_thread() we > should check for 0 after rounding up, and subtract the os page size if > it is 0. However, I think we should also avoid truncating on 32-bit to > what is basically some random number. Maybe if "size" (a jlong) is > greater than UINT_MAX, then just set "sz" (a size_t) it to UINT_MAX. > Ok, I think I have this all worked out now. I've added fixes for unaligned stack sizes, 32-bit truncating of stack size, and the "aligning up to 0" problem. I also added a test. Here's the latest webrev: http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot Here's what's changed since webrev.01: os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up the stack size to be paged aligned. This fixes issues on Mac OS X (other platforms seem to be immune to this). Then check if the size is zero after rounding up to the page size. Subtract the page size in this case to produce the maximum stack size allowed. Surprisingly I got no complaint from gcc for subtracting from an unsigned value that is known to be 0. os_linux.cpp: In os::create_thread(), I also check here to make sure the size is not 0 after adding the guard page and aligning up, and subtract the os page size if it is 0. jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater than UINT_MAX, then I set the size to UINT_MAX. Note it will later be rounded up to 0 in os::Posix::get_initial_stack_size(), which will result in subtracting the os page size to get the actual maximum allowed stack size. TooSmallStackSize.java: added test case for unaligned stack sizes. TestThreadStackSizes.java: New test. Creates new threads with every size up to 320k in 1k increments. Then creates threads with a few other sizes that can be problematic. thanks, Chris > Chris > > >>> >>>> public Thread(ThreadGroup group, Runnable target, String name, >>>> long stackSize) { >>>> init(group, target, name, stackSize); >>>> } >>>> >>>> Fortunately we already force the stackSize to be at least >>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>> assert by creating a thread with size 257k. >>> >>> Note this means that OSX stack logic is broken because it will >>> currently be silently failing due to EINVAL! >> Yes, that is correct. >> >> Chris >>> >>>> I'll get another webrev out once I've made the needed fixes. I also >>>> have >>>> a new test I'd like to add. >>> >>> Ok. >>> >>> Thanks, >>> David >>> >>>> Chris >>>> >>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>> Ok, time for a new webrev: >>>>> >>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>> >>>>> >>>>> The only thing that has changed since the first webrev are the >>>>> asserts >>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>> discuss >>>>> already: >>>>> >>>>> - The assert should never happen due to the stack size being too >>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>> - The assert should never happen due to an unaligned stack size >>>>> because we always align it to the page size. >>>>> - Any assert would therefore be a VM bug and not due to user error. >>>>> - No fixing the java launcher. If the user specifies a stack that is >>>>> too small, hotspot will already detect this. If the user specifies a >>>>> stack size that is large enough but not page aligned, then we just >>>>> ignore any error (if the platform doth protest) and the user gets a >>>>> main thread with a stack size set to whatever the OS default is. >>>>> >>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>> figure >>>>> getting agreement on the changes first would be best before I bog >>>>> down >>>>> our testing resources. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>> Hello, >>>>>> >>>>>> Please review the following: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>> >>>>>> >>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>> code >>>>>> was failing: >>>>>> >>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>> >>>>>> Although we computed a minimum stack size of 72k, so -Xss72k should >>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>> result is pthread_attr_setstacksize() had no impact on the thread's >>>>>> stack size, and we ended up with the platform default of 8mb. The >>>>>> fix >>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>> computing their new values: >>>>>> >>>>>> _java_thread_min_stack_allowed >>>>>> _compiler_thread_min_stack_allowed >>>>>> _vm_internal_thread_min_stack_allowed >>>>>> >>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>>> #defined, and this needs to be done before including OS header >>>>>> files. >>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>> >>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>> it. It >>>>>> was being applied before any adjustments to the stack sizes had been >>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>> the stack ended up being larger than necessary. With the above >>>>>> fix in >>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>> now too small, the correct fix is to adjust the initial stack sizes, >>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>> looks like no adjustment is needed. I did something close to our >>>>>> nightly testing on all affect platforms, and no new problems >>>>>> turned up. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>> >>>>> >>>> >> > From david.holmes at oracle.com Tue Mar 21 00:59:58 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 21 Mar 2017 10:59:58 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: Hi Chris, Getting closer :) Have you seen pthread_attr_setstacksize return EINVAL on very large values? Or does pthread_create just give a ENOMEM? On 21/03/2017 9:29 AM, Chris Plummer wrote: > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot > > Here's what's changed since webrev.01: > > os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up the > stack size to be paged aligned. This fixes issues on Mac OS X (other > platforms seem to be immune to this). Then check if the size is zero > after rounding up to the page size. Subtract the page size in this case > to produce the maximum stack size allowed. Surprisingly I got no > complaint from gcc for subtracting from an unsigned value that is known > to be 0. I'm a little surprised at that too. :) > os_linux.cpp: In os::create_thread(), I also check here to make sure the > size is not 0 after adding the guard page and aligning up, and subtract > the os page size if it is 0. What if adding the guard page and rounding up causes overflow so that we get a very small positive stack size? > jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater > than UINT_MAX, then I set the size to UINT_MAX. Note it will later be > rounded up to 0 in os::Posix::get_initial_stack_size(), which will > result in subtracting the os page size to get the actual maximum allowed > stack size. Good catch! Nit: "-Avoid" -> "- Avoid" > > TooSmallStackSize.java: added test case for unaligned stack sizes. Ok. > TestThreadStackSizes.java: New test. Creates new threads with every size > up to 320k in 1k increments. Then creates threads with a few other sizes > that can be problematic. So this test never actually fails as long as the VM doesn't crash - is that right? 27 * @modules java.base/jdk.internal.misc 28 * @library /test/lib The above are not needed by this test. Thanks, David > thanks, > > Chris >> Chris >> >> >>>> >>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>> long stackSize) { >>>>> init(group, target, name, stackSize); >>>>> } >>>>> >>>>> Fortunately we already force the stackSize to be at least >>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>> assert by creating a thread with size 257k. >>>> >>>> Note this means that OSX stack logic is broken because it will >>>> currently be silently failing due to EINVAL! >>> Yes, that is correct. >>> >>> Chris >>>> >>>>> I'll get another webrev out once I've made the needed fixes. I also >>>>> have >>>>> a new test I'd like to add. >>>> >>>> Ok. >>>> >>>> Thanks, >>>> David >>>> >>>>> Chris >>>>> >>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>> Ok, time for a new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>> >>>>>> >>>>>> The only thing that has changed since the first webrev are the >>>>>> asserts >>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>> discuss >>>>>> already: >>>>>> >>>>>> - The assert should never happen due to the stack size being too >>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>> - The assert should never happen due to an unaligned stack size >>>>>> because we always align it to the page size. >>>>>> - Any assert would therefore be a VM bug and not due to user error. >>>>>> - No fixing the java launcher. If the user specifies a stack that is >>>>>> too small, hotspot will already detect this. If the user specifies a >>>>>> stack size that is large enough but not page aligned, then we just >>>>>> ignore any error (if the platform doth protest) and the user gets a >>>>>> main thread with a stack size set to whatever the OS default is. >>>>>> >>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>> figure >>>>>> getting agreement on the changes first would be best before I bog >>>>>> down >>>>>> our testing resources. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review the following: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>> >>>>>>> >>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>> code >>>>>>> was failing: >>>>>>> >>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>> >>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k should >>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>> result is pthread_attr_setstacksize() had no impact on the thread's >>>>>>> stack size, and we ended up with the platform default of 8mb. The >>>>>>> fix >>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>> computing their new values: >>>>>>> >>>>>>> _java_thread_min_stack_allowed >>>>>>> _compiler_thread_min_stack_allowed >>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>> >>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>>>> #defined, and this needs to be done before including OS header >>>>>>> files. >>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>> >>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>> it. It >>>>>>> was being applied before any adjustments to the stack sizes had been >>>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>>> the stack ended up being larger than necessary. With the above >>>>>>> fix in >>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>>> now too small, the correct fix is to adjust the initial stack sizes, >>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>> turned up. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>> >>>>>> >>>>> >>> >> > From chris.plummer at oracle.com Tue Mar 21 01:25:19 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 20 Mar 2017 18:25:19 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: <6f98126f-703d-81fa-e180-51eaffb114a2@oracle.com> Hi David, On 3/20/17 5:59 PM, David Holmes wrote: > Hi Chris, > > > > Getting closer :) > > Have you seen pthread_attr_setstacksize return EINVAL on very large > values? Or does pthread_create just give a ENOMEM? It gives EAGAIN. The log for the new test contains: [0.416s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 9007199254740992k, guardsize: 0k, detached. Got exception for stack size 9223372036854774784: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached I assume it's also throwing OOME, which the test is catching and (correctly) ignoring. > > On 21/03/2017 9:29 AM, Chris Plummer wrote: >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >> >> Here's what's changed since webrev.01: >> >> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up the >> stack size to be paged aligned. This fixes issues on Mac OS X (other >> platforms seem to be immune to this). Then check if the size is zero >> after rounding up to the page size. Subtract the page size in this case >> to produce the maximum stack size allowed. Surprisingly I got no >> complaint from gcc for subtracting from an unsigned value that is known >> to be 0. > > I'm a little surprised at that too. :) > >> os_linux.cpp: In os::create_thread(), I also check here to make sure the >> size is not 0 after adding the guard page and aligning up, and subtract >> the os page size if it is 0. > > What if adding the guard page and rounding up causes overflow so that > we get a very small positive stack size? We already know the stack size is os page aligned up before getting here, so actually we could just add the guard page and not align up. You would get the same result. > >> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater >> than UINT_MAX, then I set the size to UINT_MAX. Note it will later be >> rounded up to 0 in os::Posix::get_initial_stack_size(), which will >> result in subtracting the os page size to get the actual maximum allowed >> stack size. > > Good catch! > > Nit: "-Avoid" -> "- Avoid" Ok. > >> >> TooSmallStackSize.java: added test case for unaligned stack sizes. > > Ok. > >> TestThreadStackSizes.java: New test. Creates new threads with every size >> up to 320k in 1k increments. Then creates threads with a few other sizes >> that can be problematic. > > So this test never actually fails as long as the VM doesn't crash - is > that right? Yes. > > 27 * @modules java.base/jdk.internal.misc > 28 * @library /test/lib > > The above are not needed by this test. Ok. thanks, Chris > > Thanks, > David > >> thanks, >> >> Chris >>> Chris >>> >>> >>>>> >>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>> long stackSize) { >>>>>> init(group, target, name, stackSize); >>>>>> } >>>>>> >>>>>> Fortunately we already force the stackSize to be at least >>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>> assert by creating a thread with size 257k. >>>>> >>>>> Note this means that OSX stack logic is broken because it will >>>>> currently be silently failing due to EINVAL! >>>> Yes, that is correct. >>>> >>>> Chris >>>>> >>>>>> I'll get another webrev out once I've made the needed fixes. I also >>>>>> have >>>>>> a new test I'd like to add. >>>>> >>>>> Ok. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Chris >>>>>> >>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>> Ok, time for a new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>> >>>>>>> >>>>>>> >>>>>>> The only thing that has changed since the first webrev are the >>>>>>> asserts >>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>> discuss >>>>>>> already: >>>>>>> >>>>>>> - The assert should never happen due to the stack size being too >>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>> because we always align it to the page size. >>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>> error. >>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>> that is >>>>>>> too small, hotspot will already detect this. If the user >>>>>>> specifies a >>>>>>> stack size that is large enough but not page aligned, then we just >>>>>>> ignore any error (if the platform doth protest) and the user gets a >>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>> >>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>> figure >>>>>>> getting agreement on the changes first would be best before I bog >>>>>>> down >>>>>>> our testing resources. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the following: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>>> code >>>>>>>> was failing: >>>>>>>> >>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>> >>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>> should >>>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>> thread's >>>>>>>> stack size, and we ended up with the platform default of 8mb. The >>>>>>>> fix >>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>>> computing their new values: >>>>>>>> >>>>>>>> _java_thread_min_stack_allowed >>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>> >>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>> need to >>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>> PTHREAD_STACK_MIN >>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>> files. >>>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>> this >>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>> >>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>> it. It >>>>>>>> was being applied before any adjustments to the stack sizes had >>>>>>>> been >>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>> mean >>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>> fix in >>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>> sizes, >>>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>> turned up. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>> >>>>>>> >>>>>> >>>> >>> >> From chris.plummer at oracle.com Tue Mar 21 01:38:14 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 20 Mar 2017 18:38:14 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <6f98126f-703d-81fa-e180-51eaffb114a2@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <6f98126f-703d-81fa-e180-51eaffb114a2@oracle.com> Message-ID: <0553435f-06e5-8e84-b669-d74c0dde7997@oracle.com> On 3/20/17 6:25 PM, Chris Plummer wrote: > Hi David, > > On 3/20/17 5:59 PM, David Holmes wrote: >> Hi Chris, >> >> >> >> Getting closer :) >> >> Have you seen pthread_attr_setstacksize return EINVAL on very large >> values? Or does pthread_create just give a ENOMEM? > It gives EAGAIN. The log for the new test contains: > > [0.416s][warning][os,thread] Failed to start thread - pthread_create > failed (EAGAIN) for attributes: stacksize: 9007199254740992k, > guardsize: 0k, detached. > Got exception for stack size 9223372036854774784: > java.lang.OutOfMemoryError: unable to create native thread: possibly > out of memory or process/resource limits reached > > I assume it's also throwing OOME, which the test is catching and > (correctly) ignoring. Obviously it is getting the OOME because the test output above is telling us as much. Chris > >> >> On 21/03/2017 9:29 AM, Chris Plummer wrote: >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>> >>> Here's what's changed since webrev.01: >>> >>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up >>> the >>> stack size to be paged aligned. This fixes issues on Mac OS X (other >>> platforms seem to be immune to this). Then check if the size is zero >>> after rounding up to the page size. Subtract the page size in this case >>> to produce the maximum stack size allowed. Surprisingly I got no >>> complaint from gcc for subtracting from an unsigned value that is known >>> to be 0. >> >> I'm a little surprised at that too. :) >> >>> os_linux.cpp: In os::create_thread(), I also check here to make sure >>> the >>> size is not 0 after adding the guard page and aligning up, and subtract >>> the os page size if it is 0. >> >> What if adding the guard page and rounding up causes overflow so that >> we get a very small positive stack size? > We already know the stack size is os page aligned up before getting > here, so actually we could just add the guard page and not align up. > You would get the same result. >> >>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater >>> than UINT_MAX, then I set the size to UINT_MAX. Note it will later be >>> rounded up to 0 in os::Posix::get_initial_stack_size(), which will >>> result in subtracting the os page size to get the actual maximum >>> allowed >>> stack size. >> >> Good catch! >> >> Nit: "-Avoid" -> "- Avoid" > Ok. >> >>> >>> TooSmallStackSize.java: added test case for unaligned stack sizes. >> >> Ok. >> >>> TestThreadStackSizes.java: New test. Creates new threads with every >>> size >>> up to 320k in 1k increments. Then creates threads with a few other >>> sizes >>> that can be problematic. >> >> So this test never actually fails as long as the VM doesn't crash - >> is that right? > Yes. >> >> 27 * @modules java.base/jdk.internal.misc >> 28 * @library /test/lib >> >> The above are not needed by this test. > Ok. > > thanks, > > Chris >> >> Thanks, >> David >> >>> thanks, >>> >>> Chris >>>> Chris >>>> >>>> >>>>>> >>>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>>> long stackSize) { >>>>>>> init(group, target, name, stackSize); >>>>>>> } >>>>>>> >>>>>>> Fortunately we already force the stackSize to be at least >>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>>> assert by creating a thread with size 257k. >>>>>> >>>>>> Note this means that OSX stack logic is broken because it will >>>>>> currently be silently failing due to EINVAL! >>>>> Yes, that is correct. >>>>> >>>>> Chris >>>>>> >>>>>>> I'll get another webrev out once I've made the needed fixes. I also >>>>>>> have >>>>>>> a new test I'd like to add. >>>>>> >>>>>> Ok. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>> Ok, time for a new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The only thing that has changed since the first webrev are the >>>>>>>> asserts >>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>>> discuss >>>>>>>> already: >>>>>>>> >>>>>>>> - The assert should never happen due to the stack size being too >>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>>> because we always align it to the page size. >>>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>>> error. >>>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>>> that is >>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>> specifies a >>>>>>>> stack size that is large enough but not page aligned, then we just >>>>>>>> ignore any error (if the platform doth protest) and the user >>>>>>>> gets a >>>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>>> >>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>>> figure >>>>>>>> getting agreement on the changes first would be best before I bog >>>>>>>> down >>>>>>>> our testing resources. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review the following: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> While working on 8175342 I noticed our stack size on xgene was >>>>>>>>> 8mb >>>>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>>>> code >>>>>>>>> was failing: >>>>>>>>> >>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>> >>>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>>> should >>>>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>> thread's >>>>>>>>> stack size, and we ended up with the platform default of 8mb. The >>>>>>>>> fix >>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>>>> computing their new values: >>>>>>>>> >>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>> >>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>>> need to >>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>>> files. >>>>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>>> this >>>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>>> >>>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>>> it. It >>>>>>>>> was being applied before any adjustments to the stack sizes >>>>>>>>> had been >>>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>>> mean >>>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>>> fix in >>>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>>> minimum stack sizes. If for any reason one of those stack >>>>>>>>> sizes is >>>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>>> sizes, >>>>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>>> turned up. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> > From david.holmes at oracle.com Tue Mar 21 01:45:42 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 21 Mar 2017 11:45:42 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <6f98126f-703d-81fa-e180-51eaffb114a2@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <6f98126f-703d-81fa-e180-51eaffb114a2@oracle.com> Message-ID: <44b8543b-25f2-b29c-236e-280abaca2fad@oracle.com> On 21/03/2017 11:25 AM, Chris Plummer wrote: > Hi David, > > On 3/20/17 5:59 PM, David Holmes wrote: >> Hi Chris, >> >> >> >> Getting closer :) >> >> Have you seen pthread_attr_setstacksize return EINVAL on very large >> values? Or does pthread_create just give a ENOMEM? > It gives EAGAIN. The log for the new test contains: > > [0.416s][warning][os,thread] Failed to start thread - pthread_create > failed (EAGAIN) for attributes: stacksize: 9007199254740992k, guardsize: > 0k, detached. > Got exception for stack size 9223372036854774784: > java.lang.OutOfMemoryError: unable to create native thread: possibly out > of memory or process/resource limits reached > > I assume it's also throwing OOME, which the test is catching and > (correctly) ignoring. Yep that is all good. I was just wondering of pthread_attr_setstacksize did any sanity checking of big values and returned EINVAL - but it seems not. >> >> On 21/03/2017 9:29 AM, Chris Plummer wrote: >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>> >>> Here's what's changed since webrev.01: >>> >>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up the >>> stack size to be paged aligned. This fixes issues on Mac OS X (other >>> platforms seem to be immune to this). Then check if the size is zero >>> after rounding up to the page size. Subtract the page size in this case >>> to produce the maximum stack size allowed. Surprisingly I got no >>> complaint from gcc for subtracting from an unsigned value that is known >>> to be 0. >> >> I'm a little surprised at that too. :) >> >>> os_linux.cpp: In os::create_thread(), I also check here to make sure the >>> size is not 0 after adding the guard page and aligning up, and subtract >>> the os page size if it is 0. >> >> What if adding the guard page and rounding up causes overflow so that >> we get a very small positive stack size? > We already know the stack size is os page aligned up before getting > here, so actually we could just add the guard page and not align up. You > would get the same result. So we can get rid of the align_up here: 727 stack_size = align_size_up(stack_size + os::Linux::default_guard_size(thr_type), vm_page_size()); But are we assuming the guard page is always one virtual-memory page? Or put another way: what is the difference between page_size() and vm_page_size() ? Thanks, David ----- >> >>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater >>> than UINT_MAX, then I set the size to UINT_MAX. Note it will later be >>> rounded up to 0 in os::Posix::get_initial_stack_size(), which will >>> result in subtracting the os page size to get the actual maximum allowed >>> stack size. >> >> Good catch! >> >> Nit: "-Avoid" -> "- Avoid" > Ok. >> >>> >>> TooSmallStackSize.java: added test case for unaligned stack sizes. >> >> Ok. >> >>> TestThreadStackSizes.java: New test. Creates new threads with every size >>> up to 320k in 1k increments. Then creates threads with a few other sizes >>> that can be problematic. >> >> So this test never actually fails as long as the VM doesn't crash - is >> that right? > Yes. >> >> 27 * @modules java.base/jdk.internal.misc >> 28 * @library /test/lib >> >> The above are not needed by this test. > Ok. > > thanks, > > Chris >> >> Thanks, >> David >> >>> thanks, >>> >>> Chris >>>> Chris >>>> >>>> >>>>>> >>>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>>> long stackSize) { >>>>>>> init(group, target, name, stackSize); >>>>>>> } >>>>>>> >>>>>>> Fortunately we already force the stackSize to be at least >>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>>> assert by creating a thread with size 257k. >>>>>> >>>>>> Note this means that OSX stack logic is broken because it will >>>>>> currently be silently failing due to EINVAL! >>>>> Yes, that is correct. >>>>> >>>>> Chris >>>>>> >>>>>>> I'll get another webrev out once I've made the needed fixes. I also >>>>>>> have >>>>>>> a new test I'd like to add. >>>>>> >>>>>> Ok. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>> Ok, time for a new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The only thing that has changed since the first webrev are the >>>>>>>> asserts >>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>>> discuss >>>>>>>> already: >>>>>>>> >>>>>>>> - The assert should never happen due to the stack size being too >>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>>> because we always align it to the page size. >>>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>>> error. >>>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>>> that is >>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>> specifies a >>>>>>>> stack size that is large enough but not page aligned, then we just >>>>>>>> ignore any error (if the platform doth protest) and the user gets a >>>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>>> >>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>>> figure >>>>>>>> getting agreement on the changes first would be best before I bog >>>>>>>> down >>>>>>>> our testing resources. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review the following: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>>>> code >>>>>>>>> was failing: >>>>>>>>> >>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>> >>>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>>> should >>>>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>> thread's >>>>>>>>> stack size, and we ended up with the platform default of 8mb. The >>>>>>>>> fix >>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>>>> computing their new values: >>>>>>>>> >>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>> >>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>>> need to >>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>>> files. >>>>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>>> this >>>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>>> >>>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>>> it. It >>>>>>>>> was being applied before any adjustments to the stack sizes had >>>>>>>>> been >>>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>>> mean >>>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>>> fix in >>>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>>> sizes, >>>>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>>> turned up. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> > From chris.plummer at oracle.com Tue Mar 21 02:35:44 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 20 Mar 2017 19:35:44 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <44b8543b-25f2-b29c-236e-280abaca2fad@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <6f98126f-703d-81fa-e180-51eaffb114a2@oracle.com> <44b8543b-25f2-b29c-236e-280abaca2fad@oracle.com> Message-ID: On 3/20/17 6:45 PM, David Holmes wrote: > On 21/03/2017 11:25 AM, Chris Plummer wrote: >> Hi David, >> >> On 3/20/17 5:59 PM, David Holmes wrote: >>> Hi Chris, >>> >>> >>> >>> Getting closer :) >>> >>> Have you seen pthread_attr_setstacksize return EINVAL on very large >>> values? Or does pthread_create just give a ENOMEM? >> It gives EAGAIN. The log for the new test contains: >> >> [0.416s][warning][os,thread] Failed to start thread - pthread_create >> failed (EAGAIN) for attributes: stacksize: 9007199254740992k, guardsize: >> 0k, detached. >> Got exception for stack size 9223372036854774784: >> java.lang.OutOfMemoryError: unable to create native thread: possibly out >> of memory or process/resource limits reached >> >> I assume it's also throwing OOME, which the test is catching and >> (correctly) ignoring. > > Yep that is all good. I was just wondering of > pthread_attr_setstacksize did any sanity checking of big values and > returned EINVAL - but it seems not. > >>> >>> On 21/03/2017 9:29 AM, Chris Plummer wrote: >>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>>> >>>> Here's what's changed since webrev.01: >>>> >>>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round >>>> up the >>>> stack size to be paged aligned. This fixes issues on Mac OS X (other >>>> platforms seem to be immune to this). Then check if the size is zero >>>> after rounding up to the page size. Subtract the page size in this >>>> case >>>> to produce the maximum stack size allowed. Surprisingly I got no >>>> complaint from gcc for subtracting from an unsigned value that is >>>> known >>>> to be 0. >>> >>> I'm a little surprised at that too. :) >>> >>>> os_linux.cpp: In os::create_thread(), I also check here to make >>>> sure the >>>> size is not 0 after adding the guard page and aligning up, and >>>> subtract >>>> the os page size if it is 0. >>> >>> What if adding the guard page and rounding up causes overflow so that >>> we get a very small positive stack size? >> We already know the stack size is os page aligned up before getting >> here, so actually we could just add the guard page and not align up. You >> would get the same result. > > So we can get rid of the align_up here: > > 727 stack_size = align_size_up(stack_size + > os::Linux::default_guard_size(thr_type), vm_page_size()); I think so. I'm going to try the following: stack_size += os::Linux::default_guard_size(thr_type); assert(is_size_aligned(stack_size, os::vm_page_size()), "stack_size not aligned"); > > But are we assuming the guard page is always one virtual-memory page? Yes. size_t os::Linux::default_guard_size(os::ThreadType thr_type) { // Creating guard page is very expensive. Java thread has HotSpot // guard pages, only enable glibc guard page for non-Java threads. // (Remember: compiler thread is a Java thread, too!) return ((thr_type == java_thread || thr_type == compiler_thread) ? 0 : page_size()); } > Or put another way: what is the difference between page_size() and > vm_page_size() ? They are the same: static int page_size(void) { return _page_size; } static void set_page_size(int val) { _page_size = val; } Linux::set_page_size(sysconf(_SC_PAGESIZE)); int os::vm_page_size() { // Seems redundant as all get out assert(os::Linux::page_size() != -1, "must call os::init"); return os::Linux::page_size(); } I'm not sure why we have both. thanks, Chris > > Thanks, > David > ----- > >>> >>>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is >>>> greater >>>> than UINT_MAX, then I set the size to UINT_MAX. Note it will later be >>>> rounded up to 0 in os::Posix::get_initial_stack_size(), which will >>>> result in subtracting the os page size to get the actual maximum >>>> allowed >>>> stack size. >>> >>> Good catch! >>> >>> Nit: "-Avoid" -> "- Avoid" >> Ok. >>> >>>> >>>> TooSmallStackSize.java: added test case for unaligned stack sizes. >>> >>> Ok. >>> >>>> TestThreadStackSizes.java: New test. Creates new threads with every >>>> size >>>> up to 320k in 1k increments. Then creates threads with a few other >>>> sizes >>>> that can be problematic. >>> >>> So this test never actually fails as long as the VM doesn't crash - is >>> that right? >> Yes. >>> >>> 27 * @modules java.base/jdk.internal.misc >>> 28 * @library /test/lib >>> >>> The above are not needed by this test. >> Ok. >> >> thanks, >> >> Chris >>> >>> Thanks, >>> David >>> >>>> thanks, >>>> >>>> Chris >>>>> Chris >>>>> >>>>> >>>>>>> >>>>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>>>> long stackSize) { >>>>>>>> init(group, target, name, stackSize); >>>>>>>> } >>>>>>>> >>>>>>>> Fortunately we already force the stackSize to be at least >>>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>>>> assert by creating a thread with size 257k. >>>>>>> >>>>>>> Note this means that OSX stack logic is broken because it will >>>>>>> currently be silently failing due to EINVAL! >>>>>> Yes, that is correct. >>>>>> >>>>>> Chris >>>>>>> >>>>>>>> I'll get another webrev out once I've made the needed fixes. I >>>>>>>> also >>>>>>>> have >>>>>>>> a new test I'd like to add. >>>>>>> >>>>>>> Ok. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>>> Ok, time for a new webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> The only thing that has changed since the first webrev are the >>>>>>>>> asserts >>>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>>>> discuss >>>>>>>>> already: >>>>>>>>> >>>>>>>>> - The assert should never happen due to the stack size being too >>>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>>>> because we always align it to the page size. >>>>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>>>> error. >>>>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>>>> that is >>>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>>> specifies a >>>>>>>>> stack size that is large enough but not page aligned, then we >>>>>>>>> just >>>>>>>>> ignore any error (if the platform doth protest) and the user >>>>>>>>> gets a >>>>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>>>> >>>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>>>> figure >>>>>>>>> getting agreement on the changes first would be best before I bog >>>>>>>>> down >>>>>>>>> our testing resources. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>>> >>>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> Please review the following: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> While working on 8175342 I noticed our stack size on xgene >>>>>>>>>> was 8mb >>>>>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>>>>> code >>>>>>>>>> was failing: >>>>>>>>>> >>>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>>> >>>>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>>>> should >>>>>>>>>> be fine, pthreads on this platform requires the stack be at >>>>>>>>>> least >>>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>>> thread's >>>>>>>>>> stack size, and we ended up with the platform default of 8mb. >>>>>>>>>> The >>>>>>>>>> fix >>>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN >>>>>>>>>> after >>>>>>>>>> computing their new values: >>>>>>>>>> >>>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>>> >>>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>>>> need to >>>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>>>> files. >>>>>>>>>> I noticed that on solaris we were using thr_min_stack() >>>>>>>>>> elsewhere >>>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>>>> this >>>>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>>>> >>>>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>>>> it. It >>>>>>>>>> was being applied before any adjustments to the stack sizes had >>>>>>>>>> been >>>>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>>>> mean >>>>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>>>> fix in >>>>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>>>> minimum stack sizes. If for any reason one of those stack >>>>>>>>>> sizes is >>>>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>>>> sizes, >>>>>>>>>> not apply thr_min_stack() to the initial stack sizes. >>>>>>>>>> However, it >>>>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>>>> turned up. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> >>>>>>>>>> Chris >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >> From david.holmes at oracle.com Tue Mar 21 03:47:05 2017 From: david.holmes at oracle.com (David Holmes) Date: Tue, 21 Mar 2017 13:47:05 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <6f98126f-703d-81fa-e180-51eaffb114a2@oracle.com> <44b8543b-25f2-b29c-236e-280abaca2fad@oracle.com> Message-ID: On 21/03/2017 12:35 PM, Chris Plummer wrote: > On 3/20/17 6:45 PM, David Holmes wrote: >> On 21/03/2017 11:25 AM, Chris Plummer wrote: >>> Hi David, >>> >>> On 3/20/17 5:59 PM, David Holmes wrote: >>>> Hi Chris, >>>> >>>> >>>> >>>> Getting closer :) >>>> >>>> Have you seen pthread_attr_setstacksize return EINVAL on very large >>>> values? Or does pthread_create just give a ENOMEM? >>> It gives EAGAIN. The log for the new test contains: >>> >>> [0.416s][warning][os,thread] Failed to start thread - pthread_create >>> failed (EAGAIN) for attributes: stacksize: 9007199254740992k, guardsize: >>> 0k, detached. >>> Got exception for stack size 9223372036854774784: >>> java.lang.OutOfMemoryError: unable to create native thread: possibly out >>> of memory or process/resource limits reached >>> >>> I assume it's also throwing OOME, which the test is catching and >>> (correctly) ignoring. >> >> Yep that is all good. I was just wondering of >> pthread_attr_setstacksize did any sanity checking of big values and >> returned EINVAL - but it seems not. >> >>>> >>>> On 21/03/2017 9:29 AM, Chris Plummer wrote: >>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>>>> >>>>> Here's what's changed since webrev.01: >>>>> >>>>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round >>>>> up the >>>>> stack size to be paged aligned. This fixes issues on Mac OS X (other >>>>> platforms seem to be immune to this). Then check if the size is zero >>>>> after rounding up to the page size. Subtract the page size in this >>>>> case >>>>> to produce the maximum stack size allowed. Surprisingly I got no >>>>> complaint from gcc for subtracting from an unsigned value that is >>>>> known >>>>> to be 0. >>>> >>>> I'm a little surprised at that too. :) >>>> >>>>> os_linux.cpp: In os::create_thread(), I also check here to make >>>>> sure the >>>>> size is not 0 after adding the guard page and aligning up, and >>>>> subtract >>>>> the os page size if it is 0. >>>> >>>> What if adding the guard page and rounding up causes overflow so that >>>> we get a very small positive stack size? >>> We already know the stack size is os page aligned up before getting >>> here, so actually we could just add the guard page and not align up. You >>> would get the same result. >> >> So we can get rid of the align_up here: >> >> 727 stack_size = align_size_up(stack_size + >> os::Linux::default_guard_size(thr_type), vm_page_size()); > I think so. I'm going to try the following: > > stack_size += os::Linux::default_guard_size(thr_type); > assert(is_size_aligned(stack_size, os::vm_page_size()), "stack_size > not aligned"); Looks good. >> >> But are we assuming the guard page is always one virtual-memory page? > Yes. > > size_t os::Linux::default_guard_size(os::ThreadType thr_type) { > // Creating guard page is very expensive. Java thread has HotSpot > // guard pages, only enable glibc guard page for non-Java threads. > // (Remember: compiler thread is a Java thread, too!) > return ((thr_type == java_thread || thr_type == compiler_thread) ? 0 : > page_size()); > } > >> Or put another way: what is the difference between page_size() and >> vm_page_size() ? > They are the same: > > static int page_size(void) { return _page_size; } > static void set_page_size(int val) { _page_size = val; } > Linux::set_page_size(sysconf(_SC_PAGESIZE)); > > int os::vm_page_size() { > // Seems redundant as all get out > assert(os::Linux::page_size() != -1, "must call os::init"); > return os::Linux::page_size(); > } > > I'm not sure why we have both. Okay. Yeah no reason to actually define os::Linux::page_size() when we could just have a static _page_size variable in os_linux.cpp outside any class. Thanks, David ----- > thanks, > > Chris >> >> Thanks, >> David >> ----- >> >>>> >>>>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is >>>>> greater >>>>> than UINT_MAX, then I set the size to UINT_MAX. Note it will later be >>>>> rounded up to 0 in os::Posix::get_initial_stack_size(), which will >>>>> result in subtracting the os page size to get the actual maximum >>>>> allowed >>>>> stack size. >>>> >>>> Good catch! >>>> >>>> Nit: "-Avoid" -> "- Avoid" >>> Ok. >>>> >>>>> >>>>> TooSmallStackSize.java: added test case for unaligned stack sizes. >>>> >>>> Ok. >>>> >>>>> TestThreadStackSizes.java: New test. Creates new threads with every >>>>> size >>>>> up to 320k in 1k increments. Then creates threads with a few other >>>>> sizes >>>>> that can be problematic. >>>> >>>> So this test never actually fails as long as the VM doesn't crash - is >>>> that right? >>> Yes. >>>> >>>> 27 * @modules java.base/jdk.internal.misc >>>> 28 * @library /test/lib >>>> >>>> The above are not needed by this test. >>> Ok. >>> >>> thanks, >>> >>> Chris >>>> >>>> Thanks, >>>> David >>>> >>>>> thanks, >>>>> >>>>> Chris >>>>>> Chris >>>>>> >>>>>> >>>>>>>> >>>>>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>>>>> long stackSize) { >>>>>>>>> init(group, target, name, stackSize); >>>>>>>>> } >>>>>>>>> >>>>>>>>> Fortunately we already force the stackSize to be at least >>>>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>>>>> assert by creating a thread with size 257k. >>>>>>>> >>>>>>>> Note this means that OSX stack logic is broken because it will >>>>>>>> currently be silently failing due to EINVAL! >>>>>>> Yes, that is correct. >>>>>>> >>>>>>> Chris >>>>>>>> >>>>>>>>> I'll get another webrev out once I've made the needed fixes. I >>>>>>>>> also >>>>>>>>> have >>>>>>>>> a new test I'd like to add. >>>>>>>> >>>>>>>> Ok. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> Chris >>>>>>>>> >>>>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>>>> Ok, time for a new webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The only thing that has changed since the first webrev are the >>>>>>>>>> asserts >>>>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>>>>> discuss >>>>>>>>>> already: >>>>>>>>>> >>>>>>>>>> - The assert should never happen due to the stack size being too >>>>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>>>>> because we always align it to the page size. >>>>>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>>>>> error. >>>>>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>>>>> that is >>>>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>>>> specifies a >>>>>>>>>> stack size that is large enough but not page aligned, then we >>>>>>>>>> just >>>>>>>>>> ignore any error (if the platform doth protest) and the user >>>>>>>>>> gets a >>>>>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>>>>> >>>>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>>>>> figure >>>>>>>>>> getting agreement on the changes first would be best before I bog >>>>>>>>>> down >>>>>>>>>> our testing resources. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> >>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> Please review the following: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> While working on 8175342 I noticed our stack size on xgene >>>>>>>>>>> was 8mb >>>>>>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>>>>>> code >>>>>>>>>>> was failing: >>>>>>>>>>> >>>>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>>>> >>>>>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>>>>> should >>>>>>>>>>> be fine, pthreads on this platform requires the stack be at >>>>>>>>>>> least >>>>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>>>> thread's >>>>>>>>>>> stack size, and we ended up with the platform default of 8mb. >>>>>>>>>>> The >>>>>>>>>>> fix >>>>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN >>>>>>>>>>> after >>>>>>>>>>> computing their new values: >>>>>>>>>>> >>>>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>>>> >>>>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>>>>> need to >>>>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>>>>> files. >>>>>>>>>>> I noticed that on solaris we were using thr_min_stack() >>>>>>>>>>> elsewhere >>>>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>>>>> this >>>>>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>>>>> >>>>>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>>>>> it. It >>>>>>>>>>> was being applied before any adjustments to the stack sizes had >>>>>>>>>>> been >>>>>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>>>>> mean >>>>>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>>>>> fix in >>>>>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>>>>> minimum stack sizes. If for any reason one of those stack >>>>>>>>>>> sizes is >>>>>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>>>>> sizes, >>>>>>>>>>> not apply thr_min_stack() to the initial stack sizes. >>>>>>>>>>> However, it >>>>>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>>>>> turned up. >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> >>>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>> >>> > From xueming.shen at oracle.com Tue Mar 21 06:10:51 2017 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 20 Mar 2017 23:10:51 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <3553280e-6955-a574-8154-19cb52409300@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> Message-ID: <58D0C3EB.9020208@oracle.com> Hi Dean, Thanks for doing this. Though as I suggested last time personally I prefer to make minimum change to simply seal those holes in ASB at this late stage of JDK9. I'm fine with the webrev.2 and it looks better and reasonable to pull all UTF16 operations into StringUTF16.java. Just for my curiosity, does the change in String#1810 make difference? Thanks! Sherman On 3/17/17, 3:07 PM, dean.long at oracle.com wrote: > I posted two new versions, webrev.1 keeping the Trusted inner class: > > http://cr.openjdk.java.net/~dlong/8158168/webrev.1/ > > and webrev.2 with it removed: > > http://cr.openjdk.java.net/~dlong/8158168/webrev.2/ > > dl > > On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >> >>>> I have the same concern. Can we fix the immediate problem in 9 and >>>> integrate verification logic in 10? >>>> >>> >>> OK, Tobias is suggesting having verification logic only inside the >>> intrinsics. Are you suggesting removing that as well? >> >> Yes and put them back in 10. >> >>> I'm OK with removing all the verification, but that won't reduce the >>> library changes much. I could undo the renaming to Trusted.getChar, >>> but >>> we would still have the bounds checks moved into StringUTF16. >> >> I suggest to go with a point fix for 9: just add missing range checks. > From dean.long at oracle.com Tue Mar 21 07:37:49 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 21 Mar 2017 00:37:49 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <58D0C3EB.9020208@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <58D0C3EB.9020208@oracle.com> Message-ID: <5a1b6d88-265d-eaa7-1fed-77e804dc79e8@oracle.com> On 3/20/17 11:10 PM, Xueming Shen wrote: > Hi Dean, > > Thanks for doing this. > > Though as I suggested last time personally I prefer to make minimum > change to simply > seal those holes in ASB at this late stage of JDK9. I'm fine with the > webrev.2 and it looks > better and reasonable to pull all UTF16 operations into StringUTF16.java. > > Just for my curiosity, does the change in String#1810 make difference? > Yes, it's in the spirit of the comment "return immediately where possible", and allows me to have 474 assert fromIndex >= 0; in StringUTF16.lastIndexOf(). This is because fromIndex can go negative at String#1808. Thanks for the review. dl > Thanks! > Sherman > > > > On 3/17/17, 3:07 PM, dean.long at oracle.com wrote: >> I posted two new versions, webrev.1 keeping the Trusted inner class: >> >> http://cr.openjdk.java.net/~dlong/8158168/webrev.1/ >> >> and webrev.2 with it removed: >> >> http://cr.openjdk.java.net/~dlong/8158168/webrev.2/ >> >> dl >> >> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>> >>>>> I have the same concern. Can we fix the immediate problem in 9 and >>>>> integrate verification logic in 10? >>>>> >>>> >>>> OK, Tobias is suggesting having verification logic only inside the >>>> intrinsics. Are you suggesting removing that as well? >>> >>> Yes and put them back in 10. >>> >>>> I'm OK with removing all the verification, but that won't reduce the >>>> library changes much. I could undo the renaming to >>>> Trusted.getChar, but >>>> we would still have the bounds checks moved into StringUTF16. >>> >>> I suggest to go with a point fix for 9: just add missing range checks. >> > From thomas.stuefe at gmail.com Tue Mar 21 08:08:38 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 21 Mar 2017 09:08:38 +0100 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: Hi Chris, looks fine to me. small nit: os_linux_cpp: + // In that cast just subract the page size to get the maximum possible stack size. subract->substract I would have preferred SIZE_MAX instead of UINT_MAX, but both is fine to me. -- As for the size_overflow problem, I wonder whether we are overthinking this. Maybe a more pragmatic approach would have been to just define a reasonable upper maximum for stack sizes which could be well below UINT_MAX. Somehow I cannot see users wanting to start threads with more than ~4GB of thread stack size. Kind Regards, Thomas On Tue, Mar 21, 2017 at 12:29 AM, Chris Plummer wrote: > On 3/17/17 11:37 PM, Chris Plummer wrote: > >> On 3/17/17 8:17 PM, Chris Plummer wrote: >> >>> On 3/17/17 7:01 PM, David Holmes wrote: >>> >>>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>> >>>>> Looks like this will need some more work since the added asserts are >>>>> triggering on mac os x (which is the only place we'd currently expect >>>>> them to assert). >>>>> >>>>> The problem is that the code I found that rounds up to the page size is >>>>> only applied to java threads created by the VM for which the java user >>>>> specified no stack size. The VM and Compiler thread sizes are not >>>>> rounded. The failure I saw was with >>>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java when >>>>> is >>>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit the >>>>> assert >>>>> with an EINVAL. The size is not aligned, but it could also be >>>>> complaining because it is too big. I haven't tried aligning it yet to >>>>> see. >>>>> >>>>> On Linux we do the following: >>>>> >>>>> stack_size = align_size_up(stack_size + >>>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>>> >>>>> We don't do this on BSD. I think the same on BSD would solve this >>>>> problem. I'm not sure about adding the guard size. I'll need to see if >>>>> mac os x has the same pthread bug as linux does. >>>>> >>>> >>>> At this stage I would only deal with alignment issues. IIRC the guard >>>> issue only affected Linux. >>>> >>> Yes, that's what I eventually concluded. I put the fix in >>> os::Posix::get_initial_stack_size() in os_posix.cpp, and only did the >>> page aligning, not add the guard page. That way all Posix ports are fixed >>> in one place. It seems to be working. >>> >>>> >>>> BTW, did you know java users can specify the size of the a new thread's >>>>> stack: >>>>> >>>> >>>> Yes I mentioned that in another reply - wondering whether we suitably >>>> check and aligned such requests. >>>> >>> No we don't. Below I mentioned I was able to trigger the assert with a >>> 257k stack size. I guess I wasn't clear that I did that from Java. I have a >>> new test to add that will be testing this, plus the 9007199254740991 stack >>> size (which fails to create the thread with an OOME, but that's >>> acceptable). The fix I mention above in os::Posix::get_initial_stack_size() >>> takes care of this issue also. >>> >> Rounding up triggers a new assert, this time on 32-bit x86 and arm. >> >> I should have clarified it's 9007199254740991 "K", which is >> 9223372036854774784. Unfortunately on 32bit systems that is asserting with >> EINVAL. I think we need to do a better job of dealing with 32-bit size_t >> values: >> >> jlong java_lang_Thread::stackSize(oop java_thread) { >> if (_stackSize_offset > 0) { >> return java_thread->long_field(_stackSize_offset); >> } else { >> return 0; >> } >> } >> >> jlong size = >> java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); >> // Allocate the C++ Thread structure and create the native thread. >> The >> // stack size retrieved from java is signed, but the constructor >> takes >> // size_t (an unsigned type), so avoid passing negative values >> which would >> // result in really large stacks. >> size_t sz = size > 0 ? (size_t) size : 0; >> native_thread = new JavaThread(&thread_entry, sz); >> >> 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit MAX_INT), >> which is 0xfffffc00 when cast to a size_t on a 32-bit system (close to >> 32-bit MAX_UINT). Round it up to the 4k page size and you get 0, which I >> guess pthread_attr_setstacksize() doesn't like. So I think more processing >> of the size is needed here. Maybe in os::create_thread() we should check >> for 0 after rounding up, and subtract the os page size if it is 0. However, >> I think we should also avoid truncating on 32-bit to what is basically some >> random number. Maybe if "size" (a jlong) is greater than UINT_MAX, then >> just set "sz" (a size_t) it to UINT_MAX. >> >> Ok, I think I have this all worked out now. I've added fixes for > unaligned stack sizes, 32-bit truncating of stack size, and the "aligning > up to 0" problem. I also added a test. Here's the latest webrev: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot > > Here's what's changed since webrev.01: > > os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up the > stack size to be paged aligned. This fixes issues on Mac OS X (other > platforms seem to be immune to this). Then check if the size is zero after > rounding up to the page size. Subtract the page size in this case to > produce the maximum stack size allowed. Surprisingly I got no complaint > from gcc for subtracting from an unsigned value that is known to be 0. > > os_linux.cpp: In os::create_thread(), I also check here to make sure the > size is not 0 after adding the guard page and aligning up, and subtract the > os page size if it is 0. > > jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater > than UINT_MAX, then I set the size to UINT_MAX. Note it will later be > rounded up to 0 in os::Posix::get_initial_stack_size(), which will result > in subtracting the os page size to get the actual maximum allowed stack > size. > > TooSmallStackSize.java: added test case for unaligned stack sizes. > > TestThreadStackSizes.java: New test. Creates new threads with every size > up to 320k in 1k increments. Then creates threads with a few other sizes > that can be problematic. > > thanks, > > Chris > > Chris >> >> >> >>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>> long stackSize) { >>>>> init(group, target, name, stackSize); >>>>> } >>>>> >>>>> Fortunately we already force the stackSize to be at least >>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>> assert by creating a thread with size 257k. >>>>> >>>> >>>> Note this means that OSX stack logic is broken because it will >>>> currently be silently failing due to EINVAL! >>>> >>> Yes, that is correct. >>> >>> Chris >>> >>>> >>>> I'll get another webrev out once I've made the needed fixes. I also have >>>>> a new test I'd like to add. >>>>> >>>> >>>> Ok. >>>> >>>> Thanks, >>>> David >>>> >>>> Chris >>>>> >>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>> >>>>>> Ok, time for a new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webr >>>>>> ev.hotspot >>>>>> >>>>>> The only thing that has changed since the first webrev are the asserts >>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we discuss >>>>>> already: >>>>>> >>>>>> - The assert should never happen due to the stack size being too >>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>> - The assert should never happen due to an unaligned stack size >>>>>> because we always align it to the page size. >>>>>> - Any assert would therefore be a VM bug and not due to user error. >>>>>> - No fixing the java launcher. If the user specifies a stack that is >>>>>> too small, hotspot will already detect this. If the user specifies a >>>>>> stack size that is large enough but not page aligned, then we just >>>>>> ignore any error (if the platform doth protest) and the user gets a >>>>>> main thread with a stack size set to whatever the OS default is. >>>>>> >>>>>> I still need to retest (I only ran TooSmallStackSize.java), but figure >>>>>> getting agreement on the changes first would be best before I bog down >>>>>> our testing resources. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Please review the following: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webr >>>>>>> ev.hotspot >>>>>>> >>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>> even though I was specifying -Xss72k. It turns out the following code >>>>>>> was failing: >>>>>>> >>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>> >>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k should >>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>> result is pthread_attr_setstacksize() had no impact on the thread's >>>>>>> stack size, and we ended up with the platform default of 8mb. The fix >>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>> computing their new values: >>>>>>> >>>>>>> _java_thread_min_stack_allowed >>>>>>> _compiler_thread_min_stack_allowed >>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>> >>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>>>> #defined, and this needs to be done before including OS header files. >>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>> >>>>>>> And speaking of the existing use of thr_min_stack(), I deleted it. It >>>>>>> was being applied before any adjustments to the stack sizes had been >>>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>>> the stack ended up being larger than necessary. With the above fix in >>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>>> now too small, the correct fix is to adjust the initial stack sizes, >>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>> nightly testing on all affect platforms, and no new problems turned >>>>>>> up. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>> >> > From thomas.stuefe at gmail.com Tue Mar 21 08:59:41 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 21 Mar 2017 09:59:41 +0100 Subject: RFR(xs): 8170520: Make Metaspace ChunkManager counters non-atomic Message-ID: Hi, https://bugs.openjdk.java.net/browse/JDK-8170520 http://cr.openjdk.java.net/~stuefe/webrevs/METASPACE-2-8170520-Make-Metaspace-ChunkManager-Counters-NonAtomic/jdk10-webrev.00/webrev/ may I please get a review of another small cleanup change to the metaspace. Compared with the last one (JDK-8170933), this one is smaller. I posted this originally for jdk 9 last november, and it got reviewed already: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/ 2016-November/021946.html. Mikael found a small bug and by then it was too late to bring this into jdk9, so I postponed the patch for jdk10. Now the patch got rebased to Jdk10, and it is also quite a bit simpler because it meshes well with the cleanup work done on JDK-8170933. The change replaces the calls to Atomic::inc/dec for the ChunkManager counters with simple +/-, because all code paths which modify the ChunkManager counters are under lock protection (SpaceManager::expand_lock()). This makes updating these counters cheep and thus removes the need to be frugal with the number of updates. Before the patch - when a list of chunks was returned to a ChunkManager in ~SpaceManager() - the increment values were updated once, with just one call to ChunkManager::inc_free_chunks_total(). This left a time window in which the counters did not reflect reality, so one had to be really careful where to place asserts to check the ChunkManager state. That made modifying and playing with the code error prone. Since JDK-8170933, chunks are returned to the ChunkManager via one common path, which always ends in ChunkManager::return_single_chunk(). Because of that and because updating the counters is now cheap, I moved the several invocations of ChunkManager::inc_free_chunks_total() to ChunkManager::return_single_chunk(). The rest of the changes is cosmetical: - Moved ChunkManager::inc_free_chunks_total() up to the private section of the class, because it is not called from outside anymore - renamed arguments for ChunkManager::inc_free_chunks_total() and ChunkManager::dec_free_chunks_total() to be clearer named, and gave both of them the same arguments - Added an assert to both function asserting that the increment/decrement word size value should be a multiple of the smallest chunk size I ran gtests and jtreg tests on Linux and AIX, no issues popped up. Thank you for reviewing, Thomas From ioi.lam at oracle.com Tue Mar 21 09:25:02 2017 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 21 Mar 2017 17:25:02 +0800 Subject: C++ inlining of references Message-ID: <58D0F16E.9070608@oracle.com> How good are C++ compilers with optimizing references in inline functions? I have 2 functions that have a large common header: foo() { int x = xxx(); int y = yyy(); dofoo(x, y); } bar() { int x = xxx(); int y = yyy(); dobar(x, y); } So I want to refactor it to inline void common(int&x, int &y) { x = xxx(); y = yyy(); } foo() { int x, y; common(x, y); dofoo(x, y); } bar() { int x, y; common(x, y); dobar(x, y); } Because the common header modifies 2 variables, there's no easy way to pass them back easily. So I pass them as references to the inline function. Can I expect modern C++ compilers to always generate equally optimal code for the before/after cases? Thanks - Ioi From thomas.schatzl at oracle.com Tue Mar 21 09:36:24 2017 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 21 Mar 2017 10:36:24 +0100 Subject: C++ inlining of references In-Reply-To: <58D0F16E.9070608@oracle.com> References: <58D0F16E.9070608@oracle.com> Message-ID: <1490088984.3754.2.camel@oracle.com> Hi Ioi, On Tue, 2017-03-21 at 17:25 +0800, Ioi Lam wrote: > How good are C++ compilers with optimizing references in inline? > functions? I have 2 functions that have a large common header: > > foo() { > ?????int x = xxx(); > ?????int y = yyy(); > ?????dofoo(x, y); > } > bar() { > ?????int x = xxx(); > ?????int y = yyy(); > ?????dobar(x, y); > } > > So I want to refactor it to > > inline void common(int&x, int &y) { > ?????x = xxx(); > ?????y = yyy(); > } > > foo() { > ?????int x, y; > ?????common(x, y); > ?????dofoo(x, y); > } > bar() { > ?????int x, y; > ?????common(x, y); > ?????dobar(x, y); > } > > Because the common header modifies 2 variables, there's no easy way > to pass them back easily. So I pass them as references to the inline > function. > > Can I expect modern C++ compilers to always generate equally optimal? > code for the before/after cases? ? try it yourselves:?https://godbolt.org/?:) Thanks, ? Thomas From aph at redhat.com Tue Mar 21 10:15:00 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 21 Mar 2017 10:15:00 +0000 Subject: C++ inlining of references In-Reply-To: <58D0F16E.9070608@oracle.com> References: <58D0F16E.9070608@oracle.com> Message-ID: On 21/03/17 09:25, Ioi Lam wrote: > Because the common header modifies 2 variables, there's no easy way to > pass them back easily. So I pass them as references to the inline function. > > Can I expect modern C++ compilers to always generate equally optimal > code for the before/after cases? Yes. Note that there is unlikely to be any advantage to a reference over taking the address of a variable, because they get lowered to the same thing anyway. Use whatever is easiest for people to read. Andrew. From aph at redhat.com Tue Mar 21 14:26:15 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 21 Mar 2017 14:26:15 +0000 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> Message-ID: <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> On 16/03/17 18:03, Bob Vandette wrote: > If we start putting build configuration information in the version string, then where do we stop. Two separate ports is pretty large for a "build configuration issue". How about we define a unique global dynamic symbol? Then, at least, it's just a matter of using nm or equivalent to have a look at which port we have. Andrew. From bob.vandette at oracle.com Tue Mar 21 15:25:58 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 21 Mar 2017 11:25:58 -0400 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> Message-ID: <1FBB8985-1FE6-44E1-B338-7C6D49372D93@oracle.com> I?d prefer to use something like these System properties. "java.vendor" JRE vendor name "java.vendor.url" JRE vendor URL or to add something to the ?release? file. HOTSPOT_ARCH_DIR=arm or HOTSPOT_ARCH_DIR=aarch64 Bob. > On Mar 21, 2017, at 10:26 AM, Andrew Haley wrote: > > On 16/03/17 18:03, Bob Vandette wrote: >> If we start putting build configuration information in the version string, then where do we stop. > > Two separate ports is pretty large for a "build configuration issue". > > How about we define a unique global dynamic symbol? Then, at least, > it's just a matter of using nm or equivalent to have a look at which > port we have. > > Andrew. > From aph at redhat.com Tue Mar 21 15:27:33 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 21 Mar 2017 15:27:33 +0000 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: <1FBB8985-1FE6-44E1-B338-7C6D49372D93@oracle.com> References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> <1FBB8985-1FE6-44E1-B338-7C6D49372D93@oracle.com> Message-ID: On 21/03/17 15:25, Bob Vandette wrote: > I?d prefer to use something like these System properties. > > "java.vendor" JRE vendor name > "java.vendor.url" JRE vendor URL > > or to add something to the ?release? file. > > HOTSPOT_ARCH_DIR=arm > > or > > HOTSPOT_ARCH_DIR=aarch64 OK, sounds good. I'll have a think. Andrew. From vladimir.x.ivanov at oracle.com Tue Mar 21 16:37:00 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 21 Mar 2017 19:37:00 +0300 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <3553280e-6955-a574-8154-19cb52409300@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> Message-ID: <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> > and webrev.2 with it removed: > > http://cr.openjdk.java.net/~dlong/8158168/webrev.2/ Thanks, Dean. I started with webrev.2 and tried to minimize the changes. I ended up with the following version: http://cr.openjdk.java.net/~vlivanov/dlong/8158168/webrev.00/ Some clarifications: ============ src/java.base/share/classes/java/lang/String.java: The bounds check is needed only in String.nonSyncContentEquals when it extracts info from AbstractStringBuilder. I don't see how out of bounds access can happen in String.contentEquals: if (n != length()) { return false; } ... for (int i = 0; i < n; i++) { if (StringUTF16.getChar(val, i) != cs.charAt(i)) { ============ src/java.base/share/classes/java/lang/StringConcatHelper.java: I think bounds checks in StringConcatHelper.prepend() are skipped intentionally, since java.lang.invoke.StringConcatFactory constructs method handle chains which already contain bounds checks: array length is precomputed based on argument values and all accesses are guaranteed to be in bounds. ============ src/java.base/share/classes/java/lang/StringUTF16.java: + static void putChar(byte[] val, int index, int c) { + assert index >= 0 && index < length(val) : "Trusted caller missed bounds check"; Unfortunately, asserts can affect inlining decisions (since they increase bytecode size). In order to minimize possible performance impact, I suggest to remove them from the fix targeting 9. ============ private static int indexOfSupplementary(byte[] value, int ch, int fromIndex, int max) { if (Character.isValidCodePoint(ch)) { final char hi = Character.highSurrogate(ch); final char lo = Character.lowSurrogate(ch); + checkBoundsBeginEnd(fromIndex, max, value); The check is redundant here. fromIndex & max are always inbounds by construction: public static int indexOf(byte[] value, int ch, int fromIndex) { int max = value.length >> 1; if (fromIndex < 0) { fromIndex = 0; } else if (fromIndex >= max) { // Note: fromIndex might be near -1>>>1. return -1; } ... return indexOfSupplementary(value, ch, fromIndex, max); ============ I moved bounds checks from StringUTF16.lastIndexOf/indexOf to ABS.indexOf/lastIndexOf. I think it's enough to do range check on ABS.value & ABS.count. After that, all accesses should be inbounds by construction (in String.indexOf/lastIndexOf): jdk/src/java.base/share/classes/java/lang/StringUTF16.java: static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, String tgtStr, int fromIndex) { int rightIndex = srcCount - tgtCount; if (fromIndex > rightIndex) { fromIndex = rightIndex; } if (fromIndex < 0) { return -1; } jdk/src/java.base/share/classes/java/lang/StringUTF16.java: public static int lastIndexOf(byte[] src, int srcCount, byte[] tgt, int tgtCount, int fromIndex) { int min = tgtCount - 1; int i = min + fromIndex; int strLastIndex = tgtCount - 1; char strLastChar = getChar(tgt, strLastIndex); startSearchForLastChar: while (true) { while (i >= min && getChar(src, i) != strLastChar) { There are 2 places: * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - inbound * getChar(src, i); i in [ min; min+fromIndex ] min = tgtCount - 1 rightIndex = srcCount - tgtCount fromIndex <= rightIndex 0 <= min + fromIndex <= min + rightIndex == (tgtCount - 1) + (srcCount - tgtCount) == srcCount - 1 Hence, should be covered by the check on count & value: public int lastIndexOf(String str, int fromIndex) { + byte[] value = this.value; + int count = this.count; + byte coder = this.coder; + checkIndex(count, value.length >> coder); return String.lastIndexOf(value, coder, count, str, fromIndex); } Best regards, Vladimir Ivanov > On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >> >>>> I have the same concern. Can we fix the immediate problem in 9 and >>>> integrate verification logic in 10? >>>> >>> >>> OK, Tobias is suggesting having verification logic only inside the >>> intrinsics. Are you suggesting removing that as well? >> >> Yes and put them back in 10. >> >>> I'm OK with removing all the verification, but that won't reduce the >>> library changes much. I could undo the renaming to Trusted.getChar, but >>> we would still have the bounds checks moved into StringUTF16. >> >> I suggest to go with a point fix for 9: just add missing range checks. > From thomas.stuefe at gmail.com Tue Mar 21 17:46:31 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 21 Mar 2017 18:46:31 +0100 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: Hi Chris, On Tue, Mar 21, 2017 at 6:43 PM, Chris Plummer wrote: > Hi Thomas, > > On 3/21/17 1:08 AM, Thomas St?fe wrote: > > Hi Chris, > > looks fine to me. > > small nit: os_linux_cpp: > > > + // In that cast just subract the page size to get the maximum possible > stack size. > > subract->substract > > Ok. I had the same typo in os_posix.cpp. > > > I would have preferred SIZE_MAX instead of UINT_MAX, but both is fine to > me. > > I'll use SIZE_MAX. I didn't see it in limits.h so I didn't know of it's > existence. Turns out it is in stdint.h. > > > -- > > As for the size_overflow problem, I wonder whether we are overthinking > this. Maybe a more pragmatic approach would have been to just define a > reasonable upper maximum for stack sizes which could be well below > UINT_MAX. Somehow I cannot see users wanting to start threads with more > than ~4GB of thread stack size. > > I had debated doing this. I decided not to artificially limit the stack > size for a couple of reasons. (1) I figured we should make a best effort to > use the size the user requested. (2) There is no obvious large size to > choose other than the max size. If we choose a smaller size it might still > fail or might be smaller than the largest size that will succeed. > > Makes sense. > Thanks again for the review. > > Sure, thanks for taking my input. Kind regards, Thomas Chris > > > Kind Regards, Thomas > > > > On Tue, Mar 21, 2017 at 12:29 AM, Chris Plummer > wrote: > >> On 3/17/17 11:37 PM, Chris Plummer wrote: >> >>> On 3/17/17 8:17 PM, Chris Plummer wrote: >>> >>>> On 3/17/17 7:01 PM, David Holmes wrote: >>>> >>>>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>>> >>>>>> Looks like this will need some more work since the added asserts are >>>>>> triggering on mac os x (which is the only place we'd currently expect >>>>>> them to assert). >>>>>> >>>>>> The problem is that the code I found that rounds up to the page size >>>>>> is >>>>>> only applied to java threads created by the VM for which the java user >>>>>> specified no stack size. The VM and Compiler thread sizes are not >>>>>> rounded. The failure I saw was with >>>>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>>> when is >>>>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit the >>>>>> assert >>>>>> with an EINVAL. The size is not aligned, but it could also be >>>>>> complaining because it is too big. I haven't tried aligning it yet to >>>>>> see. >>>>>> >>>>>> On Linux we do the following: >>>>>> >>>>>> stack_size = align_size_up(stack_size + >>>>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>>>> >>>>>> We don't do this on BSD. I think the same on BSD would solve this >>>>>> problem. I'm not sure about adding the guard size. I'll need to see if >>>>>> mac os x has the same pthread bug as linux does. >>>>>> >>>>> >>>>> At this stage I would only deal with alignment issues. IIRC the guard >>>>> issue only affected Linux. >>>>> >>>> Yes, that's what I eventually concluded. I put the fix in >>>> os::Posix::get_initial_stack_size() in os_posix.cpp, and only did the >>>> page aligning, not add the guard page. That way all Posix ports are fixed >>>> in one place. It seems to be working. >>>> >>>>> >>>>> BTW, did you know java users can specify the size of the a new thread's >>>>>> stack: >>>>>> >>>>> >>>>> Yes I mentioned that in another reply - wondering whether we suitably >>>>> check and aligned such requests. >>>>> >>>> No we don't. Below I mentioned I was able to trigger the assert with a >>>> 257k stack size. I guess I wasn't clear that I did that from Java. I have a >>>> new test to add that will be testing this, plus the 9007199254740991 stack >>>> size (which fails to create the thread with an OOME, but that's >>>> acceptable). The fix I mention above in os::Posix::get_initial_stack_size() >>>> takes care of this issue also. >>>> >>> Rounding up triggers a new assert, this time on 32-bit x86 and arm. >>> >>> I should have clarified it's 9007199254740991 "K", which is >>> 9223372036854774784. Unfortunately on 32bit systems that is asserting with >>> EINVAL. I think we need to do a better job of dealing with 32-bit size_t >>> values: >>> >>> jlong java_lang_Thread::stackSize(oop java_thread) { >>> if (_stackSize_offset > 0) { >>> return java_thread->long_field(_stackSize_offset); >>> } else { >>> return 0; >>> } >>> } >>> >>> jlong size = >>> java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); >>> // Allocate the C++ Thread structure and create the native >>> thread. The >>> // stack size retrieved from java is signed, but the constructor >>> takes >>> // size_t (an unsigned type), so avoid passing negative values >>> which would >>> // result in really large stacks. >>> size_t sz = size > 0 ? (size_t) size : 0; >>> native_thread = new JavaThread(&thread_entry, sz); >>> >>> 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit MAX_INT), >>> which is 0xfffffc00 when cast to a size_t on a 32-bit system (close to >>> 32-bit MAX_UINT). Round it up to the 4k page size and you get 0, which I >>> guess pthread_attr_setstacksize() doesn't like. So I think more processing >>> of the size is needed here. Maybe in os::create_thread() we should check >>> for 0 after rounding up, and subtract the os page size if it is 0. However, >>> I think we should also avoid truncating on 32-bit to what is basically some >>> random number. Maybe if "size" (a jlong) is greater than UINT_MAX, then >>> just set "sz" (a size_t) it to UINT_MAX. >>> >>> Ok, I think I have this all worked out now. I've added fixes for >> unaligned stack sizes, 32-bit truncating of stack size, and the "aligning >> up to 0" problem. I also added a test. Here's the latest webrev: >> >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >> >> Here's what's changed since webrev.01: >> >> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up the >> stack size to be paged aligned. This fixes issues on Mac OS X (other >> platforms seem to be immune to this). Then check if the size is zero after >> rounding up to the page size. Subtract the page size in this case to >> produce the maximum stack size allowed. Surprisingly I got no complaint >> from gcc for subtracting from an unsigned value that is known to be 0. >> >> os_linux.cpp: In os::create_thread(), I also check here to make sure the >> size is not 0 after adding the guard page and aligning up, and subtract the >> os page size if it is 0. >> >> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater >> than UINT_MAX, then I set the size to UINT_MAX. Note it will later be >> rounded up to 0 in os::Posix::get_initial_stack_size(), which will >> result in subtracting the os page size to get the actual maximum allowed >> stack size. >> >> TooSmallStackSize.java: added test case for unaligned stack sizes. >> >> TestThreadStackSizes.java: New test. Creates new threads with every size >> up to 320k in 1k increments. Then creates threads with a few other sizes >> that can be problematic. >> >> thanks, >> >> Chris >> >> Chris >>> >>> >>> >>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>> long stackSize) { >>>>>> init(group, target, name, stackSize); >>>>>> } >>>>>> >>>>>> Fortunately we already force the stackSize to be at least >>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>> assert by creating a thread with size 257k. >>>>>> >>>>> >>>>> Note this means that OSX stack logic is broken because it will >>>>> currently be silently failing due to EINVAL! >>>>> >>>> Yes, that is correct. >>>> >>>> Chris >>>> >>>>> >>>>> I'll get another webrev out once I've made the needed fixes. I also >>>>>> have >>>>>> a new test I'd like to add. >>>>>> >>>>> >>>>> Ok. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> Chris >>>>>> >>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>> >>>>>>> Ok, time for a new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webr >>>>>>> ev.hotspot >>>>>>> >>>>>>> The only thing that has changed since the first webrev are the >>>>>>> asserts >>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>> discuss >>>>>>> already: >>>>>>> >>>>>>> - The assert should never happen due to the stack size being too >>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>> because we always align it to the page size. >>>>>>> - Any assert would therefore be a VM bug and not due to user error. >>>>>>> - No fixing the java launcher. If the user specifies a stack that is >>>>>>> too small, hotspot will already detect this. If the user specifies a >>>>>>> stack size that is large enough but not page aligned, then we just >>>>>>> ignore any error (if the platform doth protest) and the user gets a >>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>> >>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>> figure >>>>>>> getting agreement on the changes first would be best before I bog >>>>>>> down >>>>>>> our testing resources. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the following: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webr >>>>>>>> ev.hotspot >>>>>>>> >>>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>>> code >>>>>>>> was failing: >>>>>>>> >>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>> >>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k should >>>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>> result is pthread_attr_setstacksize() had no impact on the thread's >>>>>>>> stack size, and we ended up with the platform default of 8mb. The >>>>>>>> fix >>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>>> computing their new values: >>>>>>>> >>>>>>>> _java_thread_min_stack_allowed >>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>> >>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You need to >>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get PTHREAD_STACK_MIN >>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>> files. >>>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>> >>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted it. >>>>>>>> It >>>>>>>> was being applied before any adjustments to the stack sizes had been >>>>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>>>> the stack ended up being larger than necessary. With the above fix >>>>>>>> in >>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>>>> now too small, the correct fix is to adjust the initial stack sizes, >>>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>> nightly testing on all affect platforms, and no new problems turned >>>>>>>> up. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> >>> >> > > From igor.veresov at oracle.com Tue Mar 21 18:40:10 2017 From: igor.veresov at oracle.com (Igor Veresov) Date: Tue, 21 Mar 2017 11:40:10 -0700 Subject: RFR(XXXL): 8177046 Update Graal Message-ID: <122A648A-11A6-4264-ABC1-31D4B68308CA@oracle.com> This is Graal refresh in JDK10. It required some changes to jaotc mostly related how Graal options are now handled. It all seems to work - aot, jprt, rbt hs-tier0-comp. There is one failure in rbt which occurs when Graal is used as a JIT. But I think it can be addressed later and fixed in the next refresh (if there are no objections). Webrevs: http://cr.openjdk.java.net/~iveresov/8177046/webrev-jdk.00/ http://cr.openjdk.java.net/~iveresov/8177046/webrev-hotspot.00/ Changes between JDK-8166417 and JDK-8177046 : commit 532c57329aa3a453ff4d2fefeb6fddc3cc03b8aa Merge: 7347014 bd02b2e Author: Thomas W?rthinger Date: Fri Mar 17 16:50:06 2017 -0700 [GR-3285] Fixes to support JDK AOT use cases correctly. commit 7347014a9ca6751f9f7c27436bdad8d50e7ea168 Merge: f06c39e 6d22782 Author: Doug Simon Date: Fri Mar 17 16:21:09 2017 -0700 [GR-3277] Gate on jdk9-ea+161. commit f06c39e247e74ec36d88af1552ee5a19d36a7bb0 Merge: 099df9f 52727fa Author: Thomas W?rthinger Date: Fri Mar 17 15:35:44 2017 -0700 [GR-3275] Add new phase that correctly propagates deoptimize probabilities upwards through the graph to control splits. commit bd02b2edb414a309e4ba68261c5f936ef0e0a40d Author: Igor Veresov Date: Fri Mar 17 12:16:55 2017 -0700 Fix a few things to make JDK AOT work agian commit 52727fadfae472ac361c8af02e015eed9730d7dd Author: Thomas W?rthinger Date: Fri Mar 17 15:43:05 2017 +0100 Rewrite recursive algorithm in PropagateDeoptimizeProbabilityPhase to be iterative. commit 145bebb7ebceea51c375ed715ad7dd82f002e6d4 Author: Thomas W?rthinger Date: Fri Mar 17 15:22:36 2017 +0100 Add new phase that correctly propagates deoptimize probabilities upwards through the graph to control splits. commit 099df9ff553086d10a4369f692559e7035fc24bf Merge: 31e5340 5dfcdf0 Author: Thomas W?rthinger Date: Fri Mar 17 11:01:30 2017 -0700 [GR-3274] Add additional canonicalizations to the multiplication node for the case where one operand is constant. commit 6d227828278a32f78defcb78fcc58bdd19b88066 Author: Doug Simon Date: Fri Mar 17 18:31:37 2017 +0100 gate on jdk9-ea+161 commit 5dfcdf0ae1902818cb681da9f2b81b5fedb5e50f Author: Thomas W?rthinger Date: Fri Mar 17 16:48:32 2017 +0100 Add additional canonicalizations to the multiplication node for the case where one operand is constant. commit 31e5340042fc8bb9fb22b5704da7189e3e05e9e8 Merge: eb57bf9 cc28520 Author: Doug Simon Date: Fri Mar 17 07:58:15 2017 -0700 [GR-3265] Support jdk9-ea+161. commit cc2852014b474b44e4f5cd685885d347c644701f Author: Doug Simon Date: Fri Mar 17 01:11:00 2017 +0100 support jdk9-ea+161 commit eb57bf9a2e3ec66dc82f8ec6ded118f596535632 Merge: 1f9dd5e 3b24c57 Author: Andreas Woess Date: Fri Mar 17 06:06:28 2017 -0700 [GR-3256] Fix TraceTruffleAssumptions stack trace output. commit 1f9dd5e79298f1a36d1bed99f3d24017052f5f85 Merge: ef5624c aa243ee Author: Doug Simon Date: Fri Mar 17 03:50:27 2017 -0700 [GR-3267] JMH intrinsics need to be optional. commit aa243ee242d6e6c1fdd8a615407051b5609bcd00 Author: Doug Simon Date: Fri Mar 17 10:24:22 2017 +0100 JMH intrinsics need to be optional commit ef5624c9839fe9258dbc858b759a7821187d4a89 Merge: 6a44fba f741158 Author: Thomas W?rthinger Date: Thu Mar 16 07:39:13 2017 -0700 [GR-3149] Fix an issue in the conditional elimination that could lead to pi nodes for phis picking up wrong guards. commit f7411584e6bb957a83b47b4591b1e9163160c654 Author: Thomas W?rthinger Date: Thu Mar 16 15:06:53 2017 +0100 Fix an issue in the new conditional elimination that could lead to pi nodes for phis picking up wrong guards if branches are killed while preparing the info elements. commit 3b24c57e9d227d390b1b15773458c98ce27db065 Author: Andreas Woess Date: Thu Mar 16 13:48:52 2017 +0100 Enable node source positions for Truffle compilations if shouldDebugNonSafepoints() or TruffleEnableInfopoints=true commit d214ec43996729def84313aa7316d342e5cc63ce Author: Andreas Woess Date: Wed Mar 15 21:39:09 2017 +0100 Remove leftover debug code commit 829ceef47a7b14fd13e291d5dd1228d7c1990990 Author: Andreas Woess Date: Wed Mar 15 18:19:23 2017 +0100 Cleanup: no need to subclass CachingPEGraphDecoder commit 001322b49be3822638d8a9e1652692dedaa57784 Author: Andreas Woess Date: Wed Mar 15 19:17:39 2017 +0100 [GR-3256] Fix TraceTruffleAssumptions stack trace output. commit 6a44fba8e8004fe12934dc31e4f5dbb8dc6b26b3 Merge: 2cdb24c 1fd6484 Author: Doug Simon Date: Wed Mar 15 17:38:06 2017 -0700 [GR-3226] Add ZipPy based OOPSLA14 paper. commit 2cdb24c193d7f491dbd2dcbb0ae115bdcdc8c689 Merge: 1879aca 7afd70a Author: Doug Simon Date: Wed Mar 15 05:45:42 2017 -0700 [GR-3246] Handle exceptions while opening a Debug scope better. commit 7afd70a750ffbe2f2df2885c837f2e5d94609264 Author: Doug Simon Date: Wed Mar 15 12:53:25 2017 +0100 better reporting of exception when opening a Debug scope commit 1879acafd6342623873b2cc69132960abb4e327e Merge: 2e50f8e e3e8dfe Author: Thomas W?rthinger Date: Wed Mar 15 04:17:21 2017 -0700 [GR-3245] Improve canonicalizations of NarrowNode and ZeroExtendNode. Add additional canonicalization after fixing reads. commit e3e8dfeb54443fcc0f5772f3273cf86b899e3365 Author: Thomas W?rthinger Date: Wed Mar 15 11:22:45 2017 +0100 Use result bits instead of stamp in ZeroExtendNode. commit 1fc9abbc64212df38add4df80bd45bead25451a8 Author: Thomas W?rthinger Date: Tue Mar 14 19:20:24 2017 +0100 Also clear last location access in fix reads phase. commit 9ecb01147f54dd7808dcbf0ee0147427e7f7b04d Author: Thomas W?rthinger Date: Tue Mar 14 19:13:12 2017 +0100 Make sure short circuit or nodes are not introduced by canonicalization after logic expansion phase was run. commit 9aa8fd04caec7458d78b5fb14be771a7464aa9ca Author: Thomas W?rthinger Date: Tue Mar 14 16:26:36 2017 +0100 Remove memory phis during fix reads phase. commit 7ae30083e7e8f8d6ab163e2caf5da731032c125c Author: Thomas W?rthinger Date: Tue Mar 14 16:26:09 2017 +0100 Run a canonicalizer without GVN after fixing reads and removing pi nodes. commit 9f496abc97c020e406f2073d1ff0d8bd4730224a Author: Thomas W?rthinger Date: Tue Mar 14 16:25:48 2017 +0100 Add ability to disable GVN in canonicalizer. commit 409d5d42f9929a4790a6d0ec30c7b8188a61e660 Author: Thomas W?rthinger Date: Tue Mar 14 15:51:56 2017 +0100 Improved canonicalizations for NarrowNode and ZeroExtendNode. commit 2e50f8edb5a3f33f772510b1eb0480a28eac7db2 Merge: 8ffb5cf cf1dd8f Author: Thomas W?rthinger Date: Wed Mar 15 03:11:00 2017 -0700 [GR-3244] Load all non simm13 constants for OP3 ops. commit cf1dd8ff253c115efba7e6ec5ed9496fa45aad8e Author: Stefan Anzinger Date: Wed Mar 15 09:22:01 2017 +0000 [GR-3244] Load all non simm13 constants for OP3 ops. commit 8ffb5cf6ff82a5bf33f265da780c3e4a322f2127 Merge: 4182260 92abdc9 Author: Doug Simon Date: Tue Mar 14 17:59:03 2017 -0700 [GR-3243] Make check for option descriptor existence overridable. commit 92abdc9d0a82db0b2679d92ac834e8cd70b8d9ea Author: Doug Simon Date: Wed Mar 15 00:20:06 2017 +0100 allow option descriptor existence check to be overridden commit 418226004e091c7b511c97ee62467c138bd9bf0f Merge: ff3feb9 c2a2447 Author: Doug Simon Date: Tue Mar 14 15:16:04 2017 -0700 [GR-3212] [GR-3057] Replace PiNode+NodeIntrinsicStamp with placeholder node. commit c2a2447ad86f404b66e2c102e375cfcb00ab34aa Author: Doug Simon Date: Mon Mar 13 17:27:45 2017 +0100 use placeholders for PiNodes that get their stamp from a snippet replacee commit ff3feb9b4d246779f37d551522782c9ef66243dd Merge: 8d3e056 853af95 Author: Doug Simon Date: Tue Mar 14 14:45:33 2017 -0700 [GR-3241] JDK9 aware mechanism for options loading must be universally applied. commit 853af9549616df9829af52860c8a2ee9bc40150d Author: Doug Simon Date: Tue Mar 14 21:56:53 2017 +0100 loading options must always be jdk9-modules aware commit 8d3e05605e59da9c4d576895c81ae04b4250efe8 Merge: 2cfd95a 3dac632 Author: Doug Simon Date: Tue Mar 14 14:13:36 2017 -0700 [GR-3240] Remove static field Stub.stubs. commit 1fd64845a4f1dc9b17d75edf045d4e88464d43f6 Author: Doug Simon Date: Tue Mar 14 22:11:19 2017 +0100 add "Accelerating iterators in optimizing AST interpreters" paper (OOPSLA14) commit 3dac632a0a789232cd405a622ef973ed3d2bc816 Author: Doug Simon Date: Tue Mar 14 21:04:19 2017 +0100 replace Stub.getStubs() with HotSpotForeignCallProvider.getStubs() commit 2cfd95a6a5ab2cd0785c67a1fe5264a5c3fdb42d Merge: 06fbf00 492bf4f Author: Thomas W?rthinger Date: Tue Mar 14 12:44:05 2017 -0700 [GR-3238] Improve killCFG to reliably delete nodes. commit 492bf4f48c3a7e8f5bfd0060b0ca681cbb0ddd5b Author: Gilles Duboscq Date: Thu Mar 9 17:42:44 2017 +0100 [GR-3195] Fix killCFG leaving 'unsafe' inputs Avoid using null for the input of Phi nodes, use Poison instead. commit 06fbf000db55cea8c15e0099415010adcb4984b0 Merge: 21ec724 0da5287 Author: Thomas W?rthinger Date: Tue Mar 14 06:53:40 2017 -0700 [GR-3227] Simplify and improve address lowering to swap index and base when appropriate. commit 0da528791a95721947a41ec5a33e65726ae443b8 Author: Thomas W?rthinger Date: Tue Mar 14 14:03:05 2017 +0100 Simplify and improve address lowering to swap index and base when appropriate. commit 21ec724defa8a0f08bbe5a512f45d0b8ed5146ad Merge: 7a12664 8fdd855 Author: Doug Simon Date: Tue Mar 14 05:47:15 2017 -0700 [GR-3222] Try detect missing @Option annotation processor. commit 7a1266444ead236c3dbb727b7faf39e3436f6d6c Merge: e78391d 6d84784 Author: Thomas W?rthinger Date: Tue Mar 14 04:26:27 2017 -0700 [GR-3225] Recognize implicit null checks as part of trapping null checks phase. commit 6d84784cb648102c191a1712e3dbfaad5a69f09e Author: Thomas W?rthinger Date: Tue Mar 14 01:44:07 2017 +0100 Recognize implicit null checks as part of trapping null checks phase. commit 8fdd8558be437b463cee5f04565eff2319530744 Author: Doug Simon Date: Mon Mar 13 23:38:55 2017 +0100 try detect missing @Option annotation processor dependency commit e78391dedad2787f274f28f156279472deb9aa45 Merge: bd6f7bf 89e208f Author: Thomas W?rthinger Date: Mon Mar 13 12:59:01 2017 -0700 [GR-3218] Fix a bug in the EconomicMap implementation that could lead to skipping an element during iteration when removing an element triggers a compression. commit 89e208f171d14306d1d6746e208ccb049e1df4be Author: Thomas W?rthinger Date: Mon Mar 13 20:32:02 2017 +0100 Fix a bug in the EconomicMap implementation that could lead to skipping an element during iteration when removing an element triggers a compression. commit bd6f7bffb0020b21a0d7b9628bd21bf03cd9b2bc Merge: 95ddee1 6f36b03 Author: Thomas W?rthinger Date: Mon Mar 13 10:48:35 2017 -0700 [GR-3217] Improve code for pointer compression. commit 6f36b03649c4012fbb2ed8656db4b01e451eb9fe Author: Thomas W?rthinger Date: Mon Mar 13 17:33:02 2017 +0100 Introduce mayNullCheckSkipConversion method in ConvertNode interface. Move address lowering after fixed read phase. commit 74fe0007e19dced56b07155b3ee5e95c7c0ce67e Author: Thomas W?rthinger Date: Mon Mar 13 15:16:07 2017 +0100 Add option to avoid barrier on unsafe stores. Rename unsafe store and load nodes to "raw" unsafe store and load nodes. commit 4a53d32d315925b766337f030cdaaf52ff0686be Author: Thomas W?rthinger Date: Thu Mar 9 14:18:15 2017 +0100 Fix write barrier verification phase. commit 9f67aa8d206369fba444345e4cc16389af8ebfd1 Author: Thomas W?rthinger Date: Wed Mar 8 17:56:31 2017 +0100 Adjust compressed null check test to not modify the graph and for the fact that this canonicalization now occurs in the IsNullNode. commit 45523aac92e8fcf2f4cb6725a70952ee0caea510 Author: Thomas W?rthinger Date: Wed Mar 8 15:32:14 2017 +0100 Add skip through lossless convert functionality to IsNullNode. commit 5ca64dd073772f43f10173cb7cef400823624dd9 Author: Thomas W?rthinger Date: Sun Feb 26 21:28:14 2017 +0100 Introduce init location and use this location to decide whether a write is an initialization write or not. commit 19ce207634905a612f94a441bd3e4881c9699995 Author: Thomas W?rthinger Date: Sun Feb 26 20:27:54 2017 +0100 Simplify ReadNode and WriteNode constructors. commit 05a3d7d8e53d16ec243d364ef24b6ef492b4cf0f Author: Thomas W?rthinger Date: Sun Feb 26 18:28:18 2017 +0100 Remove DirectObjectStoreNode and replace usages with UnsafeStoreNode. commit 60ba844f0bb94bbc9a0e4bdad2112cbc92fc72b4 Author: Thomas W?rthinger Date: Sun Feb 26 17:34:29 2017 +0100 Move CompressEncoding from HotSpot-specific to general part. Remove unused alignment field. commit 95ddee1ea392e4454b83264e4c0ef728821d0fad Merge: 64194d0 b560415 Author: Gilles Duboscq Date: Mon Mar 13 08:26:22 2017 -0700 [GR-2555] Fold `x - 1 |<| x || x == 0`. commit b56041582a6bbc19d871c5227b7fd6eff89f320b Author: Gilles Duboscq Date: Mon Mar 13 15:03:00 2017 +0100 ZeroExtend: avoid problems with empty stamps commit 64194d0b9be6e22ef948abaae3300b8051844407 Merge: 7281bc5 284e39f Author: Christian Humer Date: Mon Mar 13 05:59:25 2017 -0700 [GR-3182] Do not reuse frame descriptor for OSR root nodes. commit 42eb5d00d9abc466c8faa9f4bde5551a0807aa3c Author: Gilles Duboscq Date: Mon Feb 27 16:29:39 2017 +0100 Add NumUtil.sameSign and IntegerStamp.sameSignBounds commit 3dee10a429eef95104c192473a89909ef2d61f5f Author: Gilles Duboscq Date: Fri Feb 24 17:45:26 2017 +0100 Make sure splitIfAtPhi is not appliest before FSA commit 2163d91dac75656609a5b160f69e58db4ac9cbfd Author: Gilles Duboscq Date: Fri Feb 24 17:43:19 2017 +0100 Add more tests commit 1d453dfc3b7df282a842e1cb01d14d52e9d6ba80 Author: Gilles Duboscq Date: Fri Feb 24 17:41:18 2017 +0100 Canonicalize NormalizeCompare patterns commit e196bac444f1e95c018d0909af12d9a049b4be24 Author: Gilles Duboscq Date: Tue Mar 7 07:57:42 2017 -0800 Canonicalize `a + MIN < b + MIN` to `a |<| b` commit 2ef1eb30f194d140bbc762161dc5664c861f1c62 Author: Gilles Duboscq Date: Fri Feb 24 18:10:40 2017 +0100 Canonicalize x + c < x commit 85cc9ffe2a6b6e2d64c2a36a0e2ee9ab0b0a491b Author: Gilles Duboscq Date: Fri Feb 24 17:35:33 2017 +0100 Canonicalize x + c1 == c2 commit 4567b9e3602866ed367a8eea12959d7ba4d84f9a Author: Gilles Duboscq Date: Fri Jan 27 18:33:32 2017 +0100 [GR-2555] Fold `x - 1 |<| x || x == 0`. Added the `y + c |<| y` and `x |<| x + c` patterns for conditional elimination. Added `StampInverter`, make `IntegerConvert` implement it and use it in the new conditional elimination. This allows it to get a stamp for x when it has one for zeroExtand(x). commit 878a6d72927e4ca53f4ed028badde8be24e905a5 Author: Gilles Duboscq Date: Tue Feb 14 15:46:17 2017 +0100 Move NumUtil to core.common Also remove common and merge it into core.common commit 7281bc5cc25cb83e4ec7414b4804d33dab44384c Merge: a0848f5 a8a5023 Author: Doug Simon Date: Fri Mar 10 12:42:32 2017 -0800 [GR-3196] Remove obsolete ImmutableCode functionality. commit a8a50239411881f0f9f7a0fa215d7c38447ffdf1 Author: Doug Simon Date: Fri Mar 10 15:42:29 2017 +0100 removed obsolete ImmutableCode logic commit a0848f59192d1a60d1b8b7842b9afd7a13cd9762 Merge: 6453e48 7eceda8 Author: Tom Rodriguez Date: Fri Mar 10 08:55:24 2017 -0800 [GR-3179] Fixed register usage for string indexof assembly. commit 6453e489e796f009e1a4bcfec15f47aa393ed1aa Merge: bd0c1e6 27c119f Author: Vojin Jovanovic Date: Fri Mar 10 08:44:20 2017 -0800 [GR-3200] Update intellij instructions. commit 27c119fa93233a9662800b8ba57b8740e4d1e2b3 Author: Vojin Jovanovic Date: Fri Mar 10 16:22:49 2017 +0100 [doc] update intellij instructions commit 08c8f8ac24c366f70be86a113416dede8597e282 Author: Doug Simon Date: Fri Mar 10 12:43:35 2017 +0100 options should be taken from graph if possible commit bd0c1e6c2c405d2735f52d7459a377ba8ed00354 Merge: 9dfc7ba ec0bd51 Author: Doug Simon Date: Fri Mar 10 01:50:19 2017 -0800 [GR-2577] Make it possible to build Graal using only jdk9. commit ec0bd511c177a86742a9c9ea1e48b40ecb15817f Author: Doug Simon Date: Thu Mar 9 22:54:20 2017 +0100 update mx version commit 9dfc7baa47e299e224dda6d98963ad4491eda5ac Merge: 1bc5b55 e4baf69 Author: Thomas W?rthinger Date: Fri Mar 10 01:18:37 2017 -0800 [GR-3188] Add receiver null check for Truffle load nodes. commit 1bc5b55d432067f53b2c3ab2ae3769c773c740db Merge: 628fd73 ac4d892 Author: Aleksandar Prokopec Date: Thu Mar 9 14:30:22 2017 -0800 [GR-3186] Add the unfreeze method to Graph. commit e4baf69da8e898edfc4aeff245b38503fae9e123 Author: Thomas W?rthinger Date: Thu Mar 9 23:11:49 2017 +0100 Add receiver null check for Truffle load nodes. commit 628fd73bc0eabd520776b6ff70e594b628824779 Merge: 999da54 653fb60 Author: Josef Eisl Date: Thu Mar 9 14:07:54 2017 -0800 [GR-3175] Run DaCapo style benchmarks on both nodes in parallel. commit 7eceda86f64250769900d6822e2b003cc3f29f50 Author: Tom Rodriguez Date: Wed Mar 8 16:33:20 2017 -0800 Fixed register usage for string indexof assembly commit 653fb601e3dec18a4bee56ca80a662a036f5d982 Author: Josef Eisl Date: Thu Mar 9 13:21:39 2017 +0100 [ci] increase tmpfs for x32 to 25g commit a664309dd7012fb10443354112153ab10d1d395a Author: Josef Eisl Date: Thu Mar 9 11:37:26 2017 +0100 [ci] use curly braces for environment variables commit a37e70600b9bbb0f584aaacbc242a15e31a38cf6 Author: Josef Eisl Date: Thu Mar 9 10:54:47 2017 +0100 [ci] rename results file environment variables commit 15a25de31654dfddb9741ff929107a55198d19f6 Author: Josef Eisl Date: Thu Feb 23 16:54:44 2017 +0100 [ci] run DaCapo-style benchmarks in parallel on multi-node machines commit ac4d892824d40e9eec14d09bdd266ff127b0e8f4 Author: Aleksandar Prokopec Date: Thu Mar 9 14:02:05 2017 +0100 [feature] Add temporary graph freeze state, and allow unfreezing a graph. commit 3df3392bb818125288ab7f5a373e56061e77caaa Author: Doug Simon Date: Thu Mar 9 16:41:11 2017 +0100 IGV doesn't yet work with JDK 9 commit 999da5413324bfb8ed4e80ee432580a18f299928 Merge: feac9f6 21a99d7 Author: Thomas W?rthinger Date: Thu Mar 9 05:09:49 2017 -0800 [GR-3072] Fix non-null stamp machinery for other non-object pointers. commit 2d278ebaced888850305403c9e231eeb22ce8c09 Author: Doug Simon Date: Thu Mar 9 13:36:39 2017 +0100 made String.indexOf intrinsic optional commit 9eea440de6c54932fa07b4035931013aea20bc76 Author: Gilles Duboscq Date: Wed Feb 1 15:34:41 2017 +0100 make it possible to build against 9 alone Use newer mx Replace uses of internal ASM in test with external ASM jars Made some substitutions optional commit 284e39fda0613ec9e0f5f4bfcf44e0b95a4b6838 Author: Lukas Stadler Date: Thu Mar 9 12:02:55 2017 +0100 fix warning problem commit 5d5de01bec53de85de0142846db01f4cdea24a35 Author: Lukas Stadler Date: Thu Mar 9 11:53:04 2017 +0100 do not reuse frame descriptor for OSR root nodes commit feac9f656ade524ae81808ee81f47cb286f6653e Merge: 3cead4d 1ed06ff Author: Thomas W?rthinger Date: Thu Mar 9 04:00:21 2017 -0800 [GR-3077] The isSingleValueUser method returns wrong answer for guards. commit 3cead4d7405b6d55d1a7e6162c1295c987d5197d Merge: e9098f7 623d6f4 Author: Thomas W?rthinger Date: Thu Mar 9 03:20:41 2017 -0800 [GR-3184] Improve out of loop schedule to take frequency of latest block and block before the loop header into account. commit e9098f79a7c052ec6efe024cf1a3d76abf198301 Merge: 4edf987 d3c879f Author: Josef Eisl Date: Thu Mar 9 03:18:16 2017 -0800 [GR-3180] Move dump-on-error flags to gate command. commit 4edf987dc522a6147117a504c328dec3aca3fd52 Merge: 78521e4 681750b Author: Stefan Anzinger Date: Thu Mar 9 03:17:11 2017 -0800 [GR-2191] Extend inliner to start with a fixed list of invocations. commit 78521e45d34bac7ab4beb745c8ab9ec555d64d5a Merge: 2f7198e 9df9b45 Author: Lukas Stadler Date: Thu Mar 9 02:17:46 2017 -0800 [GR-3115] Inline methods with substitutions during partial evaluation. commit d3c879f84babaa6daa6a6d10c9d65c5bd4a86d33 Author: Josef Eisl Date: Thu Mar 9 10:59:02 2017 +0100 [ci] add DumpOnError and PrintGraphFile flags to the gate command commit 3bc433871313065fc8e3238e6359e6759baf6ecd Author: Josef Eisl Date: Thu Mar 9 10:56:37 2017 +0100 Revert "[feature] Capture bgv files on error." This reverts commit d9c4c19dd92c6149f0a1306b48b348e66dee6b39. commit 2f7198e7f388e0d2f703a8de23c13880358c000f Merge: fde4a7c f61e0db Author: Codrut Stancu Date: Wed Mar 8 15:05:04 2017 -0800 [GR-3157] Trace constant roots. commit 623d6f44695a5670b1ee04d58f75b7894a763c3e Author: Thomas W?rthinger Date: Wed Mar 8 21:56:49 2017 +0100 Always move out of loops when value proxies are in the graph. commit 38468d9cf613d4f59a3f5706535b44f8b4f5993c Author: Thomas W?rthinger Date: Wed Mar 8 17:55:56 2017 +0100 Modify memory schedule test to include loop frequencies. commit f5353a213553b16e07a8ba4a5a54aba6e050ce5e Author: Thomas W?rthinger Date: Wed Mar 8 17:34:07 2017 +0100 Improve out of loop schedule to take frequency of latest block and block before the loop header into account. commit f61e0db76eb16726ffd2179ae2c2ee44cc075c0e Author: Codrut Stancu Date: Fri Mar 3 19:51:46 2017 -0800 Trace constant roots WIP. commit fde4a7c26c2d8d5323c51bc56e4bdc3392e73035 Merge: 8b69e82 e85077e Author: Josef Eisl Date: Wed Mar 8 10:06:13 2017 -0800 [GR-3039] Remove usage of AbstractBlockEndOp. commit e85077e645e4556c5256a545f22f9d99038ac38d Author: Josef Eisl Date: Fri Feb 24 22:09:18 2017 +0100 [fix] TraceRA: remove AbstractBlockEndOp commit 85f497f35b6bd544728f1057ad577a76add79e2a Author: Josef Eisl Date: Tue Feb 28 17:43:35 2017 +0100 [refactor] move outgoingValues to JumpOp (needed for phis only) commit 69af182ef55df980a8637763224a09ed40809eab Author: Josef Eisl Date: Tue Feb 28 17:50:37 2017 +0100 [fix] remove outgoing from BlockEndOp commit 19a7d71645326678af072f00faed7bcc6175cf1b Author: Josef Eisl Date: Wed Mar 8 09:49:11 2017 +0100 [fix] Remove addOutgoingValues ad forEachOutgoingValue from BlockEndOp commit 681750bf03f5f9bc9c63dd8d6d8d537ae1681ea3 Author: Stefan Anzinger Date: Fri Feb 24 13:26:42 2017 +0000 [feature] Extend inliner to start with a fixed list of invocations. commit 8b69e82739f0f8696b7e61cb2d0d10fa338d68a6 Merge: 6e73e85 d9c4c19 Author: Aleksandar Prokopec Date: Wed Mar 8 04:39:09 2017 -0800 [GR-3160] Always capture bgv files when Graal builds fail. commit d9c4c19dd92c6149f0a1306b48b348e66dee6b39 Author: Aleksandar Prokopec Date: Wed Mar 8 11:15:56 2017 +0100 [feature] Capture bgv files on error. commit 6e73e855621572d2df86d9aa0aae9c7eab158b2d Merge: abf7690 3de73d2 Author: Roland Schatz Date: Wed Mar 8 03:12:02 2017 -0800 [GR-3161] Update truffle import. commit 3de73d21cc5e946f80366c3b330737ea967b94ff Author: Roland Schatz Date: Wed Mar 8 11:30:44 2017 +0100 Update truffle import. commit abf76900647664115b38cc46e4fcfd0a76c9f94e Merge: b3b9eb7 1e2e897 Author: Thomas W?rthinger Date: Wed Mar 8 01:28:39 2017 -0800 [GR-3080] Fix issues with pi rewiring and picking up wrong conditions in the conditional elimination. commit 1e2e89737fa43878186d94ae36cb5aacdb7dc7d9 Author: Thomas W?rthinger Date: Tue Mar 7 22:38:26 2017 +0100 Add documentation on the methods that pick up stamps and change getSafeStamp for the primary value. commit a9d7655bbc623e730291b95dc95b35a1376da90c Author: Thomas W?rthinger Date: Tue Mar 7 21:23:39 2017 +0100 Adding back AndNode second level structural conditional elimination. commit 42fe25cbfa75fd055cdc548bab626e6868d89560 Author: Thomas W?rthinger Date: Tue Mar 7 16:12:29 2017 +0100 Only look up info elements in own pi chain. commit 6f310e81d7733b91394fae785545828637aaf5ed Author: Thomas W?rthinger Date: Mon Mar 6 20:58:02 2017 +0100 Make getSafeStamp more aggressive. commit 875e510f8197bbe15de0f3b0af4707c76007f907 Author: Thomas W?rthinger Date: Mon Mar 6 19:46:50 2017 +0100 Do not rewire pis in new conditional elimination. Use safe stamps on both sides of binary operations. commit b3b9eb76f239a1d01cb2d2329c3c8d964c95f288 Merge: 553d83c b7fb999 Author: Thomas W?rthinger Date: Wed Mar 8 00:34:51 2017 -0800 [GR-3159] Make FixReadsPhase extensible to allow changing scheduling strategy. commit b7fb9991b3be11483e8318c2409eacca80535f92 Author: Thomas W?rthinger Date: Tue Mar 7 21:58:09 2017 +0100 Make FixReadsPhase extensible to allow changing scheduling strategy. commit 553d83c86231497effd130c5bd4e7ba92a04f37f Merge: 950f4d0 d11b37d Author: Aleksandar Prokopec Date: Tue Mar 7 07:31:02 2017 -0800 [GR-3143] Do profiling on @TruffleBoundary and not @TruffleCallBoundary. commit d11b37df3b28ab01cb294614f1b01ab61f170f88 Author: Aleksandar Prokopec Date: Tue Mar 7 12:34:07 2017 +0100 [fix] Instrument TruffleBoundary calls instead of TruffleCallBoundary calls. commit 950f4d030178eef043fb2ee2b19c31612790e774 Merge: 82d2de2 5122025 Author: Josef Eisl Date: Tue Mar 7 05:58:25 2017 -0800 [GR-3039] TraceRA: no longer store global liveness information in LIR. commit 512202516f20a4886f259eac8b24260bd55421ed Author: Josef Eisl Date: Wed Mar 1 15:01:46 2017 +0100 [refactor] TraceRA[GLI]: rename addIncoming/addOutgoing to setIncoming/setOutgoing commit d73a39e114c7c8903ef195e8cc62f37cacea1052 Author: Josef Eisl Date: Wed Mar 1 13:23:31 2017 +0100 [refactor] TraceRA[BU]: inline PhiVisitor commit 4367403e3926209294ee809763f6025b092ef073 Author: Josef Eisl Date: Sat Feb 25 15:16:32 2017 +0100 [feature] TraceRA[LS]: only materialize the locations array if there is an inter-trace edge commit 4e92fde8266aeb98b24ce53d13c4c2a16447a32e Author: Josef Eisl Date: Tue Feb 28 11:26:36 2017 +0100 [fix] TraceRA[LS]: do not insert spill moves at block begin commit 3047f501f54a0a7b6af351a78386496a77726677 Author: Josef Eisl Date: Sat Feb 25 14:09:09 2017 +0100 [feature] TraceRA[LS]: set inter-trace hints only if there is none commit 2beb022cc6fd0f90d338ffb8af299e6874eabaeb Author: Josef Eisl Date: Sat Feb 25 14:02:44 2017 +0100 [fix] TraceRA[LS]: set hints for phis again commit da6bbc89aa9edfa40a815f5923b3de65b699ddda Author: Josef Eisl Date: Fri Feb 24 23:32:38 2017 +0100 [refactor] TraceRA: remove remaining SSI references commit 376c4318fd2d26d4a7c6db015c3d7f99669ba7ac Author: Josef Eisl Date: Fri Feb 24 23:12:56 2017 +0100 [refactor] TraceRA: rename SSIConstructionPhase to GlobalLivenessAnalysisPhase commit 5a80c97c9307808a5f7be393dc87b7904cf52cc1 Author: Josef Eisl Date: Fri Feb 24 22:59:25 2017 +0100 [refactor] TraceRA: inline FastSSIBuilder commit c610210904138d997566ef71250f688a1cbcf429 Author: Josef Eisl Date: Fri Feb 24 22:49:29 2017 +0100 [refactor] TraceRA: remove SSIBuilderBase commit d243ee9785788cd8b2494253fb152496b2cad416 Author: Josef Eisl Date: Fri Feb 24 22:38:54 2017 +0100 [refactor] TraceRA: remove old SSIBuilder commit 5ad64f66d489e65b896fb32407231d9b25640924 Author: Josef Eisl Date: Fri Feb 24 21:43:48 2017 +0100 [refactor] TraceRA: remove SSIUtil commit adbec4d62dbdc279e86a8143f3b8ac5026c36743 Author: Josef Eisl Date: Tue Feb 28 17:50:09 2017 +0100 [fix] TraceRA: no longer use BlockEndOp commit 05b04309d260619f46c01816d6085f7a171344e6 Author: Josef Eisl Date: Fri Feb 24 18:17:02 2017 +0100 [fix] TraceRA: remove obsolete verification code commit ff40a6950ec55e9a122ad09a37feb09a3f0347c2 Author: Josef Eisl Date: Fri Feb 24 18:11:00 2017 +0100 [fix] TraceRA: reenable inter-trace hints commit 490a30c2bdecbbf4008622979c5e1001a4525bcd Author: Josef Eisl Date: Wed Feb 22 13:41:28 2017 +0100 [doc] TraceRA[BU]: fix javadoc commit 84e0436c5373100b74372e225a1c5698b6095cac Author: Josef Eisl Date: Fri Feb 24 17:31:47 2017 +0100 [feature] TraceRA: remove indirection for getBlockIn and getBlockOut in GlobalLivenessInfo commit dcf20531d5594bffd02f6dbbdeba804106c68657 Author: Josef Eisl Date: Fri Feb 24 17:08:23 2017 +0100 [feature] TraceRA: do not initialize live vars with empty lists commit 9f21a8cddef25b4c02d6e710cd90201f85b4cbdc Author: Josef Eisl Date: Fri Feb 24 15:24:29 2017 +0100 [feature] TraceRA: support GlobalLivenessInfo in CFGPrinter commit f9f749f707be597af432e59b8aa701f817b4a4f9 Author: Josef Eisl Date: Fri Feb 24 15:26:34 2017 +0100 [feature] TraceRA: do no longer store global liveness info in LIR commit 91947056239d8f3c0dcf7e2bf91c532e91bbd6a6 Author: Josef Eisl Date: Fri Feb 24 15:27:45 2017 +0100 [fix] Improve assertion message in RedundantMoveElimination commit 82d2de215cbec3582663556d92faaa2f4483fe94 Merge: ef0b935 736c34a Author: Doug Simon Date: Tue Mar 7 03:17:57 2017 -0800 [GR-2865] Print absolute IGV dump paths. commit 1ed06ff10c3b155131a81fbff31abb1aa4fd6779 Author: Tom Rodriguez Date: Tue Feb 28 14:07:34 2017 -0800 isSingleValueUser returns wrong answer for guards commit 9df9b455e1a62101055b5267c2711ce5e8b6e0fc Author: Christian Wimmer Date: Mon Mar 6 15:17:08 2017 -0800 Partial evaluation tests for String.hashCode commit ea7a1ace84c5f20498b3c66642e0c828e2edd385 Author: Christian Wimmer Date: Mon Mar 6 14:01:22 2017 -0800 Inline methods with substitutions during partial evaluation commit 736c34a602a67275222f69beecd7b56ec37f4844 Author: Stefan Anzinger Date: Mon Mar 6 18:16:13 2017 +0000 [GR-2865] Print absolute IGV dump paths. commit ef0b935e346c7708cf871ac0ec07d88fbd30921b Merge: 485dc46 490067c Author: Doug Simon Date: Mon Mar 6 03:31:11 2017 -0800 [GR-3048] Re-enable gating on JDK 9 snapshots. commit 490067c9cab133f2ee472f787cb6dd28df2a9cfb Author: Doug Simon Date: Sun Mar 5 21:01:47 2017 +0100 include app class loader when searching for services in modules that extend Graal commit f9ec74f473ffeba34506d416d85838d2618f21f7 Author: Doug Simon Date: Sun Mar 5 20:31:28 2017 +0100 added utility for concatenating iterables commit 5c6c4078ced8ce32a9d3f95d4458e9bc2aec68e7 Author: Doug Simon Date: Wed Mar 1 16:04:41 2017 +0100 re-enable JDK 9 snapshot related builds commit 155a9821cda95f01d87b47f4b3e4b625f76b95b5 Author: Doug Simon Date: Sun Feb 26 15:03:57 2017 +0100 fall back to app class loader for LayoutFactory provider on JDK 9 commit ebd6ffc196f496fa12d66083756bba7c119c09e8 Author: Doug Simon Date: Sun Feb 26 22:44:57 2017 +0100 adapt to JDK-8174879 commit 485dc462a0776cabadb5c8ac20932383b7465459 Merge: 05e6896 a2d19fb Author: Thomas W?rthinger Date: Fri Mar 3 13:22:15 2017 -0800 [GR-3068] [GR-3074] Try global value numbering after canonicalization. commit a2d19fbe720ad3e8ef35e1d853ee1f9503984472 Author: Gilles Duboscq Date: Fri Mar 3 19:24:16 2017 +0100 [fix] Try global value numbering after canonicalization. commit 05e6896b52143c50817a3ba62f3573711def8f96 Merge: d483351 e1fe589 Author: Thomas W?rthinger Date: Fri Mar 3 10:23:58 2017 -0800 [GR-3107] Compute better stamps for values at merges by meeting the stamps of values at the end of the incoming branches. commit d483351be40e8c047eda7d79efd20021f78ee088 Merge: fd4b395 e7351d8 Author: Lukas Stadler Date: Fri Mar 3 08:50:41 2017 -0800 [GR-3106] Stop processing canonicalized nodes at VirtualObjectNodes. commit e1fe589fdddf6a9927c480fc4ea5701d15ef125d Author: Thomas W?rthinger Date: Fri Mar 3 15:17:13 2017 +0100 Clean up and adding documentation. commit e7351d87bb0f18b5c4a0f0329a68c6b5325d1561 Author: Lukas Stadler Date: Fri Mar 3 16:24:51 2017 +0100 stop processing canonicalized nodes at VirtualObjectNodes commit 1567b6c23626864f89a512cc2c448bc423b12da6 Author: Thomas W?rthinger Date: Thu Mar 2 14:41:25 2017 +0100 Allow fixed read phase to be configurable with boolean whether inputs should be replaced with constants. Make sure canDeoptimize in SafeDivRemNode cannot change after removing PiNodes. commit 47fed0986421f0938f8379ba151529c0f8991039 Author: Thomas W?rthinger Date: Sun Feb 26 00:17:04 2017 +0100 Be more conservative when inserting pis to improve phis in order to prevent update cycles. commit 952ccb4f04c9535a964d3af990b4279527c32044 Author: Thomas W?rthinger Date: Sat Feb 25 23:32:59 2017 +0100 Introduce pi nodes in conditional elimination if phis can obtain a better stamp. commit 337a710bdc077322f05028c7e4dd7678a590581f Author: Thomas W?rthinger Date: Sat Feb 25 21:28:05 2017 +0100 Try to also improve phi stamps at merge points. commit 5b6b1c2815e2d8b4c0216f4c37e609836174bf4f Author: Thomas W?rthinger Date: Sat Feb 25 20:46:03 2017 +0100 Calculate union stamps at merge points. commit 6f22827967f1c7adecee26ba9d67856380f4189e Author: Thomas W?rthinger Date: Sat Feb 25 18:39:07 2017 +0100 Make sure constants with potentially wrong kind are not propagated by word cast. commit 1fca899db56f37100dc6c50bb1cd7b333b147dae Author: Thomas W?rthinger Date: Sat Feb 25 15:48:02 2017 +0100 Put constant input replacement behind flag. commit 11fd15f04cfd2989527a157f9a2f3069b4499d07 Author: Thomas W?rthinger Date: Sat Feb 25 15:35:42 2017 +0100 Add new test case for double compare against 0.0 and fix FloatStamp#asConstant to be more conservative due to 0.0 and -0.0 cases. commit 7315323b298c0b3cb54f4dc47bd538ce9717acd3 Author: Thomas W?rthinger Date: Sat Feb 25 14:14:08 2017 +0100 Replace inputs with constants when stamp is constant in raw conditional elimination. commit 8654aff9d4abc1b9be85c66f783c5ae52d07c2fa Author: Thomas W?rthinger Date: Sat Feb 25 13:50:47 2017 +0100 Handle conditional nodes in raw conditional elimination. commit fd4b395934002dceb55750d72de754faf2b3d271 Merge: eed04a6 d3e2667 Author: Gilles Duboscq Date: Fri Mar 3 05:36:37 2017 -0800 [GR-3074] Add unused `GuardNode` to loop fragment. commit d3e2667d06114c24d514f2ea0313ebbf7e9aa713 Author: Gilles Duboscq Date: Fri Mar 3 14:10:48 2017 +0100 [GR-3074] Add unused `GuardNode` to loop fragement This will help answer questions about induction variables. However it's not correct enough to make loop transformations: it might say some nodes are in the loop even though they can only be scheduled below the loop. commit eed04a6891a97fb9761326bbeacdfeee94c58684 Merge: 75791e5 e271f7e Author: Gilles Duboscq Date: Fri Mar 3 05:09:10 2017 -0800 [GR-3095] Keep workdir if dacapo benchmark fails. commit e271f7e8fd6383bda15ede51f176a9405128cac2 Author: Gilles Duboscq Date: Wed Mar 1 15:34:25 2017 +0100 [GR-3095] Keep workdir if dacapo benchmark fails This makes it easier to investigate the problem commit 75791e585e65ef34aff08aa0e0b2c3aad15a7bea Merge: 6293a8e aeff877 Author: Christian Wimmer Date: Thu Mar 2 16:16:35 2017 -0800 [GR-2032] Improve handling of NodeSourcePosition in Truffle. commit aeff877454c86a9bcba8d710b645cf01a4034510 Author: Christian Wimmer Date: Tue Feb 28 16:53:29 2017 -0800 Do not store Truffle options in static final field commit 51fb6c3aafbedef8aab3920e4b119426a0a4c590 Author: Christian Wimmer Date: Tue Feb 28 13:10:40 2017 -0800 Propagate NodeSourcePosition in more phases commit 3fc9d98f75b979433d29bb463a5102832500795d Author: Christian Wimmer Date: Tue Feb 28 13:09:53 2017 -0800 Allow customization of CompilationResult commit c597eef0d9c16797f748e6a9bf0549f8d31b3fd5 Author: Christian Wimmer Date: Tue Feb 28 13:09:30 2017 -0800 Allow to set debugId commit 6293a8e889d11b0d75156041221cc0e9bb169093 Merge: efe0365 999c554 Author: Christian Haeubl Date: Thu Mar 2 02:28:58 2017 -0800 [GR-3065] Reduce encoded graph footprint. commit efe03652d6408cb9f854f87a539ffa890a00f691 Merge: d142e7c bc959a3 Author: Christian Haeubl Date: Wed Mar 1 04:51:11 2017 -0800 [GR-3054] Performance- and footprint-related optimizations for the graph decoder. commit 999c55476888eba67716d59461bce3898a18d46b Author: Christian Haeubl Date: Tue Feb 28 10:40:58 2017 +0100 Reduce encoded graph footprint. commit d142e7cca297ad1dcacfb78e85ebd07384c606f7 Merge: 13902d9 29500eb Author: Boris Spasojevic Date: Tue Feb 28 22:24:26 2017 -0800 [GR-2148] Don't invalidate targets if a call to interpreter is made and the target is still valid. commit 13902d94221889f2ecfceed1c103fec3189798fd Merge: c616d03 1bb1135 Author: Tom Rodriguez Date: Tue Feb 28 13:59:15 2017 -0800 [GR-2955] Casts for MethodHandle arguments must be properly guarded. commit bc959a328ad099e7e0740049d9ab2251e9e09adc Author: Christian Haeubl Date: Mon Feb 27 13:25:07 2017 +0100 Minor cleanups. commit 094143a255235b914fd58319ab59fa4508dcb61e Author: Christian Haeubl Date: Mon Feb 27 13:06:15 2017 +0100 Place inputs before successors in the encoded graph. commit 15c6ded2fb529f3e5707e15d992ef097776ba84f Author: Christian Haeubl Date: Mon Feb 27 13:03:52 2017 +0100 Replace data structures to increase partial-evaluation performance while decreasing memory footprint. commit 0970ddd20594fb4fd28dec6cfde622ac1ead99ba Author: Christian Haeubl Date: Mon Feb 27 11:34:03 2017 +0100 Minor performance-related changes. commit 1bb1135a7c1c18723005017bb62f41c42145c31d Author: Tom Rodriguez Date: Fri Feb 24 11:27:34 2017 -0800 Casts for MethodHandle arguments must be properly guarded commit c616d03c0fa3315acb3a9db5b2a4fcb7b54baa4f Merge: a36e6c9 b71b9ab Author: Boris Spasojevic Date: Tue Feb 28 04:36:08 2017 -0800 [GR-3040] Per thread map passed to TraceCompilationListener. commit a36e6c94a62bc55f9d5bb8aebbd21862bb257f19 Merge: 2b5e28e 7ba0daa Author: Christian Humer Date: Tue Feb 28 04:14:05 2017 -0800 [GR-3056] More deterministic PerformanceTruffleInliningTest. commit 7ba0daa31a951806ce0c7a186355f9ae4d530409 Author: Boris Spasojevic Date: Tue Feb 28 10:44:20 2017 +0100 Formatting. commit 04c51474a9729ae5fd22c0be1fd17130960a1a75 Author: Boris Spasojevic Date: Tue Feb 28 10:31:09 2017 +0100 Formatting. commit 303e8784d6d338d6c203470593ed2192a219d75f Author: Boris Spasojevic Date: Tue Feb 28 10:12:26 2017 +0100 Run tests 10 times and use minimum duration to eliminate noise. commit 21a99d7cbab78f2d404a8e4e32a65e987de4e510 Author: Tom Rodriguez Date: Mon Feb 27 11:02:04 2017 -0800 Fix non-null stamp machinery for other non-object pointers commit 2b5e28e6ef4d66b1a08a81f6209d6c2d466e5940 Merge: 7b1a5b4 6fd1d1a Author: Christian Wimmer Date: Mon Feb 27 10:52:06 2017 -0800 [GR-3025] Add more hooks for subclasses to influence synthetic start / unwind code. commit 6fd1d1adb449a2d4c342086aa1b8435745454821 Author: Christian Wimmer Date: Thu Feb 23 16:43:59 2017 -0800 Add more hooks for subclasses to influence synthetic start / unwind code commit 7b1a5b4a0c934ab8e142fbbafa548ecc1c81e15c Merge: 519eba1 7d06bef Author: Christian Humer Date: Mon Feb 27 08:46:33 2017 -0800 [GR-2688] Use new copy uninitialized from Truffle. commit 7d06bef1cc636cf13129e20a44d8b198711998f4 Author: Christian Humer Date: Fri Feb 24 16:33:40 2017 +0100 Update Truffle version. commit 94cda073783b45bd9b500339adffbb282984d440 Author: Christian Humer Date: Tue Feb 21 12:25:33 2017 +0100 Simplify clone uninitialized support. commit d21c7ac9c24f568a1500b3aaed30845035ff1885 Author: Christian Humer Date: Tue Feb 21 12:25:15 2017 +0100 Fix typo in TVMCI. commit 7d9581a3f0a82ff7d5e9a9eb03642d1eb12b0dc2 Author: Christian Humer Date: Wed Feb 15 14:15:30 2017 +0100 Use TMVCI to access isCloneUnitializedSupported and cloneUnitialized . commit 596758d9acfd630414eeeea9dbb631a4d095c9fc Author: Christian Humer Date: Wed Feb 8 21:24:34 2017 +0100 Use new copy uninitialized from Truffle. commit b71b9ab7bc1aec3b50570bdde3547c720963c6c2 Author: Boris Spasojevic Date: Mon Feb 27 14:42:15 2017 +0100 Formatting. commit f454ea3e55695b360e2a5973c9cef22c9932d92a Author: Boris Spasojevic Date: Mon Feb 27 14:24:45 2017 +0100 Removing unused member map from TraceCompilationListener. commit 519eba1c375461b221d3cc22fa5a37cd379f079a Merge: 6ea8f00 ba001b4 Author: Lukas Stadler Date: Mon Feb 27 05:24:09 2017 -0800 [GR-2727] Remove faulty precondition check in PEA for virtual object compatibility. commit 6ea8f00f39f5070da80e92fa278725d5cfc0a10a Merge: 7c02f8d 14dc595 Author: Thomas W?rthinger Date: Mon Feb 27 05:21:17 2017 -0800 [GR-3049] Convert Math.*Exact operations to non-exact operations. commit bc9c311f02a6bbd2e446347d1849d07d696bf24f Author: Boris Spasojevic Date: Mon Feb 27 14:17:42 2017 +0100 Avoid problems stemming from a map shared among threads by passing a map to the listener per thread. commit ba001b4c2042ea668dc8b41445242f59b4567329 Author: Lukas Stadler Date: Mon Feb 27 13:55:23 2017 +0100 PEA: remove faulty precondition check for virtual object compatibility commit 14dc5950a9658b2ef5452a5e1b54b722adba5336 Author: Christian Haeubl Date: Wed Feb 22 17:16:52 2017 +0100 Variable name cleanup. commit cf9fb5af86173824049212b4ecdd1821a088fb39 Author: Christian Haeubl Date: Fri Feb 17 09:27:08 2017 +0100 Add exact math canonicalizations/simplifications. commit 342eccb0cf6e604c39f529a9cfe8ab8ef7781ca1 Author: Thomas W?rthinger Date: Sun Nov 6 19:22:01 2016 +0100 Canonicalize exact add to normal add if overflow can never happen based on the operand stamps. commit 7c02f8de6c98d8fdace6b399fe304ff3b9e4289e Merge: 72806d4 d278c21 Author: Boris Spasojevic Date: Sun Feb 26 04:50:02 2017 -0800 [GR-2632] [GR-1624] Truffle inlining performance improvements and tests. commit d278c21a7d95634a864fe26accccbaf9e652dfe1 Merge: 005fec5 72806d4 Author: Boris Spasojevic Date: Sun Feb 26 11:34:37 2017 +0100 Merge branch 'master' into tests/gr-2632 commit 72806d4c6606f754eafc0ec441980b04834e0ae1 Merge: f54ad5d c6f6374 Author: Thomas W?rthinger Date: Sat Feb 25 03:57:24 2017 -0800 [GR-3046] New conditional elimination after fixing reads. commit c6f63742edcf4349de0b273b1c59beeef7356a19 Author: Thomas W?rthinger Date: Sat Feb 25 01:04:15 2017 +0100 Correctly handle default successor when calculating switch node succeeding stamps. commit 5430c2ceba5e65db0530bb342cd2edeb136681db Author: Thomas W?rthinger Date: Fri Feb 24 23:34:26 2017 +0100 Remove redundant conditional elimination phase. commit b42e007bd1083f2e1c817b503e50ff77b34a95a9 Author: Thomas W?rthinger Date: Fri Feb 24 22:08:57 2017 +0100 Fix arithmetic stamp improveWith implementation to take stamp compatibility into account. commit b88a1b62f3af258d230ebee17d7aa37d83454b13 Author: Thomas W?rthinger Date: Fri Feb 24 22:03:31 2017 +0100 Replace nodes with constant stamps with constants in raw conditional elimination. commit 6d8e4f6a468fa70d41d5f545fb815cd0e2e2daf8 Author: Thomas W?rthinger Date: Fri Feb 24 21:42:05 2017 +0100 Add stamp parameters to binary logic node succeeding stamp method. commit 1d0cddeb2f4ff7316b92cd96712892215ae3f3a6 Author: Thomas W?rthinger Date: Fri Feb 24 20:29:08 2017 +0100 Fixed an issue in the getSuperclass snippet that could lead to an unsafe memory read. commit 81ef1cb31197ffdc58ca8c9fd709dcc491a62237 Author: Thomas W?rthinger Date: Fri Feb 24 19:45:45 2017 +0100 Add flag to stress test scheduling reads early. commit f54ad5d72a2d616e27b89dba6f15f676fe9fc48f Merge: 7d1ad12 3efa7a0 Author: Aleksandar Prokopec Date: Fri Feb 24 10:39:27 2017 -0800 [GR-3042] Correctly repair the number of measurements when a benchmark fails. commit 8b286f4b41c65cdaed10f9fee103a9a2c170e865 Author: Thomas W?rthinger Date: Fri Feb 24 17:55:08 2017 +0100 Propagate stamps for unary and binary operations. commit 3efa7a0ddf33b66292929cb89b2cf6efe3ead701 Author: Aleksandar Prokopec Date: Fri Feb 24 19:17:44 2017 +0100 [fix] Fix bug in calculation of total iterations when a benchmark fails. commit c60e255617564e654522e68294bd99ef6a4d00d7 Author: Thomas W?rthinger Date: Fri Feb 24 16:56:26 2017 +0100 Simplify integer switches in raw conditional elimination. commit fd006365d35c4fb539465fc422a1cfea0b58c204 Author: Thomas W?rthinger Date: Fri Feb 24 16:37:02 2017 +0100 Move succeeding stamp calculation to switch nodes as common utility and use from raw conditional elimination. commit 75f1d85125aeda2253fab45c65a38afb04c7b609 Author: Thomas W?rthinger Date: Fri Feb 24 16:28:45 2017 +0100 Fixed an issue in NewConiditionalEliminationPhase that prevented switches from correctly registering stamps. commit a7ca9170642ac1d7900869ddde34b02a9d7ad2ad Author: Thomas W?rthinger Date: Fri Feb 24 16:13:16 2017 +0100 Add new conditional elimination after fixing reads. commit bc4ef64f2c3218855d94dce45134b5768cf0340a Author: Thomas W?rthinger Date: Fri Feb 24 14:25:49 2017 +0100 Change block iteration order of FixReadsPhase to be dominator tree based. commit 7d1ad12cbd24723767be368d78399ca1e1f3a39a Merge: 193b418 730703a Author: Lukas Stadler Date: Fri Feb 24 05:09:57 2017 -0800 [GR-3034] Add message to assertion in ArithmeticStamp. commit 730703a1a2d3e895ad0401ada04bc8925ebe79da Author: Lukas Stadler Date: Fri Feb 24 12:12:36 2017 +0100 add message to assertion in ArithmeticStamp commit 193b41808b1145325d634b12a2965eef4838395e Merge: 5841886 455484f Author: Thomas W?rthinger Date: Fri Feb 24 02:44:50 2017 -0800 [GR-3031] Introduce FixReadsPhase that fixes floating reads and deletes pi nodes from the graph. commit 455484fcf177118659d994c08cf1794356f00d5a Author: Thomas W?rthinger Date: Fri Feb 24 00:02:20 2017 +0100 Correctly take killing begin nodes into account when creating latest schedule. commit 58370156a1463b75617adaee996c8560f25b09dc Author: Thomas W?rthinger Date: Thu Feb 23 22:48:23 2017 +0100 Adjust AheadOfTimeCompilationTest to new fixed reads structure. commit 5313605f2da9cff0c1eeca32f13c3d7e33412a40 Author: Thomas W?rthinger Date: Thu Feb 23 22:33:29 2017 +0100 Introduce FixReadsPhase that fixes floating reads and deletes pi nodes from the graph. commit 58418863f1ed601afa49f188f25e9630c193b49c Merge: 1efc1c5 67de314 Author: Thomas W?rthinger Date: Thu Feb 23 11:13:57 2017 -0800 [GR-3018] No more value anchors required for guards, new guard movement, removed push-through-pi as extra phase. commit 67de314e805596d19e1c9eaf7d3496fa3cf4a150 Author: Thomas W?rthinger Date: Thu Feb 23 19:36:58 2017 +0100 Add checkstyle directive to avoid complaint about jtt test. commit 8192e0c1f1d3ea537032702c88dd37e3b22e5ca1 Author: Thomas W?rthinger Date: Thu Feb 23 18:25:22 2017 +0100 Correctly handle empty instanceof branches and add jtt test for them. commit 0f75d70df46940ef95a68a08e842581273200e82 Author: Thomas W?rthinger Date: Thu Feb 23 15:44:45 2017 +0100 Fix an issue in the new guard movement that was adding new guards too early to the schedule. commit 016fed781f5a4d7e1f904b1df1956ed13524ae09 Author: Thomas W?rthinger Date: Thu Feb 23 04:22:55 2017 +0100 Relax FloatingReadNode constructor assertion. commit 279ac4015ed82a96cd3d46fc4e40cadd0e3375e8 Author: Thomas W?rthinger Date: Thu Feb 23 03:34:19 2017 +0100 Remove conditional elimination load field constant fold test. commit 4359681c9773d323705dfa162ce16822b5d9db05 Author: Thomas W?rthinger Date: Thu Feb 23 03:24:24 2017 +0100 Remove deprecated TODO in ReadNode. commit 3aab8cbb12d3ccf46906e4f1a470200c15509c51 Author: Thomas W?rthinger Date: Thu Feb 23 03:20:11 2017 +0100 Remove constant pointer comparison conditional elimination. commit ee1716fc5b0ed25ef7cf17b458f2718a900cbced Author: Thomas W?rthinger Date: Thu Feb 23 03:19:37 2017 +0100 Make implicit null check conversion more conservative. commit 0d6841907f45c6b9ce7983c2b31becd6a346bd2f Author: Thomas W?rthinger Date: Thu Feb 16 16:32:28 2017 +0100 Introduce new BytecodeExceptionMode.ExplicitOnly mode. Fix an issue in the write barrier verification phase. Fix IfNode canonicalization to take anchors on begin nodes into account. Adjust allowed code size increase of EarlyReadEliminationPhase. Correctly assume receiver type for special invoke. Do not skip over pi in instanceof node. Fix succeeding stamps. commit c250d432035442b078b54a29e33d31df450597e3 Author: Thomas W?rthinger Date: Thu Feb 16 14:39:11 2017 +0100 Add optional stamp parameter to FixedValueAnchorNode. Skip pi for instanceof only if no information is lost. commit 6c70f8efdb5a7a5f21b7f254a5bc9eef79b7d050 Author: Thomas W?rthinger Date: Wed Feb 15 23:43:39 2017 +0100 Make sure more nodes are canonicalized before they are even created. commit da2fc6f2d703eb3678a2faf8a3f277b9bfcb9dc7 Author: Thomas W?rthinger Date: Tue Feb 14 23:22:05 2017 +0100 Clean up phases. commit 7f4c1e5617be30b6d91c73a5bb33a6ed68da0e08 Author: Thomas W?rthinger Date: Mon Feb 13 00:40:13 2017 +0100 Keep guards alive while their anchor is alive. No longer require ValueAnchorNode for this purpose. commit c53b367770069e1a4a5cf4ae981611908061dd7d Author: Thomas W?rthinger Date: Sun Feb 12 23:56:05 2017 +0100 Make sure guards are always marked live and processed before any other node in earliest schedule. commit bb2905a5516cf4ba9771ba110794764d9c290278 Author: Thomas W?rthinger Date: Sun Feb 12 23:52:02 2017 +0100 Reduce number of node iterable types. commit a24455c0c77731ee92e137234a484ed1a1014087 Author: Thomas W?rthinger Date: Sun Feb 12 23:14:07 2017 +0100 Fix and rename GuardEliminationCornerCasesTest. commit 14fd0e523aa5b9ce0de6c7e9aeb9038e2c017f45 Author: Thomas W?rthinger Date: Sun Feb 12 22:29:08 2017 +0100 Refactor and simplify explicit exception code in the bytecode parser. commit 256a17b0ac3b9c8e1a9a70a5aec292239e74709e Author: Thomas W?rthinger Date: Sun Feb 12 21:07:37 2017 +0100 Special case the frequent case of GETFIELD/PUTFIELD after ALOAD*. commit 7049a6e41ce949ae7a8c0301bd71d28454b6883f Author: Thomas W?rthinger Date: Sun Feb 12 20:36:03 2017 +0100 Short cut creation of InstanceOfNode followed by IFEQ or IFNE bytecode. commit 68a7efa84e55775a1329a69a4012c1e67ebcd24d Author: Thomas W?rthinger Date: Sun Feb 12 19:48:56 2017 +0100 Skip pi when creating IsNullNode or InstanceofNode. Add pass to conditional elimination for moving guards present on both sides of an if upwards. Add conditional elimination tests. commit 1efc1c543acd7ed447c59788aeabc223be13e774 Merge: ba38953 7569452 Author: Doug Simon Date: Thu Feb 23 03:49:03 2017 -0800 [GR-2741] Remove OptionValues.GLOBAL. commit 75694522c823d828e12a677a303df3383cc4a09e Author: Doug Simon Date: Thu Feb 23 11:50:12 2017 +0100 consolidated test for whether assertions are enabled commit 8ab1a68fc84e83e742a3a8473c902acce2fe818f Author: Doug Simon Date: Thu Feb 23 11:49:39 2017 +0100 fixed regressions in CooperativePhaseTest commit 5c15ec46fada7f3aadc4f5197d8823e68319bc03 Author: Doug Simon Date: Thu Feb 23 01:17:22 2017 +0100 fixed issues detected by OptionsVerifierTest commit 1648cb56d88cc08c0426420618b5a0f3208a3b37 Author: Doug Simon Date: Thu Feb 23 00:47:42 2017 +0100 reduced reads of some options in hot code commit da08f0d7bc3f905f1891b76ae4191c8421410e6e Author: Doug Simon Date: Thu Feb 23 00:46:04 2017 +0100 restructured CompilationAlarm to reduce reads of CompilationExpirationPeriod option commit 14dcf1e9dde0cb1144e97137498dc28226a15f80 Author: Doug Simon Date: Thu Feb 23 00:41:17 2017 +0100 removed stats on how often OptionKeys are read commit a57a4a1363d86b55e5b916b43225583115d0bb60 Author: Doug Simon Date: Thu Feb 23 00:39:49 2017 +0100 added stats on how often OptionKeys are read commit da27ea11cfd5f5d5154eb5a1b0843a738acc4e64 Author: Doug Simon Date: Thu Feb 23 00:36:08 2017 +0100 removed OptionKey.defaultValue() commit bcdc9a1ecb06a03cd2f69ba8cc0dc4fbb7390308 Author: Doug Simon Date: Tue Feb 21 22:15:01 2017 +0100 Graal Truffle options made be accessed without a GraalTruffleRuntime being initialized commit 2bf40c5c843b79cf206285fc35106704048a1bd0 Author: Doug Simon Date: Thu Feb 23 12:21:49 2017 +0100 made snippet counters non-global - they are passed in by a Graal runtime replaced all uses of GLOBAL with GraalCompilerTest.getInitialOptions() check the invariant that @ConstantParameter, @VarargsParameter and @NonNullParameter are only used in @Snippet annotated methods removed VerifyNodeCostOnAccess option replaced all other uses of OptionValues.GLOBAL with options available from local context removed OptionValues.GLOBAL commit ba38953f2e560decfdc005a95736ae503fa5d3ce Merge: 06f78ea d95792a Author: Doug Simon Date: Thu Feb 23 02:05:52 2017 -0800 [GR-2946] Disable pinning mx to a revision in CI. commit 7fcb7ab158dc6132e339877f72b5264d331b39aa Author: Doug Simon Date: Mon Feb 20 17:06:23 2017 +0100 removed CloneNodesWithUnsafe option commit e0168e01b008b1790892120ac7ba5299c329fabc Author: Doug Simon Date: Wed Feb 15 23:45:46 2017 +0100 use service provider to get option values commit d69cad07503da0228f683c3df6ab93a15e18b833 Author: Doug Simon Date: Mon Feb 13 17:33:55 2017 +0100 removed GLOBAL from StructuredGraph.Builder commit 06f78ea471fae780af57cf762258c9a09501710a Merge: 5082f0e df2865d Author: Stefan Anzinger Date: Wed Feb 22 09:12:50 2017 -0800 [GR-2987] Force compile generated method in LargeConstantSectionTest. commit df2865d05f75381a9c78db1a9af82df404c75c5c Author: Stefan Anzinger Date: Wed Feb 22 14:31:41 2017 +0000 [fix] Force compile generated method in LargeConstantSectionTest commit 5082f0e7ee0919a5e670b9fe7712573e242a03eb Merge: 9898d2f b00edb0 Author: David Leopoldseder Date: Wed Feb 22 06:46:24 2017 -0800 [GR-1994] Inital groundwork for better compile cancelation from Truffle. commit d95792abb3e652cd5c6fff4994cb5740a2d3c840 Author: Doug Simon Date: Wed Feb 22 14:33:40 2017 +0100 disable pinning mx to a revision in CI commit b00edb0d8e6446ae20002e1a0f777cb5d1fc0853 Author: David Leopoldseder Date: Mon Feb 13 16:20:58 2017 +0100 [feature] Graal: Add cooperative compilation cancellation. commit 24d766e8b09bf0752ae49d44ce04e67a6502c6bd Author: Boris Spasojevic Date: Fri Feb 10 12:00:43 2017 +0100 Removed synchronisation and stored the CancelableCompileTask object in a local variable. commit 9e3e8f9b5f245b1e10bdfccb3ad85a12d224d8d5 Author: Boris Spasojevic Date: Fri Feb 10 08:56:39 2017 +0100 Synchronize assignment to compilation task. commit e889d0be9e95d4f6526bf2923fad2cae98541d87 Author: Boris Spasojevic Date: Fri Feb 10 07:51:52 2017 +0100 Javdoc for Cancelable. commit 7db8a020b755f26ad361081d24f67986a36b68ef Author: Boris Spasojevic Date: Fri Feb 10 07:45:26 2017 +0100 If CancelableCompileTask is canceled, soft cancel the future as well. commit 25ffd4f0286a5415927f2738f7fe7b4e28054cc2 Author: Boris Spasojevic Date: Thu Feb 9 18:13:14 2017 +0100 Moved the check to allow Cancelable to be loaded to checkAllowedGraalClasses.` commit 9a3a77f2db7c65303d0894cc733a21a3d769e985 Author: Boris Spasojevic Date: Thu Feb 9 17:59:17 2017 +0100 Renamed task to future. commit 582af20c9437a7ea1af1df00b3194edc4ea7bb5f Author: Boris Spasojevic Date: Thu Feb 9 15:45:36 2017 +0100 Added the CancelableCompileTask to the StructuredGraph. commit 57875f66bfd68641a8a66b85bcacd3151b063af0 Author: Boris Spasojevic Date: Thu Feb 9 15:42:02 2017 +0100 Renamed parameter to avoid hiding a field. commit 9809b131e10544dcc8ddfe3feeddd6ec7fde7ca8 Author: Boris Spasojevic Date: Thu Feb 9 15:32:51 2017 +0100 Adding a Cancelable field to StructuredGraph. commit 6dbd424731ae46fb9ee7b592b1d26c1315cd1ac1 Author: Boris Spasojevic Date: Thu Feb 9 15:04:08 2017 +0100 Extracted the CancelableCompileTask class and Cancelable interface. commit 46e2aadae72b85d6a043766d8968299431d6edb9 Author: Boris Spasojevic Date: Thu Feb 9 12:01:59 2017 +0100 Adding the CancelableCompileTask to wrap Futures resulting from submitting for compilation. This CancelableCompileTask object gets propagated until the partial evaluator builds the StructuredGraph. Tests pass null to all methods that expect a CancelableCompileTask object. commit 9898d2f9af65011a94fd0752a8e551a7cfdf9d9b Merge: 1bac1de 095a6dd Author: Josef Eisl Date: Wed Feb 22 03:06:58 2017 -0800 [GR-1021] TraceRA: Bottom-Up allocator does not need a full local data-flow resolution. commit 29500eb2158a944cb91f5b47b0cc106be92d0eab Author: Boris Spasojevic Date: Wed Feb 22 11:31:44 2017 +0100 Don't invalidate targets if a call to interpreter is made and the target is still valid. commit 1bac1de20fa3c78c824ea99b7caeb894bae3812e Merge: 45899d9 e340234 Author: David Leopoldseder Date: Wed Feb 22 02:22:48 2017 -0800 [GR-2956] Update overlay revision. commit 095a6dd95dd0dfd1348e40710159b04375cb0052 Author: Josef Eisl Date: Wed Feb 22 09:55:37 2017 +0100 [refactor] TraceRA[BU]: directly call resolve method commit dbdbd88cf4537abbd2793bf1c7701f28fb4507dc Author: Josef Eisl Date: Tue Feb 21 18:44:05 2017 +0100 [refactor] TraceRA[BU]: split loop back-edge handling and forward edges commit afc3cb46cea26c070e0908de7958c3e77890cb89 Author: Josef Eisl Date: Tue Feb 21 15:50:40 2017 +0100 [fix] TraceRA[BU]: no need for full local data-flow resolution commit e340234512f883f734c1bd4b3b995a2583780ad9 Author: David Leopoldseder Date: Tue Feb 21 16:44:54 2017 +0100 [dep] Update overlay revision. commit 45899d9b431f14adc86808d51542a6f14dccef91 Merge: 79f28ca 8db556e Author: Doug Simon Date: Wed Feb 22 00:55:03 2017 -0800 [GR-2980] Update overlay version. commit 8db556e58427c47aae2e7db5189f9fa10f67e886 Author: Doug Simon Date: Wed Feb 22 09:34:51 2017 +0100 update overlay version commit 005fec5ecd2ae5c6bc1646fc0213252bede13cc6 Author: Boris Spasojevic Date: Wed Feb 22 07:54:55 2017 +0100 Proper logging of cached decisions. commit 79f28caa4b0d88e4b4ff9026266378bf288f6959 Merge: d7761c8 7ce2c23 Author: Boris Spasojevic Date: Tue Feb 21 22:30:29 2017 -0800 [GR-2687] Remove ThreadLocal field from TraceCompilationListener. commit d7761c87824f64329c95d352b3468e9ed48768c8 Merge: 288aa91 32d37bb Author: Doug Simon Date: Tue Feb 21 08:23:47 2017 -0800 [GR-2976] Update doc regarding building JVMCI JDK8 binaries. commit 288aa9126e5185cb8466d2bbe526a149917da6f7 Merge: 11c757e b2cf4b0 Author: Doug Simon Date: Tue Feb 21 07:47:54 2017 -0800 [GR-2957] Update overlay to enable graalvm downstream testing. commit 11c757e12a1b4e35bfecc0b76defc161c8846cb1 Merge: 388e343 76b5cc3 Author: Aleksandar Prokopec Date: Tue Feb 21 07:36:50 2017 -0800 [GR-2962] Mx benchmark command should produce data points for failing benchmarks. commit 0aa31f3c01ec7363e1d8ff5cf933555355f02e8a Author: Josef Eisl Date: Thu Dec 22 12:32:45 2016 +0100 [fix] TraceRA: no need for delete for rewritten stack to stack moves commit d67931d90867994b4352b2fe500c1e4335e5e7fc Author: Josef Eisl Date: Tue Jan 17 16:12:47 2017 +0100 [fix] TraceRA[BU]: remove redundant code commit 54759284609e9ded8975fb65a47ed9059e1a7085 Author: Boris Spasojevic Date: Tue Feb 21 15:58:11 2017 +0100 Be more restrictive in what you can cache. commit 76b5cc3f344cf58034462654ece0bef18cecc1f7 Author: Aleksandar Prokopec Date: Tue Feb 21 14:33:06 2017 +0100 [feature] Repair partial DaCapo results after a benchmark failure. commit 32d37bb3add0346c7bece7020dd3bd8efb279304 Author: Doug Simon Date: Tue Feb 21 15:48:03 2017 +0100 update doc regarding building JVMCI JDK8 binaries commit 298580fc4cd9e4aa7f21bfc182a0ea48cf87923a Author: Boris Spasojevic Date: Tue Feb 21 10:27:21 2017 +0100 Added deep rejection cache. commit 7ce2c234fbf55a21aecb7805496aac70eb6520bb Author: Boris Spasojevic Date: Tue Feb 21 09:35:30 2017 +0100 Replaced the ThreadLocal with a map. commit dfb7a6c6af95572d2f1eefca184672cdbc8380d1 Merge: 371bafe 388e343 Author: Boris Spasojevic Date: Tue Feb 21 08:00:40 2017 +0100 Merge branch 'master' into bugfix/gr-2687. commit 371bafe23701a25fe4d3d6361a2d0832f2f6cc43 Author: Boris Spasojevic Date: Tue Feb 21 07:43:48 2017 +0100 Use ConcurrentHashMap instead of HashMap with sync. commit 388e343257342090512ec645290c79d6dee0d0af Merge: 6250cc2 192a2e7 Author: Doug Simon Date: Mon Feb 20 11:35:23 2017 -0800 [GR-2961] Remove all static state from the InstrumentPhase. commit b2cf4b067e456f0e371f8708dc276e4d9ff00146 Author: Doug Simon Date: Mon Feb 20 19:40:05 2017 +0100 update overlay to enable graalvm downstream testing commit 192a2e7c12a9c75937cd8f8bcbe1f61b0dd69b46 Author: Aleksandar Prokopec Date: Mon Feb 20 19:15:21 2017 +0100 [refactor] Move instrumentation state to the Truffle runtime. commit 6250cc29ca65d563eba167d189cfb07cc93f9f51 Merge: 065b10e 3f15d3a Author: Lukas Stadler Date: Mon Feb 20 07:26:39 2017 -0800 [GR-2904] Conditional Elimination: Always improve pi stamp if new guard provides additional information. commit 3f15d3a883967db64f79fb942a9ddc4f4310f713 Author: David Leopoldseder Date: Mon Feb 20 14:27:01 2017 +0100 [fix] Conditional Elimination: Always improve pi stamp if new guard provides additional information. commit 065b10eec31f3067027c97ab8c6fe76e3f1c4e75 Merge: 68af508 3cffbc8 Author: Lukas Stadler Date: Fri Feb 17 10:33:49 2017 -0800 [GR-2904] Revert change in NewConditionalEliminationPhase.registerNewStamp. commit 68af50839b4827eed0886c7a6e5129ba7bc682c2 Merge: 59e3dbd eabc45b Author: Tom Rodriguez Date: Fri Feb 17 09:54:19 2017 -0800 [GR-2910] Be more strict about whether StubRoutines are expected to be available in JDK8. commit 59e3dbdfd8a3da7db3b4e201076170ce3cdb4f7f Merge: 40b3c7c 158a5b2 Author: Christian Wimmer Date: Fri Feb 17 09:22:46 2017 -0800 [GR-1633] Use object from frame state for monitor exit. commit 40b3c7c32cac44a608df231262b8b69830ac5ace Merge: 71b0b4b deee6ed Author: Doug Simon Date: Fri Feb 17 08:35:44 2017 -0800 [GR-2916] Adapt to jdk.vm.* being renamed to jdk.internal.vm.*. commit deee6ed9c8c3c24b51531bcb306d252f6701783d Author: Doug Simon Date: Fri Feb 17 16:47:41 2017 +0100 fix tests that define snippets and replacements to work on jdk9 commit 402f092bc0d257e2398e3bf6051d41b5f2d93459 Author: Doug Simon Date: Fri Feb 17 16:44:40 2017 +0100 adapt to jdk.vm.* -> jdk.internal.vm.* renaming commit 6cf5d3711c6028602b95fbef470928499b396825 Author: Doug Simon Date: Fri Feb 17 16:41:12 2017 +0100 install hsdis library into JAVA_HOME if it doesn't already exist commit 0bfab5f11fbd863df3d62715bf3262f66e5315bb Merge: 563b95a 71b0b4b Author: Boris Spasojevic Date: Fri Feb 17 15:03:39 2017 +0100 Merge 'master' into tests/gr-2632 commit 3cffbc8f88f90590ff36c9d13fad7d84afd7cd4b Author: Lukas Stadler Date: Fri Feb 17 13:41:14 2017 +0100 revert change in NewConditionalEliminationPhase.registerNewStamp commit 71b0b4b9fd0fc495f534854ecc4f4697eb69d625 Merge: cb17ff5 cdf16cc Author: Roland Schatz Date: Fri Feb 17 02:57:40 2017 -0800 [GR-2901] Update truffle import to include native interface. commit cdf16cc83370c9bd1b0208b779abda247c74c517 Author: Roland Schatz Date: Thu Feb 16 17:16:09 2017 +0100 Update truffle import to include native interface. commit 563b95a69a6ffb9d8bf041b5fb085c8074773ea1 Merge: 8c9c09f cb17ff5 Author: Boris Spasojevic Date: Fri Feb 17 07:43:53 2017 +0100 Merge branch 'master' into tests/gr-2632 commit cb17ff5739546609f9a49b97808dbf377d4f9a5e Merge: 4501f91 4a02210 Author: Tom Rodriguez Date: Thu Feb 16 21:59:47 2017 -0800 [GR-2911] Ensure constants are on the right in commutative operations. commit 4a0221087ad448049a46ea5dfdde180ce778f99f Author: Tom Rodriguez Date: Sun Feb 12 23:09:01 2017 -0800 Ensure constants are on the right in commutative operations commit eabc45bdd389da076f3135d85bea533be261c7dc Author: Tom Rodriguez Date: Mon Feb 13 10:43:23 2017 -0800 Be more strict about whether StubRoutines are expected to be available in JDK8 commit 158a5b29d8a8dcee211c9573b64cb5388d102879 Author: Christian Wimmer Date: Thu Feb 16 10:43:45 2017 -0800 Use object from frame state for monitor exit commit 4501f91b32f16c55e9998554abb41e0a507fcdaf Merge: 803c0ec f524733 Author: Doug Simon Date: Thu Feb 16 06:39:26 2017 -0800 [GR-2897] Update CI to use jvmci-0.25. commit 803c0ecb5b0d35d1bc1789b4a89b0fd5494da3d6 Merge: cfbeb9a 65a9388 Author: David Leopoldseder Date: Thu Feb 16 06:16:45 2017 -0800 [GR-2898] MethodMetrics: Fix printing setup. commit f524733b51391748cd8c32da4c1134aaaa400880 Author: Doug Simon Date: Thu Feb 16 11:53:39 2017 +0100 update CI to use jvmci-0.25 commit cfbeb9a4b647e47e4c25c57f16dd7233e5ac6d4f Merge: 1691631 a105ce5 Author: David Leopoldseder Date: Thu Feb 16 05:48:01 2017 -0800 [GR-2877] Unsafe PEA Bug. commit 169163127a1e7d78218e324b8ae1d59a80a8af9d Merge: 7d7bc36 5debce5 Author: Aleksandar Prokopec Date: Thu Feb 16 04:02:49 2017 -0800 [GR-2710] Move static state out of InstrumentPhase. commit 5debce5fa8ca9bc18ce05b1c21cdcf47c19cd02d Author: Aleksandar Prokopec Date: Wed Feb 15 16:52:29 2017 +0100 [refactor] Use Truffle-runtime-specific instance of the instrumentation access table. commit 65a9388d2ee50c7d9be0186c4c6f1640efb7318c Author: David Leopoldseder Date: Thu Feb 16 12:17:01 2017 +0100 [fix] MethodMetrics: Fix printing setup. commit 8c9c09f6a6d1d8ca28c72a03f74ec6bbef6aba62 Author: Boris Spasojevic Date: Thu Feb 16 11:47:01 2017 +0100 Removed invisible UTF characters. commit a105ce5337f5d633f161e12ff37bc3105125c57a Author: David Leopoldseder Date: Thu Feb 16 10:44:08 2017 +0100 [fix] PEA: Keep better stamp information during node replacement for all nodes. commit b6425f9bdd2051f846752497048de5f7a45170bf Author: David Leopoldseder Date: Wed Feb 15 13:44:32 2017 +0100 [fix] PEA: Must respect stamps when virtualizing unsafe stores to one slot fields. commit 126441b8a5034d9e88de7f33daaf8546d53a8106 Author: Boris Spasojevic Date: Wed Feb 15 16:04:47 2017 +0100 Suppress Warning. commit dcd7db23164b876db36852ab093d7616e37056db Author: Boris Spasojevic Date: Wed Feb 15 13:16:02 2017 +0100 Non static counting of visited nodes. commit 7d7bc365d1fc52fcbcce4b9d505512c018c09c49 Merge: 7a18b1d 0c9f8cd Author: Tom Rodriguez Date: Tue Feb 14 17:08:38 2017 -0800 [GR-2872] Improve handling of PiNodes in address expressions. commit 0c9f8cd33a61ad92ba20204cc42fcee332a45e41 Author: Tom Rodriguez Date: Mon Feb 13 23:09:38 2017 -0800 Improve handling of PiNodes in address expressions commit 7a18b1d927dd324859f54665f1b6e00e670cd45d Merge: d0d9139 217d3f0 Author: Andreas Woess Date: Tue Feb 14 12:05:12 2017 -0800 [GR-2047] Use hasNeverInlineDirective() in PEGraphDecoder. commit 217d3f05eb042936fb78202925edbcc2009dc670 Author: Andreas Woess Date: Tue Feb 14 19:28:53 2017 +0100 Use hasNeverInlineDirective() instead of canBeInlined() in PEGraphDecoder. commit 1eae4894d43df492d4323d573e908d133e1cef88 Author: Boris Spasojevic Date: Tue Feb 14 17:23:11 2017 +0100 Cache and reuse inline decisions, and don't explore too many call sites. commit f22534c9416104559dbb8c75cb9bd04a6c22f608 Author: Boris Spasojevic Date: Tue Feb 14 17:21:31 2017 +0100 Removed junk and increased timeout for tests to 1 second. commit d0d913941f7cfc82f8da74c8f9d4e7858c910a04 Merge: 638ba23 4f2fe2b Author: Doug Simon Date: Tue Feb 14 01:43:44 2017 -0800 [GR-2842] Clean up some IGV options. commit 4f2fe2bbcdd02f48744dd8c434ec1a8dc4d5a5d1 Author: Doug Simon Date: Mon Feb 13 22:35:05 2017 +0100 deprecated options with ?Ideal? in their name commit 638ba239e23344fbc09056e45b7f764c2daa1298 Merge: bdd86ff 3bb4c58 Author: Thomas W?rthinger Date: Mon Feb 13 16:13:11 2017 -0800 [GR-2633] Fix assertions related to read elimination and PEA. commit 3bb4c5864ed9055810451feca45d97fd7bfd0768 Author: Thomas W?rthinger Date: Mon Feb 13 18:18:54 2017 +0100 Make sure value of proxy nodes is added before proxy node is added in PEA. commit 5598d39cf1bf68fee046709c97806bbf28445fb9 Author: Thomas W?rthinger Date: Mon Feb 13 16:58:02 2017 +0100 Make sure stamp does not get less precise when applying Graph effects from read elimination and PEA. commit 487759f848cb25889eb1482d4d3bd969390c0bb7 Author: Doug Simon Date: Mon Feb 13 20:58:18 2017 +0100 automatically fall back to dumping graphs to files when IGV is unavailable/unreachable commit bdd86ff9211ab26b82b2b295b79b9dee23927899 Merge: dde2f03 dbe905e Author: Doug Simon Date: Mon Feb 13 08:38:17 2017 -0800 [GR-2827] Clean up option changes merge artifacts. commit dbe905e43c3fdb5bbbe5095ab10cc4ca469a69f6 Author: Doug Simon Date: Mon Feb 13 17:18:05 2017 +0100 re-deleted accidentally revived test commit dde2f03e0f7e98fc8539e34a8be4d2789d7ea1ac Merge: 4372ef7 f67aaf1 Author: Doug Simon Date: Mon Feb 13 07:13:56 2017 -0800 [GR-2800] Update truffle import. commit 4372ef7f9a24f8d6c659b8d3c0ff2b5f6ca3946e Merge: 759d763 a1249cb Author: Doug Simon Date: Mon Feb 13 02:26:39 2017 -0800 [GR-2825] Explicitly select compiler configuration for graal-core. commit a1249cb10101f0636bb44708397a35765d24ece9 Author: Doug Simon Date: Mon Feb 13 11:04:37 2017 +0100 explicitly select compiler configuration for graal-core commit 759d7635dedbea1c7efb835e06d8f904615ace33 Merge: 7d5433e c07bcb5 Author: Thomas W?rthinger Date: Sat Feb 11 11:44:02 2017 -0800 [GR-235] Avoid aggressive guard optimization for guards without action. commit c07bcb594a381c3409d4186f145db7a0f21dd4b5 Author: Thomas W?rthinger Date: Sat Feb 11 16:20:36 2017 +0100 New assertions when replacing nodes. Change PiNode canonicalization. Fix conditional elimination to currectly work with unproxified input. commit 582bd97675e05ba51a9c3493c6a8c2206e93d56b Author: Thomas W?rthinger Date: Fri Feb 10 15:00:09 2017 +0100 Avoid aggressive guard optimization for guards without action. commit 7d5433ec8c8594f33f68b46624e0b5f4a434479a Merge: 06c1a89 0a5fd96 Author: Codrut Stancu Date: Fri Feb 10 14:42:59 2017 -0800 [GR-2813] Illegal stream usage fix. commit 06c1a8934e40520649ede585cf51bb10bb110831 Merge: cf6665b 9f29f84 Author: Thomas W?rthinger Date: Fri Feb 10 14:23:47 2017 -0800 [GR-2819] Modify schedule phase to guarantee earliest schedule for floating nodes. Fixes #252. commit cf6665bba490ca2161031bc980553c7c42f59d6a Merge: e16c69a 523bb8d Author: Christian Wimmer Date: Fri Feb 10 14:04:46 2017 -0800 [GR-2816] Update javaCompliance of NFI project to 1.8. commit 9f29f840c11992d70dcbbf99cafb919f58261aa1 Author: Thomas W?rthinger Date: Fri Feb 10 22:05:07 2017 +0100 Document and simplify new version of earliest schedule. commit 76863a35cf96b273469ce131020efc0255cd62d3 Author: Tom Rodriguez Date: Fri Feb 10 11:58:45 2017 -0800 Handle both inputs of BinaryOpLogicNode when folding tests commit fd3a9f440fe836b5ea13731be078eb6b3ae2c22b Author: Thomas W?rthinger Date: Fri Feb 10 18:31:00 2017 +0100 Modify schedule phase to guarantee earliest schedule for floating node - i.e., a floating node is always scheduled before a fixed node if it does not depend on it. commit e1f64bf4cab3abd3e80fb926e4e532646adf055a Author: Thomas W?rthinger Date: Thu Feb 9 23:34:11 2017 +0100 Create test case exercising a corner case of guard movement. commit e16c69a1af7c625f0f1f0b45f24cf335556f6382 Merge: 5b8cebe 3831c99 Author: Christian Wimmer Date: Fri Feb 10 11:48:34 2017 -0800 [GR-2796] More aggressive constant folding of useFrameWithoutBoxing. commit 0a5fd96a31bda0e0c3fcff10ae85facead7dd353 Author: Codrut Stancu Date: Fri Feb 10 11:38:50 2017 -0800 Illegal stream usage fix. commit 523bb8d670cf3eb8ab68b10b58f109d613aae5e4 Author: Christian Wimmer Date: Fri Feb 10 11:28:08 2017 -0800 Update javaCompliance of NFI project to 1.8 commit 3831c9902c57aaedbdde799b09e476bc5f263dd8 Author: Christian Wimmer Date: Fri Feb 10 10:55:56 2017 -0800 More aggressive constant folding of useFrameWithoutBoxing commit 5b8cebee8a66240869de0449d200280ac103b642 Merge: e1a8546 958d3ce Author: Vojin Jovanovic Date: Fri Feb 10 08:31:52 2017 -0800 [GR-2654] Adapt to jvmci-0.24 changes. commit 958d3ce16c527b3d1b9d6338cfa03a32c1518ca1 Author: Vojin Jovanovic Date: Fri Feb 10 16:34:17 2017 +0100 [ci] remove dev from jvmci commit c8db502996f6c2f44202b8ba0dc2dfd98d4b77b5 Author: Boris Spasojevic Date: Fri Feb 10 14:22:58 2017 +0100 Replaced the ThreadLocal with a map. commit 5ee7023084bca89ba3e662d3494d3d2bf997e9f9 Author: Doug Simon Date: Wed Feb 8 15:03:00 2017 +0100 update jvmci-0.24 reference commit 4e2a862c94a9b2e09e660471488251f7365c17ba Author: Doug Simon Date: Mon Feb 6 21:46:45 2017 +0100 broaden class file search for test with @Snippet methods commit f3d7bce9cc2b2ddbf4d5792a141c192c6202fad5 Author: Doug Simon Date: Sat Feb 4 11:12:05 2017 +0100 fixed internal field equivalence test commit 3e2893ba7c5c965748ec404bb32fc260836ef8c1 Author: Doug Simon Date: Fri Feb 3 12:00:04 2017 +0100 update required jvmci version commit 0dc4878d300e5aa6cde422afcf2f2fc6e81dedf1 Author: Doug Simon Date: Fri Feb 3 11:42:41 2017 +0100 adapt to removal of HotSpotVMConfigAccess.getTypeSize commit f67aaf13324e2eae6eaff4691d63f7c19e0fdfc3 Author: Doug Simon Date: Fri Feb 10 11:13:39 2017 +0100 update truffle import commit faa0ae2eaaaba6d4bba88489c615a157f6d4b1ec Author: Boris Spasojevic Date: Fri Feb 10 11:10:15 2017 +0100 Removed some accidental logging. commit 327307f15f8e06cbdf9a3687d633d9cff7bbe31b Author: Boris Spasojevic Date: Fri Feb 10 11:03:05 2017 +0100 Initial commit for Truffle inlining performance tests. commit e1a854653dea3c067320d4e6ea4bfcfc84b4eb72 Merge: 1960e3e 53425c3 Author: David Leopoldseder Date: Fri Feb 10 01:02:27 2017 -0800 [GR-1383] Add weekly G1GC benchmarks. commit 1960e3e2b45acac3766a26844739ab7e12f3638f Merge: 0584bce f8fcb50 Author: Codrut Stancu Date: Thu Feb 9 14:36:23 2017 -0800 [GR-2726] Refactor stream usage. commit f8fcb500150f40d3e52d5954a868f489842a7b64 Author: Codrut Stancu Date: Fri Feb 3 21:41:17 2017 -0800 Refactor stream usage. commit 0584bce96fddfe28fcbdae2f5f13855033a7d982 Merge: 0d9e753 0253981 Author: Christian Wimmer Date: Thu Feb 9 13:23:26 2017 -0800 [GR-2748] Allow forced re-initialization of debug environment. commit 025398147a1937ca08f98d70f63f7a3dffe1d4de Author: Christian Wimmer Date: Thu Feb 9 12:59:51 2017 -0800 Allow forced re-initialization of debug environment commit 0d9e7533a49564f341cb9db085ffea0295f72e33 Merge: 8b26660 c01850f Author: Thomas W?rthinger Date: Thu Feb 9 09:36:39 2017 -0800 [GR-2790] Do not drop guards on reads during read elimination. commit c01850fb004167b5f939d96a562f0bade3629f44 Author: Thomas W?rthinger Date: Thu Feb 9 18:02:04 2017 +0100 Do not drop guards on reads during read elimination. commit 8b26660b523198d1b7485957d845d67577b0ee41 Merge: 9d3fe02 2f73a38 Author: Boris Spasojevic Date: Thu Feb 9 06:17:24 2017 -0800 [GR-2632] Adding a test suite for Truffle Inlining. commit 2f73a38fc28c4bfffdcda7090bb66358e2550736 Merge: 3594f1e 9d3fe02 Author: Boris Spasojevic Date: Thu Feb 9 15:01:46 2017 +0100 Merge branch 'master' into tests/inlining commit 3594f1e44c5ac24f8e5f7e17909402eedf5df858 Author: Boris Spasojevic Date: Wed Feb 8 08:52:10 2017 +0100 Updated the usage of compiler options. commit 4f57bc029daa3b69c5310572155e34169c533422 Merge: 4bb7407 f72bfeb Author: Boris Spasojevic Date: Wed Feb 8 08:51:28 2017 +0100 Merge branch 'master' into tests/inlining commit 9d3fe02ee3f556c8efa7286e7cda6caaebae531a Merge: 757c76a 32e6808 Author: Thomas W?rthinger Date: Thu Feb 9 04:25:40 2017 -0800 [GR-2770] Set new PiStamp if new guarding condition proves more precise information. commit 32e6808116d7dd30c49bd74453f0debd3d2c1249 Author: Thomas W?rthinger Date: Thu Feb 9 11:49:21 2017 +0100 Set new PiStamp if new guarding condition proves more precise information. commit 757c76a40a46593b22f5baf0a211395d11739224 Merge: 4b3ccb1 675e604 Author: Thomas W?rthinger Date: Wed Feb 8 15:21:12 2017 -0800 [GR-2754] Use pi instead of guard for read access. commit 675e60477ab0fb3bfecfd9ba9c487706c638dd1d Author: Thomas W?rthinger Date: Wed Feb 8 17:25:13 2017 +0100 Fix dynamic object allocation snippet. commit 62442f5f48c70b41e5f4cd0e8a3d3ec7556d50ee Author: Thomas W?rthinger Date: Tue Feb 7 20:48:14 2017 +0100 Fix array store exception stub and Unsafe.allocateInstance intrinsification. commit 2ee3be6a47df9fc6a17a3ec27ab7da2d361afa04 Author: Thomas W?rthinger Date: Tue Feb 7 20:19:43 2017 +0100 Fix constant pool intrinsification to check for non-null. commit d0f3293c827dbe9eaba883c12f8de18e61e76658 Author: Thomas W?rthinger Date: Tue Feb 7 16:59:51 2017 +0100 Fix createClassCastException to have NonNullParameter. commit b82278f3bcc9d8de8b00329c5235a46e6a06aef7 Author: Thomas W?rthinger Date: Tue Feb 7 15:15:59 2017 +0100 Add assertion that FloatingReadNode inputs have non-null stamp. Fix violators. commit d077baf51b8bce9d4a3a9909b0764b8915a45a53 Author: Thomas W?rthinger Date: Tue Feb 7 13:35:31 2017 +0100 Reduce number of guarded nodes. commit bfb911da3ecae08d637d4e763bcd03ea2f49dd77 Author: Thomas W?rthinger Date: Mon Feb 6 16:48:53 2017 +0100 Modify WriteBarrierVerificationTest. commit 469811fcdf0ca13207a3b8559318cc4b3eaa147c Author: Thomas W?rthinger Date: Mon Feb 6 04:43:22 2017 +0100 Fix lowering phase to clear guards only when actually exiting the block. commit 03b3023b17e635e3e646d27832855523b1d24a61 Author: Thomas W?rthinger Date: Mon Feb 6 02:08:50 2017 +0100 Use guard instead of pi for array bounds check. Fix klass pointer null check. Simplify read node and clean up after graph building. commit 867d9def5d68605dd83600d47126e8ccfd65347a Author: Thomas W?rthinger Date: Sun Feb 5 20:18:42 2017 +0100 Change lowering for memory access nodes to use PiNode instead of directly guarding the access. commit 4b3ccb1cec1b4aec6474076656a6b82a8db4b619 Merge: 5e2d27f d47aa7d Author: Tom Rodriguez Date: Wed Feb 8 10:31:50 2017 -0800 [GR-2672] Properly handle 0 length strings in String.indexOf intrinsic. commit d47aa7dce4954cdab51223c2755c2167cc06dff7 Author: Tom Rodriguez Date: Mon Feb 6 17:40:31 2017 -0800 Properly handle 0 length strings in String.indexOf intrinsic commit 5e2d27fe53b72f958a536997ac8c9856dc66320a Merge: 33635af fb14123 Author: Doug Simon Date: Wed Feb 8 09:08:43 2017 -0800 [GR-2752] Fixed NPE in DebugValuesPrinter.printDebugValues. commit fb141236280a0241b6a4b5a620824cbed908e94c Author: Doug Simon Date: Wed Feb 8 17:44:16 2017 +0100 fixed NPE in DebugValuesPrinter.printDebugValues commit 33635afdc032789ee58b05ef9fd650cd8beddb79 Merge: 2a56c90 c943010 Author: Doug Simon Date: Wed Feb 8 08:06:06 2017 -0800 [GR-2736] Cleanup after GR-1979. commit 2a56c903712f457cc13b09da7f6b914ff7d0d2f4 Merge: 4176c41 20f5567 Author: David Leopoldseder Date: Wed Feb 8 05:22:20 2017 -0800 [GR-2724] OsrLockTest: Remove printing to stdout. commit 4176c41ea0785ad5875b96f0bbee78279733567f Merge: f72bfeb 9498f74 Author: Josef Eisl Date: Wed Feb 8 05:12:08 2017 -0800 [GR-2729] Reduce the number of TraceRA task in the CI. commit c94301037e76531fc191b1558ba289c4de4056b1 Author: Doug Simon Date: Wed Feb 8 11:58:53 2017 +0100 fixed comment to reflect code changes commit 0545cdc7a321b6d503624807f3c8550d9ebca20a Author: Doug Simon Date: Wed Feb 8 11:58:32 2017 +0100 removed accidentally added code commit 9498f7488df6a54852983c10bd864a9f8c26bdcc Author: Josef Eisl Date: Wed Feb 8 11:55:06 2017 +0100 [ci] update overlay revision commit 20f5567072248e713cba51914a0dcc8a85118bac Author: David Leopoldseder Date: Wed Feb 8 11:50:21 2017 +0100 [fix] OsrLockTest: Remove printing to stdout. commit b6971b5982996b6fda257ed56195313e4c672656 Author: Josef Eisl Date: Wed Feb 8 10:11:39 2017 +0100 [ci]: reduce TraceRA tasks commit f72bfeb6def8c41ae6426661f6d4aaead4cb94df Merge: 60b1818 9fa7538 Author: Doug Simon Date: Tue Feb 7 14:15:45 2017 -0800 [GR-1979] Replace static options with context based options. commit 9fa753893414aa6c86763d6606bea6a053ec5217 Author: Doug Simon Date: Tue Feb 7 20:49:52 2017 +0100 removed accidentally added code commit 9298ad801978e49c0c35b1e4def2e5bda3aba5c1 Merge: e3b1959 41d3dc9 Author: Doug Simon Date: Tue Feb 7 17:11:33 2017 +0100 Merge. commit e3b19596f886be9bad689f36014c20cac678b9f5 Author: Doug Simon Date: Tue Feb 7 17:03:32 2017 +0100 merge fixes commit 76d0454add6142ee6dd3e757f9ed4fd2ed00f081 Merge: 33b07c3 60b1818 Author: Doug Simon Date: Tue Feb 7 16:03:29 2017 +0100 Merge. commit 33b07c33d295af8b51164704abf277d459881445 Merge: 1aebf17 39ed5ab Author: Doug Simon Date: Tue Feb 7 15:31:12 2017 +0100 Merge branch 'master' into topic/non-static-options commit 4bb7407cd19cf8ba837e59542cde2196bb8f9ed1 Author: Boris Spasojevic Date: Tue Feb 7 14:33:50 2017 +0100 Removed a comment to please Checkstyle. commit aec325ead8a20227be53cc3dc62ba74c2d96fcc1 Author: Boris Spasojevic Date: Tue Feb 7 14:14:33 2017 +0100 Separate methods to build targets and decisions. commit edfa581f5a914238d99ec938dea1107aac6e54a5 Author: Boris Spasojevic Date: Tue Feb 7 11:08:59 2017 +0100 Fixed Checkstyle issues. commit 7c1889bac1f4c83e82e2b0781a7d22ee6491f3e5 Author: Boris Spasojevic Date: Tue Feb 7 10:50:28 2017 +0100 Removed unused builtins. commit 00cbe331a1d81bb1a4cb5ea31d9e0dfeb420149e Author: Boris Spasojevic Date: Tue Feb 7 10:18:20 2017 +0100 Deleted the SL based inline tests. commit cb02c7a667a90db66f1bb3ce70afd399ba4c5686 Author: Boris Spasojevic Date: Tue Feb 7 10:07:37 2017 +0100 Added a few more test inspired by the SL inlining tests. commit c9453c55f3360e3a867138d92fdd66b9bf93a530 Merge: 18f66f4 60b1818 Author: Boris Spasojevic Date: Tue Feb 7 09:11:06 2017 +0100 Merge branch 'master' into tests/inlining commit 60b1818bee085341e23569c44deb5454fbb509b4 Merge: 83a58e7 38e128b Author: Gilles Duboscq Date: Mon Feb 6 14:44:18 2017 -0800 [GR-2691] Refactored magic number to a named constant. commit 38e128bf668b5d2735222b8bdbcf6d6fee291714 Author: Doug Simon Date: Mon Feb 6 23:22:08 2017 +0100 fixed capitalization commit 83a58e75857f1ba333263df812d8815bd2161d16 Merge: dac323d e70fb16 Author: Tom Rodriguez Date: Mon Feb 6 10:55:34 2017 -0800 [GR-2528] Stubs hang onto intermediate compiler state forever. commit dac323d62da179252b59f0c6c6c7aafcacc07585 Merge: 39ed5ab b71e065 Author: Tom Rodriguez Date: Mon Feb 6 10:16:40 2017 -0800 [GR-2663] Fix types in GraalHotSpotVMConfigNode.inlineContiguousAllocationSupported(). commit 8e935d8dd9f5a301e89bcef488b9fdcfe56a60ce Author: Doug Simon Date: Mon Feb 6 19:03:44 2017 +0100 refactored magic number to a named constant commit 39ed5ab670fbb3493ed2907b3289934b5577d5c0 Merge: 34dc32c ca6b5e8 Author: Aleksandar Prokopec Date: Mon Feb 6 07:31:51 2017 -0800 [GR-2686] Rename extra.value.name field to metric.object. commit ca6b5e81d90cba476e833023ca152bf88d64ebad Author: Aleksandar Prokopec Date: Mon Feb 6 15:33:44 2017 +0100 Rename extra.value.name field to metric.object. commit 53425c33a3055d9614cbdfa7ca27ec08fe6cf180 Author: David Leopoldseder Date: Mon Feb 6 15:35:09 2017 +0100 [ci] Add g1gc weekly benchmarks. commit c26fc090a79f6a3a76ac64f55974ee0695877b79 Author: David Leopoldseder Date: Mon Feb 6 15:34:36 2017 +0100 [feature] Add g1gc jvm config. commit 18f66f4e9c59ec178ff87067ef0ca5f819267631 Author: Boris Spasojevic Date: Mon Feb 6 15:28:25 2017 +0100 Fixed the SuppressWarnings annotation. commit ff50da30517c0cbbf49ffc314790a8b54541886a Author: Boris Spasojevic Date: Mon Feb 6 15:10:07 2017 +0100 Merged target instructions in to one class and one map. commit d810f2c84799a0755b7d42512330f6212e0b65d7 Author: Boris Spasojevic Date: Mon Feb 6 14:50:42 2017 +0100 Making fields final and a bit of cleanup. commit f1a94d4559d4fcf4fb58887c99467831ee62b0c4 Author: Boris Spasojevic Date: Mon Feb 6 14:42:09 2017 +0100 Fixed a typo. commit 5943dec3a829d40906b2847327f746934201258a Author: Boris Spasojevic Date: Mon Feb 6 14:30:34 2017 +0100 Fixed the expected/actual order in assertions. commit 56d90e3085e76b47658a716687d6f391978fbab8 Author: Boris Spasojevic Date: Mon Feb 6 14:26:57 2017 +0100 Added a test to confirm inlining if size * number of call sites is close to limit. commit 0635641eec81a202a3b7a2bc0a36b745e4e51529 Author: Boris Spasojevic Date: Mon Feb 6 14:06:46 2017 +0100 Moving each period to end of line. commit b5b4ff98c59c57b8730063c5485995ec4598c7cd Author: Boris Spasojevic Date: Mon Feb 6 14:01:15 2017 +0100 Suppress unused warning for override scope. commit 34dc32ce647bb0732428854049b0c12cdbeab35a Merge: 82e9913 f725f5a Author: David Leopoldseder Date: Mon Feb 6 04:57:14 2017 -0800 [GR-1501] Support OSR with locks on the stack. commit f725f5a1a7d924098e9cca5ff485c0778a8a903e Author: David Leopoldseder Date: Wed Jan 25 17:59:28 2017 +0100 [feature] osr: support OSR with locks Make sure all monitor operations on the same MonitorIdNode are locking the same object when building the OSR entry point Remove liveness changes and handle final barrier in OSR compiled constructor commit 74ff69efc0b036d2cba0f43dec058dcbdc099eda Author: Boris Spasojevic Date: Mon Feb 6 10:45:20 2017 +0100 Making TruffleInliningTest an abstract class. commit 92741bedf4274fdab7f15da4ffcf9851bd377974 Author: Boris Spasojevic Date: Mon Feb 6 10:17:50 2017 +0100 Extract policy as a field to enable testing different policies. commit d73100773fb990fe5bcb4918c58def77a2cb75b2 Author: Boris Spasojevic Date: Mon Feb 6 10:05:39 2017 +0100 Upgraded and assert. commit 0e38631f3579a65137fcfd0420bc5c895a1ba694 Author: Boris Spasojevic Date: Mon Feb 6 10:03:00 2017 +0100 Test the ability to disable inlining by flag. commit 82e9913cb31b93399402e1b0d312a49b0556194d Merge: 7b3aa72 9dd0eed Author: Doug Simon Date: Sun Feb 5 12:57:57 2017 -0800 [GR-2664] Avoid unnecessary annotation reification. commit 9dd0eedd13f13086a92704b107176f78bd7e7b76 Author: Doug Simon Date: Sat Feb 4 17:17:55 2017 +0100 moved annotation retrieval to exception path commit b71e0657e760003a1aa101abdb9a167c5e963c79 Author: Igor Veresov Date: Mon Jan 30 23:23:13 2017 -0800 Fix types in GraalHotSpotVMConfigNode.inlineContiguousAllocationSupported() commit 7b3aa723c71960b1ae7d80463a5186826842b3f5 Merge: 6b198a9 51dc4ac Author: Roland Schatz Date: Fri Feb 3 07:16:06 2017 -0800 [GR-2657] Add support for Truffle on Aarch64 (#248). commit 0fdb0ff41c202c8a6934f1cf656174db79e88c62 Author: Boris Spasojevic Date: Fri Feb 3 15:02:09 2017 +0100 Cut dependency to GraalTest. commit c19ca721cf4552a7d2a5f391accbf25f2538a343 Author: Boris Spasojevic Date: Fri Feb 3 14:52:09 2017 +0100 Cut dependency to GraalTest. commit 4c631fd7ee149ee8b5c27d6e85d88e055a5c1c46 Author: Boris Spasojevic Date: Fri Feb 3 14:46:34 2017 +0100 Maximize Truffle Compilation Threshold. commit 29538a8bca0a0f5b18c2e8e37b7249947186087c Author: Boris Spasojevic Date: Fri Feb 3 14:35:45 2017 +0100 Separate tests (BasicTruffleInliningTest) and base class (TruffleInliningTest). commit 5bb09aa1700d5889915eaba5e30efb4439ba52fd Author: Boris Spasojevic Date: Fri Feb 3 12:32:03 2017 +0100 Removed duplicate test. commit 48f99a4f16d37b477ae9729638efc83735d5c56e Author: Boris Spasojevic Date: Fri Feb 3 12:30:24 2017 +0100 Use custom formatting when using the builder. commit 098998155b53eb27b8c0ab5b1d086b0423e0eb63 Author: Boris Spasojevic Date: Fri Feb 3 12:06:37 2017 +0100 Changed the default execution count for call sites to 0. commit 4b71ef3a9353a45e06ec77252494b13ccebc7a40 Author: Boris Spasojevic Date: Fri Feb 3 11:53:29 2017 +0100 Fixed some Checkstyle errors. commit 21d740d27beddb73d5fc3ed10c4576ddef4a00d5 Author: Boris Spasojevic Date: Fri Feb 3 11:47:54 2017 +0100 Removed a few unused variables. commit a158cfd4831521d7c6afa4d1886df6e330f7cbcd Author: Boris Spasojevic Date: Fri Feb 3 11:42:59 2017 +0100 Removed redundant cast. commit 6fc3f0d8db3098621039fb2b84378659f78b60c5 Author: Boris Spasojevic Date: Fri Feb 3 11:05:39 2017 +0100 Adding support to control frequency. commit 6b198a931fb9c3fd7509eb74f88b1416fec7b802 Merge: fd19afc ea4de57 Author: Boris Spasojevic Date: Fri Feb 3 00:04:02 2017 -0800 [GR-1552] Use 2 compilation threads by default, regardless of processor core count. commit 04888ab5cdaee42f249c2fdd32e9bb45db25eaf6 Author: Boris Spasojevic Date: Fri Feb 3 08:53:54 2017 +0100 Removed build method with argument. commit fd19afc72851dc6b6f1a13bf0dd77b3f9c4ccc9f Merge: eeead7d eb19756 Author: Kevin Menard Date: Thu Feb 2 16:51:19 2017 -0800 [GR-2168] Update to latest ci-overlays. commit 51dc4ac45f881a628ae34425c591a528846d4b07 Merge: 39124f8 eeead7d Author: theRealAph Date: Thu Feb 2 18:20:46 2017 +0000 Merge remote-tracking branch 'master/master' into aarch64-truffle commit 39124f80d8de1aebd3c6b7770c637dd93dbd4159 Author: theRealAph Date: Thu Feb 2 17:14:12 2017 +0000 Chackstyle fix. commit e545e0dbbbfaf582ad33b4c12be9f25084f2e0a2 Author: theRealAph Date: Thu Feb 2 17:03:31 2017 +0000 Minor fix for dependencies and tests. commit eb19756edb9f74ff4387e09295149cab799b459a Author: Kevin Menard Date: Thu Feb 2 11:17:13 2017 -0500 Update to latest ci-overlays. commit aa7b73b04ba966011dd726575cedf5d39cbf9607 Author: Boris Spasojevic Date: Thu Feb 2 15:28:28 2017 +0100 Removed duplicate test and renamed another. commit 410784bd3f6f98d0a4d1f8e4f43b5252f1e7e92c Author: Boris Spasojevic Date: Thu Feb 2 15:23:30 2017 +0100 A few more tests testing deep and wide call graphs. commit 75d71f23c4f23d944241ba30468304918667cc1c Author: Boris Spasojevic Date: Thu Feb 2 14:27:11 2017 +0100 Removed unused frame slot. commit 09d0aac816d6aad1c02b85a9159a7a4cf9fa47fd Author: Boris Spasojevic Date: Thu Feb 2 14:15:03 2017 +0100 Added testIndirectRecursiveInline. commit eeead7d36a3d3559a222c27ad25b8b6ca18f8ac3 Merge: 9f6de20 fb7bc56 Author: Doug Simon Date: Thu Feb 2 05:14:11 2017 -0800 [GR-2599] Revert back to jvmci-0.23 for CI. commit c8e9a6f65cb6693433d78ae87bcb085c66d3016c Author: Boris Spasojevic Date: Thu Feb 2 13:56:03 2017 +0100 Introducing the TruffleInliningTest class. commit fb7bc5684af90202ab79eaf33408cdfe71e0f9d6 Author: Doug Simon Date: Wed Feb 1 12:38:41 2017 +0100 revert back to jvmci-0.23 for CI commit 9f6de2086654be3138018ae143cc79a0346b7566 Merge: 948bdb1 97bb567 Author: David Leopoldseder Date: Thu Feb 2 04:46:42 2017 -0800 [GR-2394] Schedule used to verify a valid graph should not mutate the graph. commit 948bdb10689ecd5be7cc517a65ab7ef865ec6fd4 Merge: f60c2af fb1573c Author: David Leopoldseder Date: Thu Feb 2 03:54:11 2017 -0800 [GR-2626] Block iteration compilation alarm bailout must be permanent. commit 97bb567bfa8029a8f5504a76e614e25df423dc0c Author: David Leopoldseder Date: Wed Jan 18 17:08:31 2017 +0100 [fix] GraphOrder: Schedule used to verify a valid graph should not mutate the graph commit fb1573c4fc181986f4b2e348c2c6c39d83083401 Author: David Leopoldseder Date: Thu Feb 2 11:27:09 2017 +0100 [fix] Block iteration compilation alarm bailout must be permanent. commit f60c2af4731aade509b1c199932849d53548b40e Merge: 2510c49 4698fcd Author: Aleksandar Prokopec Date: Wed Feb 1 13:54:41 2017 -0800 [GR-2574] Add instructions for branch instrumentation. commit 4698fcdc57eafc8def6083dfbad023c9c5483b17 Author: Aleksandar Prokopec Date: Mon Jan 30 15:29:06 2017 +0100 Add instructions for branch instrumentation. commit 2510c49fe053e6325b615b0b73d3ba9cba691f46 Merge: 0890343 3a13148 Author: Tom Rodriguez Date: Wed Feb 1 12:22:35 2017 -0800 [GR-2620] Add high level optimization for String.indexOf. commit 3a13148bb48b5ab6990fe91bf74fa7cc4d0f7433 Author: Tom Rodriguez Date: Thu Dec 8 12:04:23 2016 -0800 Add high level optimization for String.indexOf on constant strings commit 0890343e16fc45b4e8251418958f02eb1e0fbf05 Merge: ce50d02 cf66d51 Author: David Leopoldseder Date: Wed Feb 1 09:53:17 2017 -0800 [GR-2600] ControlFlowGraph visitDominatorTreeDeferLoopExits deferres exits for invalid loops. commit 9aeaff2b6cb62eb9b4d01c46ee366bb584e1b2a1 Merge: 99e2248 ce50d02 Author: theRealAph Date: Wed Feb 1 16:57:08 2017 +0000 Merge remote-tracking branch 'master/master' into aarch64-truffle Conflicts: graal/org.graalvm.compiler.hotspot.aarch64/src/org/graalvm/compiler/hotspot/aarch64/AArch64HotSpotLIRGenerator.java graal/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfig.java commit cf66d51211bb2f06fc2064b833790c886b87e804 Author: David Leopoldseder Date: Wed Feb 1 13:06:50 2017 +0100 [fix] ControlFlowGraph: Fix deferred exit traversal deferred loop computation. commit ce50d0238efb67a2fdf939678fcec2f1b76a7e9c Merge: 0c9091a 2f5ed18 Author: David Leopoldseder Date: Wed Feb 1 06:38:23 2017 -0800 [GR-2605] Revert DynamicPiNode removal. commit 2f5ed181f99699394a7721509f358cdbb0218c01 Author: David Leopoldseder Date: Wed Feb 1 14:10:22 2017 +0100 Revert "Remove DynamicPiNode (PiNode if beneficial should be inserted on demand)". commit ea4de5792dd9ca66020677210e410cb99ca7dd32 Author: Boris Spasojevic Date: Wed Feb 1 09:13:46 2017 +0100 Use 2 compilation threads if there are 4 or more available processors. commit 0c9091a5a9a5bd3d4862c7df630244e19cab368d Merge: a2c9f4d 8be794e Author: Thomas W?rthinger Date: Tue Jan 31 21:20:01 2017 -0800 [GR-2247] Remove unused parts of the code (salver, instrumentation, Graal deopt stubs). commit 8be794e2d7740d070cea792e7aeeebd2cd1ff3fc Author: Thomas W?rthinger Date: Tue Jan 31 17:21:43 2017 +0100 Remove DynamicPiNode (PiNode if beneficial should be inserted on demand). Reduce usage patterns of GuardedValueNode to only without stamp and use PiNode in other cases. commit d7ea41c93a15946e0f096b591b530eb5667698e4 Author: Thomas W?rthinger Date: Mon Jan 30 20:22:24 2017 +0100 Remove unused Graal stubs for deoptimization. Remove unused node intrinsics. commit 5028bbd39074609dfcfd4943a67b829677f369bb Author: Thomas W?rthinger Date: Mon Jan 30 19:28:06 2017 +0100 Move SnippetAnchorNode from HotSpot-specific project to general project. commit 7d281217e61a9b7621179c102fa2b5cd500d2568 Author: Thomas W?rthinger Date: Mon Jan 30 19:09:49 2017 +0100 Temporarily remove InstrumentationPhase and associated functionality. commit 35f0fcef088c323a89fa982c97074147b024510a Author: Thomas W?rthinger Date: Mon Jan 30 18:31:35 2017 +0100 General clean up. Remove unused salver project. commit a2c9f4dabd2f1dca73727f7fa08e56d45221c8a2 Merge: 36d1fc5 18ee8e8 Author: Doug Simon Date: Tue Jan 31 10:35:16 2017 -0800 [GR-2595] Fix mx version used in CI. commit 36d1fc586c855109c41d4acc4d3d917871f454c4 Merge: ffe25b8 5fa9408 Author: Doug Simon Date: Tue Jan 31 09:03:05 2017 -0800 [GR-2593] Cache methods for lookup in ClassfileBytecodeProvider. commit ffe25b87d5f22d9b2f7d351f5775f8d9e75e20e4 Merge: 9d75033 4b68038 Author: Doug Simon Date: Tue Jan 31 08:52:04 2017 -0800 [GR-2440] Removed reference to Method::_jfr_towrite. commit 18ee8e8933c0e19bff72ef07e69e2a952bac1ca4 Author: Doug Simon Date: Tue Jan 31 16:50:25 2017 +0100 fix mx used in CI to a specific commit commit 4b68038ceb3c72d2f8670a76e6a69f3643900da1 Author: Doug Simon Date: Fri Jan 20 14:50:41 2017 +0100 Method::_jfr_towrite was removed by JDK-8171960 commit 5fa9408ff9afe596e45a08ead284a5c2f317059a Author: Doug Simon Date: Tue Jan 31 16:22:44 2017 +0100 cache methods for lookup in ClassfileBytecodeProvider commit 9d7503325342a3570eda5cbff22c469ea344d991 Merge: 62cbd52 8011c0f Author: Doug Simon Date: Tue Jan 31 06:06:48 2017 -0800 [GR-2545] Add fast path for inlining trivial accessors. commit 62cbd528cca628739a1a77c13d1fe453a076c248 Merge: 6224cc5 f058bfa Author: Boris Spasojevic Date: Tue Jan 31 05:14:43 2017 -0800 [GR-2444] Rewrite SL Truffle tests to not require background compilation. commit 8011c0fa6da86c4b5114b9da619606f2c188fa0c Author: Doug Simon Date: Thu Jan 26 23:39:11 2017 +0100 added BytecodeParser fast path for inlining accessors commit 6224cc5e406256ea92fe619f16353946eeb77bcc Merge: ca81523 4f86a9c Author: Tom Rodriguez Date: Mon Jan 30 17:29:58 2017 -0800 [GR-2580] Add missing ValueNode import for generated match rules. commit e70fb16e4b76a6add5b6984fd3cf5cf3d45844a4 Author: Tom Rodriguez Date: Tue Jan 24 11:43:22 2017 -0800 Stubs hang onto intermediate compiler state forever commit 4f86a9c2d968880f1286af7f3d58ca3383531df0 Author: Tom Rodriguez Date: Mon Jan 30 12:12:47 2017 -0800 Add missing ValueNode import for generated match rules commit 41d3dc962874742e925bb051f508f74e26ce07ce Author: Christian Wimmer Date: Mon Jan 30 16:55:46 2017 -0800 Do not use GLOBAL option values in debug and graph dumping code commit f058bfa2c51da5b88733a44d1b8f4522e343525c Author: Boris Spasojevic Date: Mon Jan 30 17:07:50 2017 +0100 Removed waitForCompilation from callUntilOptimized. commit 7e7c9f24a7ec6047b4f52531305bbad519b5efc5 Author: Boris Spasojevic Date: Mon Jan 30 17:06:59 2017 +0100 Removed waitForOptimization from SL files. commit c05c3c291185142b2c4fbd56933ec71f3d9e0700 Author: Boris Spasojevic Date: Mon Jan 30 17:06:21 2017 +0100 SLTruffleGraalTestSuite extends TestWithSynchronousCompiling. commit ca815239f932dd28e0b8a32fbef8844b2489bc00 Merge: d94ca56 cb2c76d Author: Aleksandar Prokopec Date: Mon Jan 30 06:20:58 2017 -0800 [GR-2135] Add profiling for @TruffleBoundary-annotated methods. commit cb2c76d57e4b213ade8451f3c2a99ce9a82dff5f Author: Aleksandar Prokopec Date: Mon Jan 30 14:50:09 2017 +0100 Prune and merge instrumentation options, and make instrumentation filters all by default. commit d94ca5682413eb7fa1292247e2bd2a7e33ac282a Merge: 177563e 0051351 Author: Christian Humer Date: Mon Jan 30 05:00:31 2017 -0800 [GR-1766] Remove callnode frame argument. commit 005135140de06ff99f8479f1789073108e865c62 Author: Christian Humer Date: Mon Jan 30 13:39:56 2017 +0100 Remove frame argument from call node methods. commit 99e224842e06e6f0db8f94c9e501d44b8a0fb119 Author: theRealAph Date: Mon Jan 30 12:03:37 2017 +0000 Fix dependencies for AArch64 truffle. commit 177563e51882bc2a676c6c9d298325af1baf6f4b Merge: 5e0015e 6c10037 Author: Felix Kaser Date: Mon Jan 30 02:58:23 2017 -0800 [GR-2572] Remove git for darwin build. commit 7d888f7189ab3eca4cedb6ff9666db8284185d7b Author: Boris Spasojevic Date: Mon Jan 30 11:14:58 2017 +0100 Use 2 compilation threads by default, regardless of processor core count. commit 5e0015ea87fe92c527da02967e2b00d4947ca624 Merge: f137082 3b1641f Author: Boris Spasojevic Date: Sat Jan 28 02:31:58 2017 -0800 [GR-2444] Rewrite Truffle tests to not require background compilation. commit 06f33371f40f1ccb18d8da07c09a45586f32438e Author: Christian Wimmer Date: Fri Jan 27 19:40:07 2017 -0800 Remove StableOptionKey, OptionKey.setValue commit 4e886be3270d02e280c028c8c580735e58f3a69f Author: Christian Wimmer Date: Fri Jan 27 18:15:43 2017 -0800 Truffle must constant fold flag UseFrameWithoutBoxing during partial evaluation commit e0475d007d5521b1667b594c7a90a6bd62727bef Author: Christian Wimmer Date: Fri Jan 27 18:15:00 2017 -0800 Use standard way to check for disallowed null argument commit f137082e685e3827cb498cf16aa04d43092042d0 Merge: dab40f5 7b6931d Author: Thomas W?rthinger Date: Fri Jan 27 09:18:20 2017 -0800 [GR-2506] Fix calculation of join IntegerStamp and refactor to always take mask and bounds into account in constructor. commit dab40f5772d149ac62f39cc476a52cdd58ab5366 Merge: 2ba2150 f35e9a4 Author: Gilles Duboscq Date: Fri Jan 27 08:35:41 2017 -0800 [GR-2362] Skip unnecessary Pi nodes when lowering array length. commit 7b6931dd5e3f90f44139350ad3777b722245f4dc Author: Thomas W?rthinger Date: Fri Jan 27 16:12:05 2017 +0100 Add new test cases to IntegerStampTest and fix a missing canonicalization when creating IntegerStamps. commit f35e9a49e15ef59b9c3795f4918ba724a093e069 Author: Gilles Duboscq Date: Thu Jan 26 18:18:29 2017 +0100 [GR-2362] Skip unnecessary Pis when lowering array length This should help ensure that various array length of the same array common-out. Added `GraphUtil.skipPiWhileNonNull` Also factor out the code used to create array length reads in `DefaultJavaLoweringProvider`. commit 3b1641f4c29590b93468c078a83d00b5ef973413 Author: Boris Spasojevic Date: Fri Jan 27 15:41:26 2017 +0100 Removed useless throws. commit 0abbe7d37d5ab491b5bcf98e1b53a5daf6f9a5f8 Author: Boris Spasojevic Date: Fri Jan 27 15:29:18 2017 +0100 Fixed imports. commit 6c100373d5561b010ca12c07827a1c68c1f8761f Author: Felix Kaser Date: Fri Jan 27 06:01:55 2017 -0800 remove git for darwin build it's installed system wide already commit 35018a6416c5ac6051698f504199daf714e6c222 Author: Boris Spasojevic Date: Fri Jan 27 14:58:36 2017 +0100 Fixed testOuterInvalidationTriggersOSR to work when not using background compilation. commit 4f4f34e1c229937d97e9c45e42f45a351d5f0e96 Author: Aleksandar Prokopec Date: Thu Jan 26 16:20:18 2017 +0100 Add InstrumentTruffleBoundariesPhase. commit f97dd08f6ac358f0c1bf01a4a20ce34a715d3776 Author: Aleksandar Prokopec Date: Thu Jan 26 13:08:06 2017 +0100 Refactor common instrumentation functionality from branch instrumentation. commit 2ba21503435fcf890335470acae67f6f2574f0cd Merge: 6415e5c c6a1f0d Author: Codrut Stancu Date: Thu Jan 26 14:01:32 2017 -0800 [GR-2544] Unsafe access of edges should use wrappers in Edges class. commit 6415e5ccc3c8a506290578f35e2d9d34ae4f98fc Merge: 70e8985 eae4712 Author: Doug Simon Date: Thu Jan 26 13:14:21 2017 -0800 [GR-2536] Reduce dynamic usage of ResolvedJavaField.getName. commit bf6ade69b8c4662e878e3b806e60837d0398fcbd Author: Thomas W?rthinger Date: Thu Jan 26 22:07:11 2017 +0100 Fix calculation of join IntegerStamp and refactor to always take mask and bounds into account in constructor. commit eae4712e581390fa1aa8691863ac6fc4d816c495 Author: Doug Simon Date: Thu Jan 26 15:13:44 2017 +0100 replace HashMap with EconomicMap commit 45e7fbd23f64689e2d1c0f633569cd296d4cb12f Author: Doug Simon Date: Thu Jan 26 14:30:12 2017 +0100 reduce dynamic usage of ResolvedJavaField.getName() commit c6a1f0d2ea034a8360badab8adb6ea8f54fdca0b Author: Codrut Stancu Date: Thu Jan 26 12:27:45 2017 -0800 [GR-2544] Unsafe access of edges should use wrappers in Edges.class. commit 70e8985f0857ae07b8b50d8434079ec3a89daaf6 Merge: c1ed814 6d21cb6 Author: Gilles Duboscq Date: Thu Jan 26 08:58:21 2017 -0800 [feature] Allows to force `ANY_LOCATION` for a truffle unsafe store plugin commit 6d21cb6c60d67b484f3de98d5a5628571bae2430 Author: Gilles Duboscq Date: Fri Jan 13 16:34:55 2017 +0100 [GR-2482] Allow to force `ANY_LOCATION` for a truffle unsafe store plugin Adds a flag to `UnsafeStore` to avoid canonicalizations that would lose `ANY_LOCATION`. Truffle unsafe store plugins can have a constant that represents this forced `ANY_LOCATION`. `null` still represents the non-forced `ANY_LOCATION`. commit c1ed814e1b0b943c10a4ed9a63a79282cd35b8a9 Merge: 4b8d2f7 e0339c2 Author: Stefan Anzinger Date: Thu Jan 26 06:16:14 2017 -0800 [GR-2534] Use AllocatableValue in SPARC LIR instructions whenever possible. commit 4b8d2f7ce7f624cc31a5d444254587f7e2bbd70a Merge: b416c07 eef3827 Author: Lukas Stadler Date: Thu Jan 26 06:12:49 2017 -0800 [GR-2516] Cleanups/documentation in PEA commit eef3827797d57377140057967235d1f26d4cfc66 Author: Lukas Stadler Date: Wed Jan 25 17:36:03 2017 +0100 cleanups in PEA commit 5e7e6510c50e09356e0f22dc38210fc6083dce63 Author: Lukas Stadler Date: Wed Jan 25 17:03:45 2017 +0100 use ?null? to represent default values in VirtualObjectState commit e0339c22bbceffefb58ed4ad1a61a3e7edbe1244 Author: Stefan Anzinger Date: Thu Jan 26 13:30:34 2017 +0100 [fix] Use AllocatableValue in SPARC LIR instructions whenever possible. commit b416c07f65d0db9548d77d85b8372b6ad0c7055c Merge: 4159a17 1229053 Author: Doug Simon Date: Thu Jan 26 02:23:21 2017 -0800 [GR-1984] CompilerWatchdog: Swallow VMErrors. commit 4159a1753cec341089ca35fc1d56dab63d3c355e Merge: 23acd8b c3b322e Author: Doug Simon Date: Thu Jan 26 02:21:37 2017 -0800 [GR-2407] CompilationWatchDog: Disable per default. commit 1229053e031a7eff870a9900e31464a79e3d09aa Author: David Leopoldseder Date: Thu Jan 26 10:11:36 2017 +0100 [fix] CompilerWatchdog: Swallow VMErrors. commit c3b322e8efd55aae8b278dcc9a5589403abee148 Author: David Leopoldseder Date: Thu Jan 26 10:07:10 2017 +0100 [feature] Bootstrap: Enabled CompilationWatchDog explicitly during bootstrap. commit 589e1b50f588e2449ce57812fab0fb45832dddbe Author: David Leopoldseder Date: Mon Jan 23 17:32:26 2017 +0100 [fix] CompilationWatchDog: Disable per default & increase trace interval and number of fatal stack traces. commit 23acd8b195a3ccbe8ef080c4690567afc9b1fe8c Merge: 99dcac4 f2f59b1 Author: Doug Simon Date: Wed Jan 25 14:45:13 2017 -0800 [GR-2462] [GitHub PR #245] Avoid zero-shift with AOT. commit 99dcac46adc51bbeb3a7ea00c47a1e66bad5e7a2 Merge: 71af6bb 1f884e8 Author: Doug Simon Date: Wed Jan 25 14:36:06 2017 -0800 [GR-2526] Fixed regression in VerifyUsageWithEquals and fixed the problems it subsequently found. commit f2f59b1485dd4968a33f3965a71cb448ea65e6f4 Author: Igor Veresov Date: Sat Jan 21 11:31:06 2017 +0300 Avoid zero-shift compressed oops for AOT code commit 1f884e81922e5d9c5c690c3866a50b47c6b65b05 Author: Doug Simon Date: Wed Jan 25 22:27:16 2017 +0100 fixed regression in VerifyUsageWithEquals and fixed the problems it subsequently found commit cc9b186f28d1d282cafbb40829218c3446560f13 Author: theRealAph Date: Wed Jan 25 18:13:50 2017 +0000 Enable AArch64 native calls for truffle. commit 71af6bb834013817a6057f8517fd27923d7f7b07 Merge: 69dcf99 3b5290f Author: Vojin Jovanovic Date: Wed Jan 25 09:24:14 2017 -0800 [GR-2498] collect SVM compilation logs commit 3b5290f72bf1105a116339ca15e65311eda2ea11 Author: Vojin Jovanovic Date: Wed Jan 25 17:51:47 2017 +0100 [doc] explain python for intellij commit 96d56e5fc2d3b50f738edb26f5bc528d89aa8731 Author: Vojin Jovanovic Date: Wed Jan 25 16:40:56 2017 +0100 [ci] collect SVM compilation logs commit 2f82f2eea5d72707f2201217f1e4c01b3037dc32 Author: Boris Spasojevic Date: Wed Jan 25 16:47:56 2017 +0100 OptimizedOSRLoopNodeTest extends TestWithSynchronousCompiling. Ignore testOSRFrameSlotChangeDuringOSR. commit 69dcf99730429dd235bbc33bab3e88c58f1b6fd4 Merge: 92e1f5e 0a7b8f2 Author: Aleksandar Prokopec Date: Wed Jan 25 06:06:38 2017 -0800 [GR-2477] Further improve stability of DaCapo measurements commit dc80887f4c8e6f1b04e42e2eb51acfbc8f4bb088 Author: Boris Spasojevic Date: Wed Jan 25 14:13:48 2017 +0100 Updated OptimizedCallTargetTest to sublcass TestWithSynchronousCompiling. commit 1a39724354ee9add1aa72bb03808b5e05c71b4a5 Author: Boris Spasojevic Date: Wed Jan 25 14:11:46 2017 +0100 Updated ExplodeLoopBlockDuplicationTest to subclass TestWithSynchronousCompiling. commit de63397eb82668c07484a4a3f10fb0974d4968ca Author: Boris Spasojevic Date: Wed Jan 25 14:09:51 2017 +0100 Introducing TestWithSynchronousCompiling. This class serves as a superclass for Truffle tests that require synchronous (i.e. no background) compilation. commit 0a7b8f218b6d242ad7beea18151a78d6b1be2cee Author: Aleksandar Prokopec Date: Wed Jan 25 11:03:53 2017 +0100 Increase the number of warmup runs used for averaging the result. commit 92e1f5ed5ac6014ece7cad342e5bc59e4c6f162c Merge: 576bb34 0d8833b Author: Thomas W?rthinger Date: Tue Jan 24 12:22:46 2017 -0800 [GR-2502] PEA object identity improvement at merges and loops. commit 0d8833be1692c5a7165c575b92d5ea73cc2d15fc Author: Lukas Stadler Date: Tue Aug 9 15:02:02 2016 +0200 more extensive checks to determine whether object identity survives in PartialEscapeClosure commit e7bdeb7cce18027db9dfbc11b556a8b117a379e8 Author: Lukas Stadler Date: Tue Aug 9 15:00:39 2016 +0200 remove broken (and unused) Arrays.asList call in PartialEscapeClosure commit 576bb34268ff4215c7cfa529407182411478ea7f Merge: 3654604 9cef763 Author: Aleksandar Prokopec Date: Tue Jan 24 06:04:35 2017 -0800 [GR-2477] Check if benchmark warmup iterations exist before computing the average. commit 9cef76322fa9aa069586c0aae65afbb349b1d4b2 Author: Aleksandar Prokopec Date: Tue Jan 24 14:38:47 2017 +0100 Check if benchmark warmup iterations exist before computing the average. commit 3654604bf93ddb89f5e6aab5662b6c4455218362 Merge: faae2c4 0a78eaf Author: Aleksandar Prokopec Date: Tue Jan 24 04:38:48 2017 -0800 [GR-2483] Average out latest results on Renaissance. commit 0a78eaf353fc54a2f1cd1218852c44d7069f3b97 Merge: c690a0e faae2c4 Author: Aleksandar Prokopec Date: Tue Jan 24 12:26:02 2017 +0100 Merge branch 'master' into fix/average-renaissance commit c690a0eabdc319077390b1e888b9d62fc76ecf66 Author: Aleksandar Prokopec Date: Mon Jan 23 18:41:21 2017 +0100 Average out latest results on Renaissance. commit 1aebf17977a721d5dfa61eaa3d7a44a8ad478b5d Merge: 264d4f8 faae2c4 Author: Doug Simon Date: Tue Jan 24 12:03:18 2017 +0100 Merge branch 'master' into topic/non-static-options commit 264d4f83dcd8eaee2d0363ac932000efd43c94c5 Merge: 579d29b a9e612b Author: Doug Simon Date: Tue Jan 24 00:57:03 2017 +0100 Merge branch 'master' into topic/non-static-options commit faae2c42c7a6f8c3c05a3f1433f5fab3635676f1 Merge: 7ea0a75 20ad9a0 Author: Thomas W?rthinger Date: Mon Jan 23 19:37:37 2017 -0800 [GR-2450] Do not assume array length input to AbstractNewArrayNode to be >= 0. commit 7ea0a7503d37ba8b375a4c2c83f239b0e82282ef Merge: 072a47f 1c8c5ec Author: Thomas W?rthinger Date: Mon Jan 23 19:24:18 2017 -0800 [GR-2456] New approach to dominator conditional elimination phase. commit 1c8c5ec8da477187f372cfb514702651b667b5be Author: Thomas W?rthinger Date: Tue Jan 24 02:19:16 2017 +0100 Reduce usage of java.util.ArrayList for dominator tree data structure and in new conditional elimination phase. commit ad2fd50cb65f386708562eb242516374de25a1ca Author: Thomas W?rthinger Date: Mon Jan 23 00:17:48 2017 +0100 New approach to dominator conditional elimination phase. commit 072a47fc236b94d39f1945b00ef8ee54a82c4255 Merge: a9e612b c5cd1e2 Author: Tom Rodriguez Date: Mon Jan 23 16:20:46 2017 -0800 [GR-2431] Use NodeEventScope for tracking inlining graph changes commit c5cd1e22d66a2aa6d248b1004874a9f14fa0a4e5 Author: Tom Rodriguez Date: Mon Dec 12 12:26:48 2016 -0800 Simplify tracking of nodes changed by inlining commit 579d29bf2400b007584f1e298f4a3771e5b412d2 Author: Doug Simon Date: Mon Jan 23 13:54:03 2017 +0100 EconomicMap may never called hashCode on keys commit a9e612b96b51d54583cc70e5c0fb38a863e6f42b Merge: 83c63c7 8747569 Author: Thomas W?rthinger Date: Mon Jan 23 14:12:12 2017 -0800 [GR-2465] Improve EconomicMap, EconomicSet, and Pair utilities. commit 8747569833141f7a9248d72cc3a545301dcb179a Author: Thomas W?rthinger Date: Mon Jan 23 22:31:20 2017 +0100 Add more utilities to EconomicMap interface. Change ImmutableMapCursor to UnmodifiableMapCursor. commit 28321d92d862502e5cbe60ae2e2878449d8d0e12 Author: Thomas W?rthinger Date: Mon Jan 23 21:59:19 2017 +0100 Improve EconomicMap, EconomicSet, and Pair utilities. commit 83c63c73d68bebb7ea76efdc2632747234b5aff5 Merge: 21a2b8a 2067d07 Author: Aleksandar Prokopec Date: Mon Jan 23 09:14:34 2017 -0800 [GR-2477] Compute average of last 20% of iterations reported in DaCapo. commit 2067d07a18ad6ffa4b4f628476c6d0440de39aeb Author: Aleksandar Prokopec Date: Mon Jan 23 15:08:03 2017 +0100 [fix] Compute average of last 20% of iterations reported in DaCapo. commit 1d3d549be6e359eb6867b394245d24c134f2bdea Merge: 6e8740d 21a2b8a Author: Doug Simon Date: Mon Jan 23 13:23:23 2017 +0100 Merge branch 'master' into topic/non-static-options commit 6e8740de791654f665d89e2bece0a568bfa53038 Author: Doug Simon Date: Mon Jan 23 12:43:42 2017 +0100 merge fixes commit 21a2b8ad61d7443fac74efa87c37b924680eb9ce Merge: 88ecb57 de50f19 Author: Doug Simon Date: Mon Jan 23 03:57:34 2017 -0800 [GR-2455] Update truffle import. commit 6dcd2f08eb1573f4c9851348eccd5c27fe5f23c6 Merge: bbb7350 d993012 Author: Doug Simon Date: Sun Jan 22 22:57:10 2017 +0100 Merge branch 'master' into topic/non-static-options commit 88ecb5768b0bdb38e6585ecb54cf3b71f2979de7 Merge: 3f09e66 1c826a3 Author: Thomas W?rthinger Date: Sun Jan 22 06:15:19 2017 -0800 [GR-2126] Correctly rewire inputs to pi nodes when adopting guards in the dominator-based conditional elimination phase. commit 1c826a3511737a5a1f22c70dc2b4f65cfb5a2c42 Author: Thomas W?rthinger Date: Fri Jan 20 22:30:15 2017 +0100 Correctly rewire inputs to pi nodes when adopting guards in the dominator-based conditional elimination phase. commit 3f09e66b48db47cc01f3a158a61f0c40ada65c4a Merge: d993012 b5ea038 Author: Doug Simon Date: Sun Jan 22 05:34:31 2017 -0800 [GR-2454] Remove invalid DebugTimer tests. commit de50f19e96e2f4d6d3de2cf79e3239a228866e73 Author: Doug Simon Date: Sun Jan 22 14:17:40 2017 +0100 update truffle import commit b5ea038574159a30428bc74d7368a8157bf5ed7e Author: Doug Simon Date: Sun Jan 22 14:11:17 2017 +0100 remove invalid DebugTimer tests commit d99301247326a30f6ddf1d7658ef5e6edc873c1b Merge: d0a2800 c2d5d89 Author: Thomas W?rthinger Date: Sat Jan 21 05:24:12 2017 -0800 [GR-2390] Avoid need for placeholder variable as they represent mutable static state. commit c2d5d89bc9e504c2a4a23f0116bfb351e5ea16c9 Author: Thomas W?rthinger Date: Sat Jan 21 03:32:19 2017 +0100 Change the API of PhiNode#singleValue and PhiNode#singleBackValue to avoid need for marker return value. commit bc68ed9567c049ba524fa53fd887ea0478953f98 Author: Thomas W?rthinger Date: Sat Jan 21 03:11:29 2017 +0100 Remove placeholder node Graph#PLACE_HOLDER. commit 20ad9a04b287a78809ee76d96ebf622ce2157869 Author: Thomas W?rthinger Date: Sat Jan 21 02:46:05 2017 +0100 Do not assume array length input to AbstractNewArrayNode to be >= 0. commit d0a280059b27efaa8df6007a183031f64ecae476 Merge: c295f9b 127a87a Author: Thomas W?rthinger Date: Fri Jan 20 13:36:31 2017 -0800 [GR-2387] Allow metaspace constants to inline into compares. commit c295f9b498e327be9ffa418936d03d7306c4fc67 Merge: 30498e7 b48ced9 Author: Thomas W?rthinger Date: Fri Jan 20 12:23:21 2017 -0800 [GR-2427] Remove Graph#addWithoutUniqueWithInputs. commit 30498e77c6d83df754f074e0bedb8c5502b7b2b5 Merge: dca1fda a9410de Author: Christian Humer Date: Fri Jan 20 09:12:08 2017 -0800 Increase timeout for compilations in OptimizedCallTargetTest. commit a9410decba0097ad45d289ad032ca6df14c39b12 Author: Christian Humer Date: Fri Jan 20 17:49:40 2017 +0100 Increase timeout for compilations in OptimizedCallTargetTest. commit dca1fda18c83e349b9f2a40b231e17ec342a8af0 Merge: f30952d 6879b7f Author: Thomas W?rthinger Date: Fri Jan 20 08:35:01 2017 -0800 [GR-2395] Refactor RecursiveInliningTest. commit b48ced9826d90b0448e7f14bf1f23b4a94a9862b Author: Thomas W?rthinger Date: Fri Jan 20 16:48:54 2017 +0100 Remove Graph#addWithoutUniqueWithInputs. Support using Graph#addOrUniqueWithInputs in the PEA phase. commit f30952da4636c2f7b5a486b4701025e9abe6cf53 Merge: 31fc791 9e33cfa Author: Boris Spasojevic Date: Fri Jan 20 03:29:34 2017 -0800 [GR-756] Disable Truffle inlining when flag TruffleFunctionInlining is set to false. commit 31fc79174d022e2fe5dbccea06655f25bffdcd76 Merge: 922be31 8b5e20c Author: David Leopoldseder Date: Fri Jan 20 02:55:12 2017 -0800 [GR-2414] Refactor tests to avoid redundant unsafe initialization. commit 127a87a46c5d12398ca43ea99ef759b5869ae09f Author: Tom Rodriguez Date: Tue Jan 17 23:41:51 2017 -0800 allow metaspace constants to inline into compares commit 922be31e046e3295a6005ca52cbd96746824a40f Merge: 8bea4ab 48b43c5 Author: Thomas W?rthinger Date: Thu Jan 19 14:35:44 2017 -0800 [GR-2426] IterativeConditionalEliminationPhase should clean up dead nodes commit 48b43c57ec791f9aa9a0ef16036829610a9b95a0 Author: Tom Rodriguez Date: Thu Jan 19 11:25:40 2017 -0800 IterativeConditionalEliminationPhase should clean up dead nodes commit 9e33cfad08f160e456c93f8e8a06a7a2f458478e Author: Boris Spasojevic Date: Thu Jan 19 16:57:54 2017 +0100 Fixed formatting. commit 81882082b441c24309520f0b634fcdd5cce4ea30 Author: Boris Spasojevic Date: Thu Jan 19 16:49:55 2017 +0100 Use of API method to create an empty list. commit 2da27e1ef76dee63040bab24007d512d5d0a535b Author: Boris Spasojevic Date: Thu Jan 19 16:48:48 2017 +0100 Positive condition check rather then early return. commit 22b9733d01bc0f5d933a670e58e41f7e6d446104 Author: Boris Spasojevic Date: Thu Jan 19 16:36:41 2017 +0100 Removed a '.*' import. commit 940ebf974e60ecdcdb63f5bb1d0faf41c75678e2 Author: Boris Spasojevic Date: Thu Jan 19 15:46:11 2017 +0100 Print an informative message if Truffle inlining is disabled but tracing of inlining is enabled. commit 1fb981b9f69bca2371530e7ce8685686d71bf59c Author: Boris Spasojevic Date: Thu Jan 19 15:28:09 2017 +0100 Stop all Truffle inlining if the TruffleFunctionInlining flag is set to false. commit 8b5e20cd527803604795a9fe1da1ee8b7adb875f Author: David Leopoldseder Date: Thu Jan 19 09:47:09 2017 +0100 Refactor tests to avoid redundant unsafe initialization. commit 6879b7f1491f7ac57d73d0599ab99d2006ba6d1c Author: David Leopoldseder Date: Wed Jan 18 18:38:46 2017 +0100 Refactor RecursiveInliningTest. commit 8bea4abff573a7ca2caad2af0772e3f8b3514b49 Merge: ef543d2 14562ce Author: Doug Simon Date: Wed Jan 18 04:33:53 2017 -0800 [GR-2397] Use the MethodParameters class file attribute if available. commit ef543d26790d28cb9b066fae8b822e9318f5fadb Merge: c965099 8d3d0a8 Author: Doug Simon Date: Wed Jan 18 02:51:48 2017 -0800 [GR-2393] workaround eclipse import static bug commit 14562ce2813f370f2c0c7ece59bc47634c47d2fe Author: Doug Simon Date: Tue Nov 8 23:45:22 2016 +0100 use the MethodParameters class file attribute if available commit 8d3d0a8a9ce9a235034e8c628e6984ee1e8b480e Author: Tom Rodriguez Date: Tue Jan 17 23:23:17 2017 -0800 workaround eclipse import static bug commit c96509900a464dfa109aa7aa0599bbc4ffb0aef7 Merge: 016786d c187ca0 Author: Codrut Stancu Date: Tue Jan 17 23:03:43 2017 -0800 [GR-2389] Remove illegals calls to @Fold annotated method. commit 016786dad08fa3b7d91ccb67fbc1c0afca64ac94 Merge: 1b6f5e4 54cc31d Author: Thomas W?rthinger Date: Tue Jan 17 17:36:55 2017 -0800 [GR-2370] Added canonicalization for cascades of short circuit or nodes. commit 54cc31dfc1c42c2f204d13f495b5f19466b7880c Author: Thomas W?rthinger Date: Wed Jan 18 02:06:17 2017 +0100 Perform systematic testing of short circuit or canonicalizations. Add boolean comparison canonicalization to normalize to the 0 test case. commit 62f1c01fc8e91dc4fb01292ab129376353dfa9c6 Author: Thomas W?rthinger Date: Tue Jan 17 14:49:50 2017 +0100 Added clarifying comments. commit 2cbd0dc112134e23a56addd2b0cbc2dfb8c3c1fe Author: Thomas W?rthinger Date: Tue Jan 17 05:26:16 2017 +0100 Added canonicalization for cascades of short circuit or nodes. commit c187ca03a5ced0d6e0cad5466e5fce8aae76e0d9 Author: Doug Simon Date: Tue Jan 17 21:46:30 2017 +0100 removed calls to @Fold methods that have a parameter annotated with @InjectedParameter commit 1b6f5e428afa21a0e34362b038dff6ed0d9d4ba2 Merge: d190b7d f14f28a Author: Thomas W?rthinger Date: Tue Jan 17 12:18:38 2017 -0800 [GR-2114] More efficient LoopsData.outerFirst and innerFirst. commit f14f28abb2e2aa59f50358eb9d3a689c9fbcf121 Merge: 29636d5 d190b7d Author: Thomas W?rthinger Date: Tue Jan 17 17:29:10 2017 +0100 Merge branch 'master' into topic/loopsdata commit d190b7d5e2bd2c5a5f4d5a99f6b4efce64be033a Merge: 13384d4 5fa3448 Author: Doug Simon Date: Tue Jan 17 06:59:16 2017 -0800 [GR-2368] Disable JDK 9 gates until JDK-8172850 is fixed. commit 13384d48e1f6b4fc9e4c9f07668e9706fc90eb40 Merge: 9a8d652 d2545b7 Author: Andreas Woess Date: Tue Jan 17 06:54:59 2017 -0800 [GR-2367] Truffle: ensure Assumption.isValid() is inlined for indirect calls commit 5fa3448f7b6d053096e4cad353e938f68329418e Author: Doug Simon Date: Tue Jan 17 15:03:18 2017 +0100 update overlay version commit 006841ed93b00065d9386e108a5038c1f9fc68be Author: Doug Simon Date: Tue Jan 17 14:34:03 2017 +0100 disable JDK 9 gates until JDK-8172850 is resolved commit 29636d5f1bc133436e0033e86c517d276326e766 Merge: 25b471c 8f1ad40 Author: Thomas W?rthinger Date: Tue Jan 17 14:21:06 2017 +0100 Merge branch 'master' into topic/loopsdata commit 92c7244c16972daf244f10afc3e5bf32a8ee8f05 Author: Doug Simon Date: Tue Jan 17 14:33:19 2017 +0100 update to current compatible JDK 9 EA build commit d2545b7e067f872137ea60f429439f488e369afa Author: Andreas Woess Date: Wed Jan 11 00:38:29 2017 +0100 [fix] Truffle: ensure Assumption.isValid() is inlined for indirect calls Use concrete type for profiledArgumentTypesAssumption field so that isValid() can be statically resolved and inlined without profiling info, i.e. during PE. commit 9a8d652b1810f1d7ffbdd225d5d8d577a11f13ca Merge: 8f1ad40 ed17a0d Author: Thomas W?rthinger Date: Tue Jan 17 04:33:09 2017 -0800 [GR-2365] Fix probabilities assigned to if nodes during logic expansion phase. commit ed17a0d7eebcb07fad0847515671542024ae04b1 Author: Thomas W?rthinger Date: Tue Jan 17 05:32:18 2017 +0100 Fix probabilities assigned to if nodes during logic expansion phase. commit 8f1ad406d78f2f57104a8edaaf5aecbb3616fd35 Merge: d1057cb f64c234 Author: Thomas W?rthinger Date: Mon Jan 16 06:31:23 2017 -0800 [GR-2325] New map and set implementations for better footprint and deterministic compilation. commit f64c234e4fa7ce8991db16f782ec355b159ce9b2 Author: Thomas W?rthinger Date: Mon Jan 16 14:47:18 2017 +0100 Rename CompareStrategy to Equivalence. Make org.graalvm.util.Pair class immutable. Fix issues and compress also during iterator removal for EconomicMapImpl. commit ff500d8a7e0017d8bdbfcea2a635f3438b2a6c8f Author: Thomas W?rthinger Date: Sun Jan 15 23:17:33 2017 +0100 Add compression functionality to EconomicMapImpl. Add unit tests measuring memory footprint. commit 1d3ee0b3c524727577ed856553d876a2013fdd84 Author: Thomas W?rthinger Date: Sun Jan 15 00:59:17 2017 +0100 Create new project org.graalvm.util and move new collection classes there. commit 1ec570a5ed6a99d72dc1847a45f4ca29b7c0bad2 Merge: e7f50be d1057cb Author: Thomas W?rthinger Date: Sat Jan 14 23:57:33 2017 +0100 Merge branch 'master' into fix/predictable-compilation commit e7f50be9af761c18c05df0c87f39baa9db25ee45 Author: Thomas W?rthinger Date: Sat Jan 14 22:53:26 2017 +0100 Add capability to customize EconomicMap/EconomicSet with CompareStrategy. commit d1057cb806157581b47ed071cf7a73ccc63dc59b Merge: 45bc46a 67a36df Author: Gilles Duboscq Date: Sat Jan 14 10:12:01 2017 -0800 [merge] Change killCFG strategy commit 67a36df61043d464f170ff2a0e1bf11cba8bf2e8 Author: Gilles Duboscq Date: Fri Jan 13 15:36:42 2017 +0100 [fix] Verify killCFG under VerifyGraalGraphEdges These verification slow things down significantly so we should not perform them all the time. Currently VerifyGraalGraphs is on by default. commit dd969b6f84370ff38ffa10c0d5d5310157d10ec4 Author: Gilles Duboscq Date: Wed Jan 11 20:27:11 2017 +0100 [fix] killCFG: only verify new unused nodes under a special flag If there were unused nodes in the intial graph it's non-trivial to make sure there will be no new unused nodes. Also it shouldn't be a big problem to create new unused nodes. The flag is still usefull has it might indicate problems in killCFG. The "unsafe node" verification remains on by default as it should hold and indicates real problem that can lead to later crashes. commit 8507c94a1f8f06913bd42ec13f13c1ce70226f5c Author: Gilles Duboscq Date: Mon Jan 9 18:48:50 2017 +0100 [fix] Change killCFG strategy Instead of killing usages eagerly, just add them to the worklist. Add removeNewNodes Make public killCFG interface operate only on FixedNode to simplify the implementation: we can assume that fixed nodes that need to be deleted will be reached while working on the CFG and not only through data-flow. commit 45bc46a3d7f73c5affbc113d57794a64952d720f Merge: 2e1fc7b 6b0d63b Author: Tom Rodriguez Date: Fri Jan 13 19:21:14 2017 -0800 [merge] [fix] Rearrange debug environment initialization commit 6b0d63b856d8eb92eb406e3d94b1d2b9c42fc4dd Author: Tom Rodriguez Date: Thu Dec 15 11:04:12 2016 -0800 [fix] Rearrange debug environment initialization commit 2e1fc7bcb1a5c02cc47218d9224fbfdc7563f789 Merge: a8c7c57 6f37e24 Author: Gilles Duboscq Date: Fri Jan 13 09:05:59 2017 -0800 [merge] changes to work with jdk9-ea+152 commit 6f37e24a702e3025e3b09bad50861b2c01dddb21 Author: Doug Simon Date: Fri Jan 13 16:49:49 2017 +0100 [ci] use non-dev jdk9 snapshot commit a32ac5a6e16caa242d0c7948a1339253f52cc495 Author: Doug Simon Date: Fri Jan 13 16:03:29 2017 +0100 [feature] updated README.md for jdk9-ea+152 commit a8c7c57cab4518d647aa751fb2161790eb0027ef Merge: 6d9a9ef 380c861 Author: Lukas Stadler Date: Fri Jan 13 06:42:33 2017 -0800 [merge] [fix] PEA: more accurately detect situations in which applying effects is necessary commit 380c861cd4b3fcc3976e460d333e71da9d4585fa Author: Lukas Stadler Date: Thu Jan 12 14:39:19 2017 +0100 [fix] PEA: more accurately detect situations in which applying effects is necessary commit ad4f9ea3091d924d2fd4185063f80672bdafc775 Author: Doug Simon Date: Fri Jan 13 13:58:53 2017 +0100 [fix] added missing header commit 3e4501b691257154e527be59697248d1ec8dec46 Author: Doug Simon Date: Fri Jan 13 11:58:35 2017 +0100 [feature] update to jdk9 ea b152 commit 2a225bcf63b5dbbecbcf367412c6ffd2eca7ea5d Author: Doug Simon Date: Tue Jan 10 12:02:02 2017 +0100 [fix] fix merge conflict commit 099ca2f8b68b5e2183613e5ce77a8e5218ba62a8 Author: Doug Simon Date: Tue Jan 10 11:21:28 2017 +0100 [feature] update CheckGraalInvariants for new module name and upgrade module path property commit 8c15872a6e73df205dc52ed91f5e3e5ea73717ef Author: Doug Simon Date: Tue Jan 10 11:20:26 2017 +0100 [feature] added more AddExports annotations as required commit c547ab53f854338ea68e72ecbedd3312021351c8 Author: Doug Simon Date: Tue Jan 3 10:17:42 2017 +1000 [feature] export ASM to all GraalCompilerTests commit e52b1328705be145460d2612a2690590d41caeb3 Author: Doug Simon Date: Fri Dec 30 23:42:57 2016 +1000 [feature] use @AddExports support; force Graal to be compiled with JDK8 commit f617bd6589a70acb0ede080f785d89b40a836631 Author: Doug Simon Date: Mon Dec 12 19:06:36 2016 +0100 [feature] support module changes from JDK-8169069 commit 6d9a9ef2e164076362f8efc30b6ec80b6317d373 Merge: 879be4e 69c7657 Author: Andreas Woess Date: Thu Jan 12 01:24:47 2017 -0800 Merge pull request #686 in G/graal-core from ~ANDREAS.WOESS_ORACLE.COM/graal-core:truffle_aarch64 to master * commit '69c7657a18a76b2d6c090b5cf777c1d4822edb4a': [feature] implement AArch64OptimizedCallTargetInstumentationFactory commit 879be4e0f21b74b7912b665f82f4ed1cca8edc24 Merge: 64cb154 a8260a7 Author: Gilles Duboscq Date: Wed Jan 11 11:33:54 2017 -0800 Merge pull request #708 in G/graal-core from topic/canonicalizer-kill-unused to master * commit 'a8260a759b9113ca82bdf718b8bed31a35c976ab': [fix] Try to kill unused nodes only once during node canonicalization commit 64cb154509391ea336cad5c50bdc5d9ad2952f0b Merge: a50992c 5561cef Author: David Leopoldseder Date: Wed Jan 11 03:19:34 2017 -0800 Merge pull request #705 in G/graal-core from ~DAVID.D.LEOPOLDSEDER_ORACLE.COM/graal-core:test/recur-inline-test-reduce-iterations to master * commit '5561cef43c5f3dd5bdb368cca540f4e3a7013e58': [test] recursive inlining test: reduce number of iterations commit a8260a759b9113ca82bdf718b8bed31a35c976ab Author: Gilles Duboscq Date: Wed Jan 11 12:07:21 2017 +0100 [fix] Try to kill unused nodes only once during node canonicalization commit a50992c84db271041a53fd33400de6e2328ee967 Merge: a2255cc 61aa854 Author: Gilles Duboscq Date: Wed Jan 11 02:51:44 2017 -0800 Merge pull request #682 in G/graal-core from topic/cas to master * commit '61aa854fccee7537caf603bac3ad7bf334b75e44': [feature] Add match rules for if(cas) [feature] Add Word CAS operations commit a2255cc27855e6c1b7c9637b6e44671c7452bb04 Merge: 0cd413d 3c054a0 Author: Gilles Duboscq Date: Wed Jan 11 02:50:45 2017 -0800 Merge pull request #706 in G/graal-core from topic/debug-timeout-test to master * commit '3c054a05728f1e7e00693079f99868860fcf259f': [fix,test] Make sure debug is initialized for timeout tests commit 0cd413d6ba3293044ec378a0a5769d300d808833 Merge: 73d6f9b 78104d6 Author: David Leopoldseder Date: Wed Jan 11 00:11:25 2017 -0800 Merge pull request #704 in G/graal-core from ~DAVID.D.LEOPOLDSEDER_ORACLE.COM/graal-core:fix/box-node-nodecosts to master * commit '78104d6a2aa30872054eebc00cfb54096e0b6218': [fix] node costs: box node is a normal allocation commit 69c7657a18a76b2d6c090b5cf777c1d4822edb4a Author: Andreas Woess Date: Wed Dec 7 18:02:45 2016 +0100 [feature] implement AArch64OptimizedCallTargetInstumentationFactory commit 3c054a05728f1e7e00693079f99868860fcf259f Author: Gilles Duboscq Date: Tue Jan 10 15:27:37 2017 +0100 [fix,test] Make sure debug is initialized for timeout tests Added GraalCompilerTest.initializeForTimeout() Call it from tests with a timeout (JUnit runs those on a different thread). commit 73d6f9ba1d2c5b5d1ee86776520935ec02955129 Merge: 2aee8e8 163d970 Author: Tom Rodriguez Date: Tue Jan 10 10:18:45 2017 -0800 Merge pull request #703 in G/graal-core from igor/aot-unverified-entry-point to master * commit '163d970ac39d810404963ffdf26a76272947cb59': [fix] Fix AOT unverified entry point commit 2aee8e88c9b9b5c1f6b17671f24e474d5b8a9589 Merge: b0175e5 3072436 Author: Christian Wimmer Date: Tue Jan 10 09:53:15 2017 -0800 Merge pull request #698 in G/graal-core from cwi/invocation-plugin to master * commit '30724369f7353e6f7a3fdb0ae423434d6228a9e4': [feature] allow a InvocationPlugin to emit an invoke with an arbitrary call target commit 25b471c8b2a80bb6b8c96da6069774113f6a315a Author: Gilles Duboscq Date: Wed Jan 4 18:42:17 2017 +0100 [fix] More efficient LoopsData Improved outerFirst This assumes the order of the loops retruned by the CFG. The loops are kept in an array instead of being sources from the maps. Removed innerFirst (added ReversedList which can be used to emulate it) Removed one of the maps commit 163d970ac39d810404963ffdf26a76272947cb59 Author: Igor Veresov Date: Thu Jan 5 15:26:18 2017 -0800 [fix] Fix AOT unverified entry point commit 61aa854fccee7537caf603bac3ad7bf334b75e44 Author: Gilles Duboscq Date: Mon Dec 19 18:56:07 2016 +0100 [feature] Add match rules for if(cas) commit b0175e57a242a454a5cc75add35b3a19e192f61a Merge: 848c88a 67d77ed Author: Josef Eisl Date: Tue Jan 10 04:15:43 2017 -0800 Merge pull request #702 in G/graal-core from ~JOSEF.E.EISL_ORACLE.COM/graal-core:compiler-test-retry-failed-code-installation to master * commit '67d77ed31c0b18169917c9d746380bc4218e0a89': [fix] GraalCompilerTest: retry compilation in case of a non-permanent bailout commit 67d77ed31c0b18169917c9d746380bc4218e0a89 Author: Josef Eisl Date: Mon Jan 9 16:12:43 2017 +0100 [fix] GraalCompilerTest: retry compilation in case of a non-permanent bailout This happens, e.g., if Graal is executed the first time and Graal code invalidates assumptions that where true before. commit 848c88a0945ae71622b49f425ee76bffe3b15c56 Merge: 11ecdc0 1cbe976 Author: Codrut Stancu Date: Tue Jan 10 01:30:05 2017 -0800 Merge pull request #659 in G/graal-core from cs/refactor-interval to master * commit '1cbe97670f53b2da33fbe7ff2e4d9a90258ad4fe': [misc] Interval and Range refactoring. commit 5561cef43c5f3dd5bdb368cca540f4e3a7013e58 Author: David Leopoldseder Date: Tue Jan 10 10:05:51 2017 +0100 [test] recursive inlining test: reduce number of iterations commit 78104d6a2aa30872054eebc00cfb54096e0b6218 Author: David Leopoldseder Date: Tue Jan 10 09:49:44 2017 +0100 [fix] node costs: box node is a normal allocation commit 1cbe97670f53b2da33fbe7ff2e4d9a90258ad4fe Author: Codrut Stancu Date: Tue Dec 6 23:49:09 2016 -0800 [misc] Interval and Range refactoring. commit 11ecdc05a48b77db4718dc7cb5c31f0ff562ad5e Merge: cebca3f 718a7da Author: Josef Eisl Date: Mon Jan 9 06:19:55 2017 -0800 [fix] use ArrayList in LIR where appropriate commit d0017ebcac25a598fcc3347549b9c34ee7f5d2c4 Author: Gilles Duboscq Date: Fri Dec 9 16:22:45 2016 +0100 [feature] Add Word CAS operations Add CAS operation to Word and use them in snippets Rename `LoweredCompareAndSwap` to `LogicCompareAndSwap` Rename `CompareAndSwap` to `UnsafeCompareAndSwap` Introduce `AbstractCompareAndSwap` and `ValueCompareAndSwap` Remove old cas nodes commit 718a7da231ba2f34354125c4deaf0c241c4bfce2 Author: Josef Eisl Date: Mon Jan 9 14:36:37 2017 +0100 [fix] TraceRA: assert that splitChildren list is always non-empty or null commit 9085352ea2337a4e49bab6f02e6a5e56fd5b2567 Author: Josef Eisl Date: Wed Dec 21 15:49:40 2016 +0100 [fix] use ArrayList instead of List in Linear Scan commit 4e87b12c86109a090b0060c80df3c9609ed4d642 Author: Josef Eisl Date: Wed Dec 21 14:17:44 2016 +0100 [fix] Use ArrayDeque in LinearScanLifetimeAnalysisPhase commit 5b922da3d7ca9e641aeb3db2dd766ba2cb1d0b33 Author: Josef Eisl Date: Wed Dec 21 14:10:53 2016 +0100 [fix] TraceRA: use ArrayList in TraceBuilderPhase commit 0f61ffd3a690e082c67dc217ced4bd40c8f7c239 Author: Josef Eisl Date: Wed Dec 21 14:01:45 2016 +0100 [fix] use ArrayList in RegisterVerifier commit 45a5e1fce556453fc2ac37f8a63966e2c72a1586 Author: Josef Eisl Date: Wed Dec 21 13:55:20 2016 +0100 [fix] TraceRA: use ArrayList in TraceLocalMoveResolver commit 432b97221d90df67ac1ce8d9b72468ad7e716bcc Author: Josef Eisl Date: Wed Dec 21 13:54:59 2016 +0100 [fix] TraceRA: use ArrayList in TraceLinearScanWalker commit 956cd75f3d938aeb00ba9d8e9e0e0487ec8248ae Author: Josef Eisl Date: Wed Dec 21 13:42:41 2016 +0100 [fix] use ArrayList for storing getLIRforBlock results commit 99c028c31a2035623269d66417073d5ce527ffd8 Author: Josef Eisl Date: Wed Dec 21 13:23:16 2016 +0100 [fix] TraceRA: use ArrayList for split children in TraceInterval commit 5dd26d98c9b48b8118047c41d3e24f9e63c594d4 Author: Josef Eisl Date: Wed Dec 21 12:18:21 2016 +0100 [fix] TraceRA: use index instead of ListIterator commit cebca3f9a2eabb1c411240eae19ea23041d3dde0 Merge: 0176906 1f349d8 Author: Christian Wimmer Date: Mon Jan 9 03:01:27 2017 -0800 [fix] Spill store optimization must not move stores into loop commit 0176906c5dc53edd512d8f489fce3a8dcd59e340 Merge: f76dc33 43efd9d Author: Doug Simon Date: Mon Jan 9 01:05:28 2017 -0800 Improve consistency in the README terminology commit 2f22f70b450f3e089480b2347520b3610b1f1f96 Author: Thomas W?rthinger Date: Sat Jan 7 00:59:53 2017 +0100 Introduce new economic map and set implementation with low footprint (in particular for small number of entries) to replace java.util.* usage. commit f76dc33b4fd41569991eb4659be8028190804f41 Merge: 0063385 39c1579 Author: Christian Wimmer Date: Fri Jan 6 09:15:08 2017 -0800 Minor fixes for my work on typed Truffle entrypoints on Substrate VM commit 39c1579fc91ca9d401e1a55f37fa8655cc2033a9 Author: Christian Wimmer Date: Thu Jan 5 16:49:50 2017 -0800 [feature] Change order of objects in equality check Allows customizing graph().method() with a marker method that has an approriate implementation of equals() commit 28a72b5bff8ca41cb023bae7cfe6e83854493b99 Author: Christian Wimmer Date: Thu Jan 5 15:58:55 2017 -0800 [fix] Avoid NullPointerException commit 30724369f7353e6f7a3fdb0ae423434d6228a9e4 Author: Christian Wimmer Date: Thu Jan 5 15:50:49 2017 -0800 [feature] allow a InvocationPlugin to emit an invoke with an arbitrary call target commit 006338540a46a1a228eeed01060c94ddbc4cb2e4 Merge: cb09b1b 07518dd Author: Christian Humer Date: Thu Jan 5 06:25:46 2017 -0800 Truffle OSR compilation: synchronize / cancel on invalidate commit cb09b1be120594b7e4f6a291b229da52e9ae6da4 Merge: 0584562 8ba423d Author: Christian Wimmer Date: Wed Jan 4 17:51:24 2017 -0800 Update Truffle version commit 8ba423d0ae840eb8ad521f76e06a89d537ec95f1 Author: Christian Wimmer Date: Wed Jan 4 16:31:19 2017 -0800 Update ci overlay version commit 4876c7208991810e53de8e7b34be51279ba05bbb Author: Christian Wimmer Date: Wed Jan 4 15:56:54 2017 -0800 Update Truffle version commit 0584562485b4e259f99d31a3999ced0be12f43c9 Merge: 3cb2e3f 5394eb5 Author: Vojin Jovanovic Date: Wed Jan 4 15:20:34 2017 -0800 Use correct intellij init command commit 5394eb535ee09020bd44bd4ca639327547b9b0b8 Author: Vojin Jovanovic Date: Wed Jan 4 23:30:57 2017 +0100 Changing the ci overlay commit c4f207aba3e7e0a314c955ec45494518228cf91e Author: Vojin Jovanovic Date: Wed Jan 4 15:19:42 2017 +0100 Use the correct IntelliJ init command commit 07518dd5f2deac85f94f3c8d8f0f13e2e778f688 Author: Lukas Stadler Date: Wed Jan 4 09:59:41 2017 +0100 cancel compilation task when OptimizedCallTargets are invalidated commit 82f2ca37f6c90bfec631b7a248972aa5e209de05 Author: Lukas Stadler Date: Wed Jan 4 09:59:13 2017 +0100 synchronize OptimizedOSRLoopNode.invalidateOSRTarget commit 3cb2e3f2343ddde3607819a479ef84ee95666f38 Merge: e4b3fb4 76d7382 Author: Christian Humer Date: Mon Jan 2 05:37:37 2017 -0800 [fix] Truffle compilation canceled itself in case of recursion commit 1f349d8c284d102e219d6ab76ec20a3a0a5c7e7b Author: Christian Wimmer Date: Fri Dec 30 14:42:41 2016 -0800 [fix] Spill store optimization must not move stores into loop commit 01fa2211d4a22a3cb50f2cfa2c047c30b5060763 Author: Thomas W?rthinger Date: Fri Dec 30 15:27:36 2016 +0100 Make compilation predictable by removing System.identityHashCode and related effects such as HashMap key iteration order as main sources of indeterminism. commit e4b3fb4ea236cadf34d407ee8ab216b8d76c5e87 Merge: 0cefb9f 2369e57 Author: Thomas W?rthinger Date: Fri Dec 30 04:07:05 2016 -0800 Add unit tests and small fixes for Truffle performance warning stack traces. commit 0cefb9fd32e48c5df87b335b8eaeaf170a286f34 Merge: f9b93f3 d2861f6 Author: Tom Rodriguez Date: Thu Dec 22 11:02:57 2016 -0800 Merge: Remove folding of loads from $assertionsDisabled in AOT commit 43efd9d41a00c3af1b8d7c269ce9cb3aaf12220a Author: Benoit Daloze Date: Thu Dec 22 17:52:29 2016 +0100 Improve consistency in the README terminology commit f9b93f32173b3fbbba6a05de9f26a0e6596aebe2 Merge: 7e25820 22a5ec9 Author: Vojin Jovanovic Date: Thu Dec 22 08:32:02 2016 -0800 Adding IntellJ instructions commit 22a5ec9ed9650c4452fe39fe7b64381360e111df Author: Vojin Jovanovic Date: Wed Dec 21 20:18:23 2016 +0100 Adding IntellJ instructions commit d2861f6fd589ce28301af30031fd3f824ccd3f12 Author: Igor Veresov Date: Tue Dec 20 11:13:10 2016 -0800 Remove folding of loads from $assertionsDisabled in AOT commit 7e258203177a6b2c7414cfeb35a133cc2e6c9670 Merge: 10af3e9 a25f5c9 Author: Tom Rodriguez Date: Mon Dec 19 16:55:09 2016 -0800 Merge: Fix argument types of tiered callbacks commit 76d738205406860fb2a85c930e3b1907cdb44cca Author: Christian Wimmer Date: Mon Dec 19 10:05:36 2016 -0800 [fix] Truffle compilation canceled itself in case of recursion commit 10af3e90b068c6298a8f185606fd2566d7cc53a1 Merge: fad8200 b225edc Author: Gilles Duboscq Date: Mon Dec 19 10:13:11 2016 -0800 Use extended JavaVm API from mx commit b225edc82382ea7ea33bbb675836b3498b333b42 Author: Gilles Duboscq Date: Wed Dec 14 16:27:12 2016 +0100 Use extended JavaVm API from mx commit fad8200eb15881121add70ba7fae6ba23fa53b90 Merge: d279402 7baf0fb Author: Josef Eisl Date: Mon Dec 19 06:21:34 2016 -0800 [ci]: increase jmh timeouts commit 7baf0fb18ab35bb60c4be5a236af34118696859e Author: Josef Eisl Date: Mon Dec 19 10:55:18 2016 +0100 [dep] update ci-overlay commit a824d7027795a386a8e5261eda19b703ae1aefaa Author: Josef Eisl Date: Mon Dec 19 10:54:54 2016 +0100 [ci] increase timelimit for graal-micros commit d27940208dd5f7fbba90c80ae6744986d8c50897 Merge: ac1842f c00c0bf Author: Matthias Grimmer Date: Mon Dec 19 02:22:45 2016 -0800 Add PUBLICATIONS.md commit ac1842ffd07e038c0833526f02f65d67c674ce0d Merge: a348ee7 260a8a6 Author: Tom Rodriguez Date: Sun Dec 18 20:57:14 2016 -0800 Merge: Allow multiple file names per thread and gracefully handle interrupted threads commit a25f5c9656b4910204f35c5a33e6a0000b45d48b Author: Igor Veresov Date: Fri Dec 16 18:23:54 2016 -0800 Fix argument types of tiered callbacks commit 2369e57d367fa81048d087e7cf9dacd3f1782b30 Author: Thomas W?rthinger Date: Sat Dec 17 01:04:29 2016 +0100 Add unit tests and small fixes for Truffle performance warning stack traces. commit a348ee713af1046b297d2a0326a6d1322bba004e Merge: 01695c8 5adca69 Author: Christian Humer Date: Fri Dec 16 09:53:32 2016 -0800 Print approximated stack trace for performance warnings. commit 5adca69e81e626443f1df00493b1a556aef81e92 Author: Christian Humer Date: Fri Dec 16 18:19:09 2016 +0100 Print approximated stack trace for performance warnings. commit 260a8a6e8059b3e2b33ad756516bd0c12d4e52ba Author: Tom Rodriguez Date: Thu Dec 15 11:04:12 2016 -0800 Allow multiple file names per thread and gracefully handle interrupted threads commit 01695c855a62af8764fe97421b6341e6a2622b63 Merge: 45e1839 56a21fb Author: Gilles Duboscq Date: Fri Dec 16 06:34:13 2016 -0800 Run bootstraps on machines with 'manycores' commit 56a21fb6586dc33fec5d38233616ef517a0fe48e Author: Gilles Duboscq Date: Fri Dec 16 14:44:51 2016 +0100 Run bootstraps on machines with 'manycores' Bootstraps are multithreaded so they benefit from having many cores. commit 45e183972bd9dc0ed6b5405c101ff1907a92d774 Merge: 9e5cf7f 36d2215 Author: Gilles Duboscq Date: Fri Dec 16 05:27:18 2016 -0800 Aarch64 graal misc fixes commit 9e5cf7f3e3d16a41c64031887a84a5278ef0c415 Merge: dd087fd 950b11e Author: David Leopoldseder Date: Fri Dec 16 04:09:16 2016 -0800 [ci] split bootstrap tasks commit dd087fd6447e1d2dc2a61920bbb1ae431457b5ef Merge: e34a1a3 a829731 Author: Tom Rodriguez Date: Wed Dec 14 12:09:49 2016 -0800 Merge: Fold narrow and sign extend into memory operations commit a8297312cd4d0da16b05bee3d50b79721590087c Author: Tom Rodriguez Date: Tue Dec 6 17:52:08 2016 -0800 Fold narrow and sign extend into memory operations commit bbb7350518fefd8792bac9735941c421d7d36470 Author: Doug Simon Date: Wed Dec 14 17:49:54 2016 +0100 remove redundant AllocationContext.options field commit d3b971522b400a4ff6991e08b107c687c65336b4 Author: Doug Simon Date: Wed Dec 14 16:17:59 2016 +0100 remove use of StableOptionKey commit dd36a08fa7da15d39b2dba20298aa1cbfae3f2c9 Author: Doug Simon Date: Wed Dec 14 15:58:52 2016 +0100 remove use of OptionKey.setValue commit dd11ad4cc1f1de2a5bb9abae86fc3aff9dfac7aa Author: Doug Simon Date: Wed Dec 14 14:53:23 2016 +0100 disable Inline and InlineDuringParsing if they have default values and -XX:-Inline is specified commit 6463f9f60fbbdeefa0e48c68b82994689c714a78 Merge: 2bfa0b1 e34a1a3 Author: Doug Simon Date: Wed Dec 14 13:23:24 2016 +0100 Merge branch 'master' into topic/non-static-options commit 2bfa0b1818bd368629091fc6e621ac87bd7acd85 Author: Doug Simon Date: Sat Dec 10 01:03:23 2016 +0100 post-merge fixups commit e34a1a3130f275d7390d50fda3ecf3f1abe42200 Merge: b332b62 686e261 Author: Gilles Duboscq Date: Wed Dec 14 03:04:12 2016 -0800 Remove unused array allocation if possible commit 950b11ecbca37ed8331e6762ee788b19f4edcf84 Author: David Leopoldseder Date: Wed Dec 14 10:59:14 2016 +0100 [ci] move bootstrap tasks to separate hocon file commit 91488eca7df066abc589300b0c45578578c68862 Author: David Leopoldseder Date: Wed Dec 14 10:50:58 2016 +0100 [ci] update mx version commit b332b62735ba82137706265157d8432a8fb48f6e Merge: 26c0792 d9708df Author: David Leopoldseder Date: Wed Dec 14 01:57:01 2016 -0800 [ci] trigger TraceRA benchmark daily only commit d9708df27a5159b851d5c333868c0cf7f5dd760f Author: Josef Eisl Date: Wed Dec 14 10:23:42 2016 +0100 [ci] trigger TraceRA benchmarks daily only commit 26c07924c31e03795c17c3e6f6861a90b14a6d21 Merge: 35cc2e8 f322ca2 Author: Gilles Duboscq Date: Tue Dec 13 08:27:49 2016 -0800 Add more cases to MonitorSnippets commit 36d22155092abd67cce70c963873e95884553c8a Merge: f812ad3 35cc2e8 Author: Gilles Duboscq Date: Tue Dec 13 16:33:58 2016 +0100 Merge branch 'master' into aarch64_graal_misc_fixes commit 35cc2e8d5ad64626fd97b3b72012299137ade43b Merge: dea00ee 977b6e1 Author: David Leopoldseder Date: Tue Dec 13 05:17:41 2016 -0800 [fix] loop optimizations must respect ControlFlowAnchored interface commit dea00eee58b2bb355755cfad45ccae95c43380b1 Merge: 87b6ac0 ef00cd9 Author: Josef Eisl Date: Tue Dec 13 03:58:20 2016 -0800 [dep] update ci overlay version commit 977b6e18628b2ddbe590c1bfc6d71aaa303dbcab Author: David Leopoldseder Date: Fri Dec 2 14:58:02 2016 +0100 [fix] control flow anchored: loop optimizations must use controlflowanchored interface commit c00c0bf719d433fc549668f551c845cc8bf93a50 Author: Matthias Grimmer Date: Mon Dec 12 10:30:14 2016 +0100 Add all Graal and Truffle related publications to Publications.md commit ef00cd905b55c1348a7cfa597150e52ad45683e3 Author: Josef Eisl Date: Mon Dec 12 13:17:01 2016 +0100 [dep] update ci overlay version commit f812ad3b8d847f797b3123adda0b330e1f0d78af Author: Andrew Haley Date: Mon Dec 12 18:09:36 2016 +0000 The parameters a and b should not be assigned. commit d224c1c6d681c83914b78752ba932639fb9d2613 Author: Andrew Haley Date: Mon Dec 12 17:25:17 2016 +0000 Remove unused arguments. commit 686e261039d357f04026d7286c0cf0f2d8c17238 Author: Gilles Duboscq Date: Mon Dec 12 17:47:08 2016 +0100 NewArray: only simplify to guards before guard lowering commit 13610fab8df4dfebc0794f8fcf5b5bf09d175662 Author: Gilles Duboscq Date: Thu Nov 17 16:16:40 2016 +0100 Simplify unused NewArray to Guard If we can not prove that the length is positive, a guard is still cheaper than the full allocation. Add tests for negative and large newarray Add tests that unused NewArray nodes are eliminated commit 53b5ee55c4ad09a473c0d596c2d485f247c0e475 Author: Gilles Duboscq Date: Wed Nov 16 18:01:46 2016 +0100 Remove unused array allocation if possible Remove unused array allocations where the length is known to be positive. commit 87b6ac064e1497442ac0e3125e31496fff517791 Merge: fb63259 cf008ba Author: Codrut Stancu Date: Sun Dec 11 22:23:39 2016 -0800 Refactor Node collections factory methods. commit fb632591c6c36b55451e936b7e20a37a67a726ea Merge: 784baba dc3ecab Author: Codrut Stancu Date: Fri Dec 9 11:21:26 2016 -0800 Refactor phase classes statistics access. commit 784babac9f9b68526d91ac17d6a305fff6a50f8a Merge: 451c012 414f878 Author: Codrut Stancu Date: Fri Dec 9 10:15:54 2016 -0800 Small refactorings. commit 451c012ae740f28e0e296ad965fe48ad9e596447 Merge: 98bdc0d 81fcc75 Author: Aleksandar Prokopec Date: Fri Dec 9 08:15:01 2016 -0800 Update package name for Renaissance bench. commit 565cdc3b08a3620ee13905af9fb7d60d4bfc07f9 Merge: 98bdc0d b200740 Author: Doug Simon Date: Fri Dec 9 17:05:23 2016 +0100 Merge branch 'aarch64_graal_misc_fixes' of https://github.com/theRealAph/graal-core into theRealAph-aarch64_graal_misc_fixes commit f322ca24e9b10082931dbf2a201673c7ffdf1c44 Author: Gilles Duboscq Date: Fri Dec 9 16:39:37 2016 +0100 Fix typos an document verify_jvmci_ci_versions commit 81fcc75fe00578f70228ee54ad5262625790f804 Author: Aleksandar Prokopec Date: Fri Dec 9 16:06:05 2016 +0100 Update package name for Renaissance bench. commit a8cf4587485ccfd78226629c54c0757d7a043b2f Author: Gilles Duboscq Date: Fri Dec 9 15:04:49 2016 +0100 Relax verify_jvmci_ci_versions Allow travis jvmci versions to lag behind *dev* version of ci.hocon jvmci. Use that and make travis use 0.23. The fastdebug versions names have been changed: s/fastdebug-dev/dev-fastdebug/ commit da9279e3eae2cdacd3b75e3f34118d09e3789698 Merge: b1e8360 949f9d4 Author: Doug Simon Date: Fri Dec 9 13:22:00 2016 +0100 Merge branch 'master' into topic/non-static-options commit dc3ecabbbfa5df7ae3ed79d9b85a69c567585dd1 Author: Codrut Stancu Date: Fri Oct 28 22:27:35 2016 -0700 Refactor phase classes statistics access. commit 414f8788c9864645099207c22cec763fa8f467b3 Author: Codrut Stancu Date: Tue Dec 6 23:44:56 2016 -0800 Replace stream use with simple for. commit e71f7dd62d6e1afab9f4da8b5ba91f4f127cdee5 Author: Codrut Stancu Date: Tue Dec 6 22:47:07 2016 -0800 Small refactorings. commit cf008ba7259bc432a3b83149c3b2a90c5fedc97a Author: Codrut Stancu Date: Tue Dec 6 22:36:54 2016 -0800 Refactor node collections factory methods. commit b20074008b6c89b94ddcf1482872a0fdef3ca960 Author: Andrew Haley Date: Thu Dec 8 15:15:04 2016 +0000 Checkstyle fixes. commit b1e836066425b1037481f5dfd7ddf916c627be05 Author: Doug Simon Date: Thu Dec 8 15:21:45 2016 +0100 made StructuredGraph final commit c560a5a8e8ada8012d06e39f31e67ac0cc90b6ca Author: Doug Simon Date: Thu Dec 8 15:02:21 2016 +0100 moved global option parsing from HotSpotGraalCompilerFactory to OptionValues removed methods from OptionKey that did not have a OptionValues context commit 21a2d2ab1ed5f6763937a761852445e0b223c16d Merge: 94016fc 949f9d4 Author: Gilles Duboscq Date: Thu Dec 8 12:57:03 2016 +0100 Merge remote-tracking branch 'master' into topic/locking commit 94016fca16a6cf5c04921c9c75ff5d257db99505 Author: Gilles Duboscq Date: Wed Dec 7 20:54:07 2016 +0100 MonitorSNippet: adjust probabilities commit c95ad5d1f5ac34e1579671060bc57b7ff6fafc70 Author: Gilles Duboscq Date: Wed Dec 7 19:39:37 2016 +0100 Cleanup: remove logging and unused location commit 9760b9b83e306d009a2083c4307706c9d3457862 Author: Gilles Duboscq Date: Tue Dec 6 11:53:03 2016 +0100 MonitorSnippets: Check for inflated lock earlier commit df4017e2d7d35c3468985a4f2251a787ee808b84 Author: Gilles Duboscq Date: Mon Dec 5 16:40:45 2016 +0100 Add SimpleFastInflatedLocking option commit 578654103c4532db520dae79ee5b44f106bd7228 Author: Gilles Duboscq Date: Mon Dec 5 15:57:44 2016 +0100 WIP: log inflated fast locking support commit 6fb059580aec479cf107495e181b01c308ffbb58 Author: Gilles Duboscq Date: Fri Dec 2 18:20:13 2016 +0100 Remove fast-path handling of succ in monitor snippets commit 38d9197d3d68a8057b57b25546c6b8f6240b1094 Author: Gilles Duboscq Date: Fri Dec 2 16:30:55 2016 +0100 Use JVMCI 0.24-dev commit 17865b46b77c91b606c9300eb6a831bdcee292e6 Author: Gilles Duboscq Date: Wed Nov 30 21:01:54 2016 +0100 Implement fast-path unlocking of inflated locks commit 98728bf2030e1e90c2cc38fb4260fc23cf329607 Author: Gilles Duboscq Date: Wed Nov 30 17:25:57 2016 +0100 Adjust monitor snippet probabilities commit 31cd4bf6e45d8ea20f114f7a094d66af9e30af05 Author: Andrew Haley Date: Tue Dec 6 19:02:34 2016 +0000 Tidy up Arrays.equals() support. commit 4dab38e0ad61633392e64ee80ec4e9018c534941 Author: Andrew Haley Date: Tue Dec 6 18:13:37 2016 +0000 Implement Arrays.equals(). commit 0201dec4f6e13b8d97d1daf2459f6a5204a36813 Author: Andrew Haley Date: Tue Dec 6 13:45:32 2016 +0000 Allow SP and ZR forms of add and subtract instructions. commit 6b899c1321a6196c3dbefab4e7c253d80a864477 Author: Andrew Haley Date: Tue Dec 6 13:19:50 2016 +0000 Disable NativeFunctionInterfaceTest on AArch64 commit 1419900803c872265815b3973950013192f3382c Author: Andrew Haley Date: Mon Dec 5 17:47:34 2016 +0000 Fix compressed OOPs. commit 61ce002370b1b242412583c878bebebef253e5ee Author: Andrew Haley Date: Mon Dec 5 17:22:29 2016 +0000 Fix flag-setting arithmetic. commit 14b4dd5156e1a2b894c784eaa0452ffc5558ee40 Author: Andrew Haley Date: Mon Dec 5 17:22:05 2016 +0000 Fix compressed OOPs. commit 49021aaee377250ac3f51a453f092c233a8fb433 Author: Andrew Haley Date: Mon Dec 5 16:25:15 2016 +0000 Fix null checks for compressed OOPs. commit 919e22adfbe7e54eb8eee580bd446aa9068a5c17 Author: Doug Simon Date: Sat Dec 3 14:41:24 2016 +0100 removed extra semicolon commit aa93c9a61b7a30cb6afa5d0e8c014559f62e5bf9 Author: Doug Simon Date: Sat Dec 3 12:41:24 2016 +0100 improved name of instrumentation graphs commit e6a181c024eb2e21f83c80723c42b678bed66577 Author: Doug Simon Date: Sat Dec 3 12:41:00 2016 +0100 fixed creation of inlineable graphs commit 2dfe1821a024735e2f80e126b2d533ac1a35ebe8 Author: Doug Simon Date: Sat Dec 3 12:40:26 2016 +0100 fixed chaining in overloaded methods commit 7241d89909511e833a6e39da4a1b83c62d841cb5 Author: Doug Simon Date: Sat Dec 3 12:39:57 2016 +0100 improved assertion message commit c90e054a12ed0d2b8d84392e2cac3be83386db6c Author: Doug Simon Date: Sat Dec 3 12:39:32 2016 +0100 also dump graph on error if -Dgraal.Dump option is non-null commit 07871cc653a07fe232121151b19b644fdf825cda Author: Doug Simon Date: Sat Dec 3 02:43:53 2016 +0100 removed incorrect assertion commit aae87fcc32a05937fa62ef7b55ae57bdfa87604a Merge: 5ec5c96 beb0da8 Author: Doug Simon Date: Sat Dec 3 02:32:03 2016 +0100 Merged from master branch and added StructuredGraph.Builder commit 2024acb03c0080ae48cb304be40bdc7c7072c2b4 Author: Andrew Haley Date: Fri Dec 2 19:00:30 2016 +0000 Exclude a couple of features which are not supported on AArch64. commit b72dd5e5aac1d2faf6a401c3dc7d0ce5e1698bfd Author: Andrew Haley Date: Fri Dec 2 18:59:46 2016 +0000 We should only load SP from FP in old versions of HotSpot. commit 5ec5c9663778c4e5e541573eb18c22ddd51cb91e Author: Doug Simon Date: Fri Dec 2 11:56:35 2016 +0100 removed OptionKey.getValue() and added OptionValues argument to all OptionKey.getValue(OptionValues) call sites commit 06875b567b71f7ae501c05530f9da96fd6736a09 Author: Doug Simon Date: Fri Dec 2 11:53:11 2016 +0100 override sl command in graal-core to run on Graal commit f56f8fb0d01a89ba2c25afcaa6feadb867ef7982 Author: Andrew Haley Date: Thu Dec 1 17:57:41 2016 +0000 Call runtime versions of intrinsics for transcendental funtions. commit 691991eb69e7f49b59a34cfaefec17dfc531e2e4 Author: Andrew Haley Date: Thu Dec 1 16:58:08 2016 +0000 Integer arithmetic notes which set flags. commit 13571ae9e90e90f7e1ec21c24d55ebe76637c3b8 Author: Andrew Haley Date: Thu Dec 1 10:58:52 2016 +0000 Implement countTrailingZeros. commit e8b6a30ab4e8a5e4040cbaabeffcd83eaa225e79 Author: Andrew Haley Date: Wed Nov 30 17:14:29 2016 +0000 Make add and sub work with 64-bit immediate values. commit 30c5b3f28a206658a5071f95a7848b0c8cb61836 Author: Andrew Haley Date: Wed Nov 30 12:38:27 2016 +0000 Correct method prologue and epilogue. commit 5f4c55161ea5e9e2903d06820780828b32249bc2 Author: Andrew Haley Date: Tue Nov 29 12:25:38 2016 +0000 Fix more unused warnings. commit d019de960477f5231a3082ad5c4cb13b69f5afda Author: Andrew Haley Date: Mon Nov 28 18:05:45 2016 +0000 Fix a few style errors. commit 806776592214b8a56e81f5a2124bf0f1b334fff4 Author: Andrew Haley Date: Mon Nov 28 12:03:49 2016 +0000 Fix a few style errors. commit a948773abcc02f51215399035c9679451325d562 Author: Andrew Haley Date: Sun Nov 27 17:36:53 2016 +0000 Fix warnings. commit 32c443b5c7c84aa6b42737d1224ead44db2730a1 Author: Andrew Haley Date: Fri Nov 25 16:10:43 2016 +0000 Fix a few style errors. commit 539cbc147dea9e1fde5e87e139eadcb5648d1e36 Author: Andrew Haley Date: Fri Nov 25 16:09:37 2016 +0000 Fix a few style errors. commit 4add54a62a32008a65f4ebe3300af52b295967c0 Author: Andrew Haley Date: Fri Nov 25 10:05:15 2016 +0000 Add missing braces. commit 5bd69ae0bd9e7ee047030c070408d92c218ca6c0 Author: Doug Simon Date: Fri Nov 25 10:53:59 2016 +0100 fixed sourcing of UseGraalInstrumentation value to be compilation local commit 49a97ed46c1e9a6830c53593b359a595e665266b Author: Doug Simon Date: Fri Nov 25 00:45:44 2016 +0100 fixed bugs in option overriding commit 64a0c239a728c1f0312d6f0687968ec53034bbab Author: Doug Simon Date: Wed Nov 23 00:31:20 2016 +0100 removed OptionKey.OverrideScope, DerivedOptionValue and BailoutAndRestartBackendException commit bb7d832717ea86a33e7415e9c49b243166cd0389 Author: Doug Simon Date: Wed Nov 16 16:10:37 2016 +0100 moved StableOptionKey invariant checking to OptionValues commit ab6f8b6760da3dcc2ede4b4589b041293498b972 Author: Doug Simon Date: Sun Nov 13 14:24:24 2016 +0100 indirect through OptionValues for getValue/setValue/hasBeenSet commit 45ffff3171466ea9539cd2d358868e5ad8183d50 Author: Doug Simon Date: Sat Nov 12 13:37:51 2016 +0100 fixed failing unit tests commit 5d981456e4dca208039e6e3986f36075713248aa Author: Doug Simon Date: Sat Nov 12 00:51:46 2016 +0100 rename OptionValue to OptionKey and added OptionValues as the store for values associated with OptionKeys commit 22edd24ca2c8af9ef010271d563b7dfdf63733e5 Author: Doug Simon Date: Fri Nov 11 18:16:35 2016 +0100 removed OptionValue profiling commit 42c3e92a20105bde82005616a0955c01deeda152 Author: Andrew Haley Date: Thu Nov 24 16:26:01 2016 +0000 Many small AArch64 fixes From dean.long at oracle.com Tue Mar 21 18:47:35 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 21 Mar 2017 11:47:35 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> Message-ID: On 3/21/17 9:37 AM, Vladimir Ivanov wrote: >> and webrev.2 with it removed: >> >> http://cr.openjdk.java.net/~dlong/8158168/webrev.2/ > > Thanks, Dean. I started with webrev.2 and tried to minimize the > changes. I ended up with the following version: > > http://cr.openjdk.java.net/~vlivanov/dlong/8158168/webrev.00/ > Thanks. The reason I didn't go with that approach from the beginning is because I couldn't convince myself that I could find all the missing bounds checks, and I wanted an interface to test against. With the bounds checks in AbstractStringBuilder, it is very hard to test all the possible race conditions, because some of the race conditions only happen when an ASB field changes half-way through the method. > Some clarifications: > > ============ > src/java.base/share/classes/java/lang/String.java: > > The bounds check is needed only in String.nonSyncContentEquals when it > extracts info from AbstractStringBuilder. I don't see how out of > bounds access can happen in String.contentEquals: > if (n != length()) { > return false; > } > ... > for (int i = 0; i < n; i++) { > if (StringUTF16.getChar(val, i) != cs.charAt(i)) { > OK. > ============ > src/java.base/share/classes/java/lang/StringConcatHelper.java: > > I think bounds checks in StringConcatHelper.prepend() are skipped > intentionally, since java.lang.invoke.StringConcatFactory constructs > method handle chains which already contain bounds checks: array length > is precomputed based on argument values and all accesses are > guaranteed to be in bounds. > This is calling the trusted version of getChars() with no bounds checks. It was a little more obvious when I had the Trusted inner class. > ============ > src/java.base/share/classes/java/lang/StringUTF16.java: > > + static void putChar(byte[] val, int index, int c) { > + assert index >= 0 && index < length(val) : "Trusted caller > missed bounds check"; > > Unfortunately, asserts can affect inlining decisions (since they > increase bytecode size). In order to minimize possible performance > impact, I suggest to remove them from the fix targeting 9. > Sure. > ============ > private static int indexOfSupplementary(byte[] value, int ch, int > fromIndex, int max) { > if (Character.isValidCodePoint(ch)) { > final char hi = Character.highSurrogate(ch); > final char lo = Character.lowSurrogate(ch); > + checkBoundsBeginEnd(fromIndex, max, value); > > The check is redundant here. fromIndex & max are always inbounds by > construction: > > public static int indexOf(byte[] value, int ch, int fromIndex) { > int max = value.length >> 1; > if (fromIndex < 0) { > fromIndex = 0; > } else if (fromIndex >= max) { > // Note: fromIndex might be near -1>>>1. > return -1; > } > ... > return indexOfSupplementary(value, ch, fromIndex, max); > OK. > ============ > I moved bounds checks from StringUTF16.lastIndexOf/indexOf to > ABS.indexOf/lastIndexOf. I think it's enough to do range check on > ABS.value & ABS.count. After that, all accesses should be inbounds by > construction (in String.indexOf/lastIndexOf): > > jdk/src/java.base/share/classes/java/lang/StringUTF16.java: > static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, > String tgtStr, int fromIndex) { > > int rightIndex = srcCount - tgtCount; > if (fromIndex > rightIndex) { > fromIndex = rightIndex; > } > if (fromIndex < 0) { > return -1; > } > > jdk/src/java.base/share/classes/java/lang/StringUTF16.java: > public static int lastIndexOf(byte[] src, int srcCount, > byte[] tgt, int tgtCount, int > fromIndex) { > int min = tgtCount - 1; > int i = min + fromIndex; > int strLastIndex = tgtCount - 1; > char strLastChar = getChar(tgt, strLastIndex); > > startSearchForLastChar: > while (true) { > while (i >= min && getChar(src, i) != strLastChar) { > > There are 2 places: > * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - inbound > > * getChar(src, i); i in [ min; min+fromIndex ] > min = tgtCount - 1 > rightIndex = srcCount - tgtCount > fromIndex <= rightIndex > > 0 <= min + fromIndex <= min + rightIndex == (tgtCount - 1) > + (srcCount - tgtCount) == srcCount - 1 > > Hence, should be covered by the check on count & value: > public int lastIndexOf(String str, int fromIndex) { > + byte[] value = this.value; > + int count = this.count; > + byte coder = this.coder; > + checkIndex(count, value.length >> coder); > return String.lastIndexOf(value, coder, count, str, fromIndex); > } > OK, I will go with your version if it's OK with Sherman. dl > Best regards, > Vladimir Ivanov > >> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>> >>>>> I have the same concern. Can we fix the immediate problem in 9 and >>>>> integrate verification logic in 10? >>>>> >>>> >>>> OK, Tobias is suggesting having verification logic only inside the >>>> intrinsics. Are you suggesting removing that as well? >>> >>> Yes and put them back in 10. >>> >>>> I'm OK with removing all the verification, but that won't reduce the >>>> library changes much. I could undo the renaming to >>>> Trusted.getChar, but >>>> we would still have the bounds checks moved into StringUTF16. >>> >>> I suggest to go with a point fix for 9: just add missing range checks. >> From daniel.daugherty at oracle.com Tue Mar 21 19:51:36 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 21 Mar 2017 13:51:36 -0600 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot src/os/aix/vm/os_aix.cpp No comments. src/os/bsd/vm/os_bsd.cpp No comments. src/os/linux/vm/os_linux.cpp L729: // In that cast just subract the page size to get the maximum possible stack size. Typo: 'cast' -> 'case' Typo: 'subract' -> 'subtract' (Thomas also commented on it) src/os/posix/vm/os_posix.cpp L263: // aligning up could have resulted in the size being 0. In that case just subract the Nit: 'aligning' -> 'Aligning' (since it's a sentence) Typo: 'subract' -> 'subtract' src/os/solaris/vm/os_solaris.cpp No comments. src/share/vm/prims/jvm.cpp L2812: // -Avoid truncating on 32-bit platforms if size is greater than UINT_MAX Nit: needs a period at the end like L2813. test/runtime/Thread/TooSmallStackSize.java No comments. test/runtime/Thread/TestThreadStackSizes.java L26: * @summary Test user threads with various stacks sizes. Typo?: "stacks sizes" -> "stack sizes" L36: super(null, null, "TestThreadStackSizes"+stackSize, stackSize); Nit: spaces around the "+". L46: TestThreadStackSizes testThreadStackSize = new TestThreadStackSizes(stackSize); Nit: extra space before '='. So this test makes 326 createThread() calls... how long does it take to run? Thumbs up! I don't need to see another webrev if you choose to fix these minor typos... Dan On 3/20/17 5:29 PM, Chris Plummer wrote: > On 3/17/17 11:37 PM, Chris Plummer wrote: >> On 3/17/17 8:17 PM, Chris Plummer wrote: >>> On 3/17/17 7:01 PM, David Holmes wrote: >>>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>>> Looks like this will need some more work since the added asserts are >>>>> triggering on mac os x (which is the only place we'd currently expect >>>>> them to assert). >>>>> >>>>> The problem is that the code I found that rounds up to the page >>>>> size is >>>>> only applied to java threads created by the VM for which the java >>>>> user >>>>> specified no stack size. The VM and Compiler thread sizes are not >>>>> rounded. The failure I saw was with >>>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>> when is >>>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit the >>>>> assert >>>>> with an EINVAL. The size is not aligned, but it could also be >>>>> complaining because it is too big. I haven't tried aligning it yet >>>>> to see. >>>>> >>>>> On Linux we do the following: >>>>> >>>>> stack_size = align_size_up(stack_size + >>>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>>> >>>>> We don't do this on BSD. I think the same on BSD would solve this >>>>> problem. I'm not sure about adding the guard size. I'll need to >>>>> see if >>>>> mac os x has the same pthread bug as linux does. >>>> >>>> At this stage I would only deal with alignment issues. IIRC the >>>> guard issue only affected Linux. >>> Yes, that's what I eventually concluded. I put the fix in >>> os::Posix::get_initial_stack_size() in os_posix.cpp, and only did >>> the page aligning, not add the guard page. That way all Posix ports >>> are fixed in one place. It seems to be working. >>>> >>>>> BTW, did you know java users can specify the size of the a new >>>>> thread's >>>>> stack: >>>> >>>> Yes I mentioned that in another reply - wondering whether we >>>> suitably check and aligned such requests. >>> No we don't. Below I mentioned I was able to trigger the assert with >>> a 257k stack size. I guess I wasn't clear that I did that from Java. >>> I have a new test to add that will be testing this, plus the >>> 9007199254740991 stack size (which fails to create the thread with >>> an OOME, but that's acceptable). The fix I mention above in >>> os::Posix::get_initial_stack_size() takes care of this issue also. >> Rounding up triggers a new assert, this time on 32-bit x86 and arm. >> >> I should have clarified it's 9007199254740991 "K", which is >> 9223372036854774784. Unfortunately on 32bit systems that is asserting >> with EINVAL. I think we need to do a better job of dealing with >> 32-bit size_t values: >> >> jlong java_lang_Thread::stackSize(oop java_thread) { >> if (_stackSize_offset > 0) { >> return java_thread->long_field(_stackSize_offset); >> } else { >> return 0; >> } >> } >> >> jlong size = >> java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); >> // Allocate the C++ Thread structure and create the native >> thread. The >> // stack size retrieved from java is signed, but the >> constructor takes >> // size_t (an unsigned type), so avoid passing negative values >> which would >> // result in really large stacks. >> size_t sz = size > 0 ? (size_t) size : 0; >> native_thread = new JavaThread(&thread_entry, sz); >> >> 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit MAX_INT), >> which is 0xfffffc00 when cast to a size_t on a 32-bit system (close >> to 32-bit MAX_UINT). Round it up to the 4k page size and you get 0, >> which I guess pthread_attr_setstacksize() doesn't like. So I think >> more processing of the size is needed here. Maybe in >> os::create_thread() we should check for 0 after rounding up, and >> subtract the os page size if it is 0. However, I think we should also >> avoid truncating on 32-bit to what is basically some random number. >> Maybe if "size" (a jlong) is greater than UINT_MAX, then just set >> "sz" (a size_t) it to UINT_MAX. >> > Ok, I think I have this all worked out now. I've added fixes for > unaligned stack sizes, 32-bit truncating of stack size, and the > "aligning up to 0" problem. I also added a test. Here's the latest > webrev: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot > > Here's what's changed since webrev.01: > > os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up > the stack size to be paged aligned. This fixes issues on Mac OS X > (other platforms seem to be immune to this). Then check if the size is > zero after rounding up to the page size. Subtract the page size in > this case to produce the maximum stack size allowed. Surprisingly I > got no complaint from gcc for subtracting from an unsigned value that > is known to be 0. > > os_linux.cpp: In os::create_thread(), I also check here to make sure > the size is not 0 after adding the guard page and aligning up, and > subtract the os page size if it is 0. > > jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is > greater than UINT_MAX, then I set the size to UINT_MAX. Note it will > later be rounded up to 0 in os::Posix::get_initial_stack_size(), which > will result in subtracting the os page size to get the actual maximum > allowed stack size. > > TooSmallStackSize.java: added test case for unaligned stack sizes. > > TestThreadStackSizes.java: New test. Creates new threads with every > size up to 320k in 1k increments. Then creates threads with a few > other sizes that can be problematic. > > thanks, > > Chris >> Chris >> >> >>>> >>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>> long stackSize) { >>>>> init(group, target, name, stackSize); >>>>> } >>>>> >>>>> Fortunately we already force the stackSize to be at least >>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>> assert by creating a thread with size 257k. >>>> >>>> Note this means that OSX stack logic is broken because it will >>>> currently be silently failing due to EINVAL! >>> Yes, that is correct. >>> >>> Chris >>>> >>>>> I'll get another webrev out once I've made the needed fixes. I >>>>> also have >>>>> a new test I'd like to add. >>>> >>>> Ok. >>>> >>>> Thanks, >>>> David >>>> >>>>> Chris >>>>> >>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>> Ok, time for a new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>> >>>>>> >>>>>> The only thing that has changed since the first webrev are the >>>>>> asserts >>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>> discuss >>>>>> already: >>>>>> >>>>>> - The assert should never happen due to the stack size being too >>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>> - The assert should never happen due to an unaligned stack size >>>>>> because we always align it to the page size. >>>>>> - Any assert would therefore be a VM bug and not due to user error. >>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>> that is >>>>>> too small, hotspot will already detect this. If the user specifies a >>>>>> stack size that is large enough but not page aligned, then we just >>>>>> ignore any error (if the platform doth protest) and the user gets a >>>>>> main thread with a stack size set to whatever the OS default is. >>>>>> >>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>> figure >>>>>> getting agreement on the changes first would be best before I bog >>>>>> down >>>>>> our testing resources. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>> >>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review the following: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>> >>>>>>> >>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>> code >>>>>>> was failing: >>>>>>> >>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>> >>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k should >>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>> result is pthread_attr_setstacksize() had no impact on the thread's >>>>>>> stack size, and we ended up with the platform default of 8mb. >>>>>>> The fix >>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>> computing their new values: >>>>>>> >>>>>>> _java_thread_min_stack_allowed >>>>>>> _compiler_thread_min_stack_allowed >>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>> >>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>> need to >>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>> PTHREAD_STACK_MIN >>>>>>> #defined, and this needs to be done before including OS header >>>>>>> files. >>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with this >>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>> >>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>> it. It >>>>>>> was being applied before any adjustments to the stack sizes had >>>>>>> been >>>>>>> made (rounding and adding red, yellow, and shadow zones). This mean >>>>>>> the stack ended up being larger than necessary. With the above >>>>>>> fix in >>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>> sizes, >>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>> turned up. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>> >>>>>> >>>>> >>> >> > > From daniel.daugherty at oracle.com Tue Mar 21 19:53:05 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 21 Mar 2017 13:53:05 -0600 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: <3c59ddcc-b4ff-2ede-61e1-a41424c5fb37@oracle.com> On 3/20/17 6:59 PM, David Holmes wrote: > Hi Chris, > > > > Getting closer :) > > Have you seen pthread_attr_setstacksize return EINVAL on very large > values? Or does pthread_create just give a ENOMEM? > > On 21/03/2017 9:29 AM, Chris Plummer wrote: >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >> >> Here's what's changed since webrev.01: >> >> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up the >> stack size to be paged aligned. This fixes issues on Mac OS X (other >> platforms seem to be immune to this). Then check if the size is zero >> after rounding up to the page size. Subtract the page size in this case >> to produce the maximum stack size allowed. Surprisingly I got no >> complaint from gcc for subtracting from an unsigned value that is known >> to be 0. > > I'm a little surprised at that too. :) Chris, would you mind checking with Kim Barrett on the advisability of doing it this way? Dan > >> os_linux.cpp: In os::create_thread(), I also check here to make sure the >> size is not 0 after adding the guard page and aligning up, and subtract >> the os page size if it is 0. > > What if adding the guard page and rounding up causes overflow so that > we get a very small positive stack size? > >> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater >> than UINT_MAX, then I set the size to UINT_MAX. Note it will later be >> rounded up to 0 in os::Posix::get_initial_stack_size(), which will >> result in subtracting the os page size to get the actual maximum allowed >> stack size. > > Good catch! > > Nit: "-Avoid" -> "- Avoid" > >> >> TooSmallStackSize.java: added test case for unaligned stack sizes. > > Ok. > >> TestThreadStackSizes.java: New test. Creates new threads with every size >> up to 320k in 1k increments. Then creates threads with a few other sizes >> that can be problematic. > > So this test never actually fails as long as the VM doesn't crash - is > that right? > > 27 * @modules java.base/jdk.internal.misc > 28 * @library /test/lib > > The above are not needed by this test. > > Thanks, > David > >> thanks, >> >> Chris >>> Chris >>> >>> >>>>> >>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>> long stackSize) { >>>>>> init(group, target, name, stackSize); >>>>>> } >>>>>> >>>>>> Fortunately we already force the stackSize to be at least >>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>> assert by creating a thread with size 257k. >>>>> >>>>> Note this means that OSX stack logic is broken because it will >>>>> currently be silently failing due to EINVAL! >>>> Yes, that is correct. >>>> >>>> Chris >>>>> >>>>>> I'll get another webrev out once I've made the needed fixes. I also >>>>>> have >>>>>> a new test I'd like to add. >>>>> >>>>> Ok. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Chris >>>>>> >>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>> Ok, time for a new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>> >>>>>>> >>>>>>> >>>>>>> The only thing that has changed since the first webrev are the >>>>>>> asserts >>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>> discuss >>>>>>> already: >>>>>>> >>>>>>> - The assert should never happen due to the stack size being too >>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>> because we always align it to the page size. >>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>> error. >>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>> that is >>>>>>> too small, hotspot will already detect this. If the user >>>>>>> specifies a >>>>>>> stack size that is large enough but not page aligned, then we just >>>>>>> ignore any error (if the platform doth protest) and the user gets a >>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>> >>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>> figure >>>>>>> getting agreement on the changes first would be best before I bog >>>>>>> down >>>>>>> our testing resources. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the following: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>>> code >>>>>>>> was failing: >>>>>>>> >>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>> >>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>> should >>>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>> thread's >>>>>>>> stack size, and we ended up with the platform default of 8mb. The >>>>>>>> fix >>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>>> computing their new values: >>>>>>>> >>>>>>>> _java_thread_min_stack_allowed >>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>> >>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>> need to >>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>> PTHREAD_STACK_MIN >>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>> files. >>>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>> this >>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>> >>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>> it. It >>>>>>>> was being applied before any adjustments to the stack sizes had >>>>>>>> been >>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>> mean >>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>> fix in >>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>> sizes, >>>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>> turned up. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>> >>>>>>> >>>>>> >>>> >>> >> > From chris.plummer at oracle.com Tue Mar 21 20:53:19 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 21 Mar 2017 13:53:19 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <3c59ddcc-b4ff-2ede-61e1-a41424c5fb37@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <3c59ddcc-b4ff-2ede-61e1-a41424c5fb37@oracle.com> Message-ID: <847101b2-7b03-adbc-a5a8-7cc32bc75e32@oracle.com> On 3/21/17 12:53 PM, Daniel D. Daugherty wrote: > On 3/20/17 6:59 PM, David Holmes wrote: >> Hi Chris, >> >> >> >> Getting closer :) >> >> Have you seen pthread_attr_setstacksize return EINVAL on very large >> values? Or does pthread_create just give a ENOMEM? >> >> On 21/03/2017 9:29 AM, Chris Plummer wrote: >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>> >>> Here's what's changed since webrev.01: >>> >>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up >>> the >>> stack size to be paged aligned. This fixes issues on Mac OS X (other >>> platforms seem to be immune to this). Then check if the size is zero >>> after rounding up to the page size. Subtract the page size in this case >>> to produce the maximum stack size allowed. Surprisingly I got no >>> complaint from gcc for subtracting from an unsigned value that is known >>> to be 0. >> >> I'm a little surprised at that too. :) > > Chris, would you mind checking with Kim Barrett on the > advisability of doing it this way? Ok. Chris > > Dan > > >> >>> os_linux.cpp: In os::create_thread(), I also check here to make sure >>> the >>> size is not 0 after adding the guard page and aligning up, and subtract >>> the os page size if it is 0. >> >> What if adding the guard page and rounding up causes overflow so that >> we get a very small positive stack size? >> >>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is greater >>> than UINT_MAX, then I set the size to UINT_MAX. Note it will later be >>> rounded up to 0 in os::Posix::get_initial_stack_size(), which will >>> result in subtracting the os page size to get the actual maximum >>> allowed >>> stack size. >> >> Good catch! >> >> Nit: "-Avoid" -> "- Avoid" >> >>> >>> TooSmallStackSize.java: added test case for unaligned stack sizes. >> >> Ok. >> >>> TestThreadStackSizes.java: New test. Creates new threads with every >>> size >>> up to 320k in 1k increments. Then creates threads with a few other >>> sizes >>> that can be problematic. >> >> So this test never actually fails as long as the VM doesn't crash - >> is that right? >> >> 27 * @modules java.base/jdk.internal.misc >> 28 * @library /test/lib >> >> The above are not needed by this test. >> >> Thanks, >> David >> >>> thanks, >>> >>> Chris >>>> Chris >>>> >>>> >>>>>> >>>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>>> long stackSize) { >>>>>>> init(group, target, name, stackSize); >>>>>>> } >>>>>>> >>>>>>> Fortunately we already force the stackSize to be at least >>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>>> assert by creating a thread with size 257k. >>>>>> >>>>>> Note this means that OSX stack logic is broken because it will >>>>>> currently be silently failing due to EINVAL! >>>>> Yes, that is correct. >>>>> >>>>> Chris >>>>>> >>>>>>> I'll get another webrev out once I've made the needed fixes. I also >>>>>>> have >>>>>>> a new test I'd like to add. >>>>>> >>>>>> Ok. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>> Ok, time for a new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The only thing that has changed since the first webrev are the >>>>>>>> asserts >>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>>> discuss >>>>>>>> already: >>>>>>>> >>>>>>>> - The assert should never happen due to the stack size being too >>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>>> because we always align it to the page size. >>>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>>> error. >>>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>>> that is >>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>> specifies a >>>>>>>> stack size that is large enough but not page aligned, then we just >>>>>>>> ignore any error (if the platform doth protest) and the user >>>>>>>> gets a >>>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>>> >>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>>> figure >>>>>>>> getting agreement on the changes first would be best before I bog >>>>>>>> down >>>>>>>> our testing resources. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review the following: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> While working on 8175342 I noticed our stack size on xgene was >>>>>>>>> 8mb >>>>>>>>> even though I was specifying -Xss72k. It turns out the following >>>>>>>>> code >>>>>>>>> was failing: >>>>>>>>> >>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>> >>>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>>> should >>>>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>> thread's >>>>>>>>> stack size, and we ended up with the platform default of 8mb. The >>>>>>>>> fix >>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>>>> computing their new values: >>>>>>>>> >>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>> >>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>>> need to >>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>>> files. >>>>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>>> this >>>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>>> >>>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>>> it. It >>>>>>>>> was being applied before any adjustments to the stack sizes >>>>>>>>> had been >>>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>>> mean >>>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>>> fix in >>>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>>> minimum stack sizes. If for any reason one of those stack >>>>>>>>> sizes is >>>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>>> sizes, >>>>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>>> turned up. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> >> > From chris.plummer at oracle.com Tue Mar 21 21:20:00 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 21 Mar 2017 14:20:00 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: <04d47957-77df-642f-8d96-1584725eea38@oracle.com> On 3/21/17 12:51 PM, Daniel D. Daugherty wrote: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot > > src/os/aix/vm/os_aix.cpp > No comments. > > src/os/bsd/vm/os_bsd.cpp > No comments. > > src/os/linux/vm/os_linux.cpp > L729: // In that cast just subract the page size to get the > maximum possible stack size. > Typo: 'cast' -> 'case' > Typo: 'subract' -> 'subtract' (Thomas also commented on it) > > src/os/posix/vm/os_posix.cpp > L263: // aligning up could have resulted in the size being 0. In > that case just subract the > Nit: 'aligning' -> 'Aligning' (since it's a sentence) > Typo: 'subract' -> 'subtract' > > src/os/solaris/vm/os_solaris.cpp > No comments. > > src/share/vm/prims/jvm.cpp > L2812: // -Avoid truncating on 32-bit platforms if size is > greater than UINT_MAX > Nit: needs a period at the end like L2813. > > test/runtime/Thread/TooSmallStackSize.java > No comments. > > test/runtime/Thread/TestThreadStackSizes.java > L26: * @summary Test user threads with various stacks sizes. > Typo?: "stacks sizes" -> "stack sizes" > > L36: super(null, null, "TestThreadStackSizes"+stackSize, > stackSize); > Nit: spaces around the "+". > > L46: TestThreadStackSizes testThreadStackSize = new > TestThreadStackSizes(stackSize); > Nit: extra space before '='. > > So this test makes 326 createThread() calls... how long does > it take to run? > This is from the results page on Mac OS x log for all the tests in runtime/Thread 1 runtime/Thread/CancellableThreadTest.java 7.033 2 runtime/Thread/Fibonacci.java 8.430 3 runtime/Thread/TestThreadDumpMonitorContention.java 34.322 4 runtime/Thread/ThreadPriorities.java 13.064 5 runtime/Thread/TooSmallStackSize.java 10.086 And 32-bit linux-arm: 1 runtime/Thread/CancellableThreadTest.java 9.359 2 runtime/Thread/Fibonacci.java 11.744 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:04.370 4 runtime/Thread/ThreadPriorities.java 18.140 5 runtime/Thread/TooSmallStackSize.java 14.919 And windows-x64: 1 runtime/Thread/CancellableThreadTest.java 8.074 2 runtime/Thread/Fibonacci.java 10.238 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:21.404 4 runtime/Thread/ThreadPriorities.java 23.134 5 runtime/Thread/TooSmallStackSize.java 24.160 > > Thumbs up! I don't need to see another webrev if you choose to > fix these minor typos... They've all been fixed. thanks for the review, Chris > > Dan > > > On 3/20/17 5:29 PM, Chris Plummer wrote: >> On 3/17/17 11:37 PM, Chris Plummer wrote: >>> On 3/17/17 8:17 PM, Chris Plummer wrote: >>>> On 3/17/17 7:01 PM, David Holmes wrote: >>>>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>>>> Looks like this will need some more work since the added asserts are >>>>>> triggering on mac os x (which is the only place we'd currently >>>>>> expect >>>>>> them to assert). >>>>>> >>>>>> The problem is that the code I found that rounds up to the page >>>>>> size is >>>>>> only applied to java threads created by the VM for which the java >>>>>> user >>>>>> specified no stack size. The VM and Compiler thread sizes are not >>>>>> rounded. The failure I saw was with >>>>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>>> when is >>>>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit the >>>>>> assert >>>>>> with an EINVAL. The size is not aligned, but it could also be >>>>>> complaining because it is too big. I haven't tried aligning it >>>>>> yet to see. >>>>>> >>>>>> On Linux we do the following: >>>>>> >>>>>> stack_size = align_size_up(stack_size + >>>>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>>>> >>>>>> We don't do this on BSD. I think the same on BSD would solve this >>>>>> problem. I'm not sure about adding the guard size. I'll need to >>>>>> see if >>>>>> mac os x has the same pthread bug as linux does. >>>>> >>>>> At this stage I would only deal with alignment issues. IIRC the >>>>> guard issue only affected Linux. >>>> Yes, that's what I eventually concluded. I put the fix in >>>> os::Posix::get_initial_stack_size() in os_posix.cpp, and only did >>>> the page aligning, not add the guard page. That way all Posix ports >>>> are fixed in one place. It seems to be working. >>>>> >>>>>> BTW, did you know java users can specify the size of the a new >>>>>> thread's >>>>>> stack: >>>>> >>>>> Yes I mentioned that in another reply - wondering whether we >>>>> suitably check and aligned such requests. >>>> No we don't. Below I mentioned I was able to trigger the assert >>>> with a 257k stack size. I guess I wasn't clear that I did that from >>>> Java. I have a new test to add that will be testing this, plus the >>>> 9007199254740991 stack size (which fails to create the thread with >>>> an OOME, but that's acceptable). The fix I mention above in >>>> os::Posix::get_initial_stack_size() takes care of this issue also. >>> Rounding up triggers a new assert, this time on 32-bit x86 and arm. >>> >>> I should have clarified it's 9007199254740991 "K", which is >>> 9223372036854774784. Unfortunately on 32bit systems that is >>> asserting with EINVAL. I think we need to do a better job of dealing >>> with 32-bit size_t values: >>> >>> jlong java_lang_Thread::stackSize(oop java_thread) { >>> if (_stackSize_offset > 0) { >>> return java_thread->long_field(_stackSize_offset); >>> } else { >>> return 0; >>> } >>> } >>> >>> jlong size = >>> java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); >>> // Allocate the C++ Thread structure and create the native >>> thread. The >>> // stack size retrieved from java is signed, but the >>> constructor takes >>> // size_t (an unsigned type), so avoid passing negative values >>> which would >>> // result in really large stacks. >>> size_t sz = size > 0 ? (size_t) size : 0; >>> native_thread = new JavaThread(&thread_entry, sz); >>> >>> 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit MAX_INT), >>> which is 0xfffffc00 when cast to a size_t on a 32-bit system (close >>> to 32-bit MAX_UINT). Round it up to the 4k page size and you get 0, >>> which I guess pthread_attr_setstacksize() doesn't like. So I think >>> more processing of the size is needed here. Maybe in >>> os::create_thread() we should check for 0 after rounding up, and >>> subtract the os page size if it is 0. However, I think we should >>> also avoid truncating on 32-bit to what is basically some random >>> number. Maybe if "size" (a jlong) is greater than UINT_MAX, then >>> just set "sz" (a size_t) it to UINT_MAX. >>> >> Ok, I think I have this all worked out now. I've added fixes for >> unaligned stack sizes, 32-bit truncating of stack size, and the >> "aligning up to 0" problem. I also added a test. Here's the latest >> webrev: >> >> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >> >> Here's what's changed since webrev.01: >> >> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up >> the stack size to be paged aligned. This fixes issues on Mac OS X >> (other platforms seem to be immune to this). Then check if the size >> is zero after rounding up to the page size. Subtract the page size in >> this case to produce the maximum stack size allowed. Surprisingly I >> got no complaint from gcc for subtracting from an unsigned value that >> is known to be 0. >> >> os_linux.cpp: In os::create_thread(), I also check here to make sure >> the size is not 0 after adding the guard page and aligning up, and >> subtract the os page size if it is 0. >> >> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is >> greater than UINT_MAX, then I set the size to UINT_MAX. Note it will >> later be rounded up to 0 in os::Posix::get_initial_stack_size(), >> which will result in subtracting the os page size to get the actual >> maximum allowed stack size. >> >> TooSmallStackSize.java: added test case for unaligned stack sizes. >> >> TestThreadStackSizes.java: New test. Creates new threads with every >> size up to 320k in 1k increments. Then creates threads with a few >> other sizes that can be problematic. >> >> thanks, >> >> Chris >>> Chris >>> >>> >>>>> >>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>> long stackSize) { >>>>>> init(group, target, name, stackSize); >>>>>> } >>>>>> >>>>>> Fortunately we already force the stackSize to be at least >>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>> assert by creating a thread with size 257k. >>>>> >>>>> Note this means that OSX stack logic is broken because it will >>>>> currently be silently failing due to EINVAL! >>>> Yes, that is correct. >>>> >>>> Chris >>>>> >>>>>> I'll get another webrev out once I've made the needed fixes. I >>>>>> also have >>>>>> a new test I'd like to add. >>>>> >>>>> Ok. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Chris >>>>>> >>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>> Ok, time for a new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>> >>>>>>> >>>>>>> The only thing that has changed since the first webrev are the >>>>>>> asserts >>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>> discuss >>>>>>> already: >>>>>>> >>>>>>> - The assert should never happen due to the stack size being too >>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>> because we always align it to the page size. >>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>> error. >>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>> that is >>>>>>> too small, hotspot will already detect this. If the user >>>>>>> specifies a >>>>>>> stack size that is large enough but not page aligned, then we just >>>>>>> ignore any error (if the platform doth protest) and the user gets a >>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>> >>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>> figure >>>>>>> getting agreement on the changes first would be best before I >>>>>>> bog down >>>>>>> our testing resources. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the following: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>> >>>>>>>> >>>>>>>> While working on 8175342 I noticed our stack size on xgene was 8mb >>>>>>>> even though I was specifying -Xss72k. It turns out the >>>>>>>> following code >>>>>>>> was failing: >>>>>>>> >>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>> >>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>> should >>>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>> thread's >>>>>>>> stack size, and we ended up with the platform default of 8mb. >>>>>>>> The fix >>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>>> computing their new values: >>>>>>>> >>>>>>>> _java_thread_min_stack_allowed >>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>> >>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>> need to >>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>> PTHREAD_STACK_MIN >>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>> files. >>>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>> this >>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>> >>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>> it. It >>>>>>>> was being applied before any adjustments to the stack sizes had >>>>>>>> been >>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>> mean >>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>> fix in >>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>> minimum stack sizes. If for any reason one of those stack sizes is >>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>> sizes, >>>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>> turned up. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>> >>>>>>> >>>>>> >>>> >>> >> >> > From daniel.daugherty at oracle.com Tue Mar 21 21:57:08 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 21 Mar 2017 15:57:08 -0600 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <04d47957-77df-642f-8d96-1584725eea38@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <04d47957-77df-642f-8d96-1584725eea38@oracle.com> Message-ID: <3c64eedb-155b-a015-089d-e7bb9afffacf@oracle.com> On 3/21/17 3:20 PM, Chris Plummer wrote: > On 3/21/17 12:51 PM, Daniel D. Daugherty wrote: >> > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >> >> src/os/aix/vm/os_aix.cpp >> No comments. >> >> src/os/bsd/vm/os_bsd.cpp >> No comments. >> >> src/os/linux/vm/os_linux.cpp >> L729: // In that cast just subract the page size to get the >> maximum possible stack size. >> Typo: 'cast' -> 'case' >> Typo: 'subract' -> 'subtract' (Thomas also commented on it) >> >> src/os/posix/vm/os_posix.cpp >> L263: // aligning up could have resulted in the size being 0. >> In that case just subract the >> Nit: 'aligning' -> 'Aligning' (since it's a sentence) >> Typo: 'subract' -> 'subtract' >> >> src/os/solaris/vm/os_solaris.cpp >> No comments. >> >> src/share/vm/prims/jvm.cpp >> L2812: // -Avoid truncating on 32-bit platforms if size is >> greater than UINT_MAX >> Nit: needs a period at the end like L2813. >> >> test/runtime/Thread/TooSmallStackSize.java >> No comments. >> >> test/runtime/Thread/TestThreadStackSizes.java >> L26: * @summary Test user threads with various stacks sizes. >> Typo?: "stacks sizes" -> "stack sizes" >> >> L36: super(null, null, "TestThreadStackSizes"+stackSize, >> stackSize); >> Nit: spaces around the "+". >> >> L46: TestThreadStackSizes testThreadStackSize = new >> TestThreadStackSizes(stackSize); >> Nit: extra space before '='. >> >> So this test makes 326 createThread() calls... how long does >> it take to run? >> > This is from the results page on Mac OS x log for all the tests in > runtime/Thread > > 1 runtime/Thread/CancellableThreadTest.java 7.033 > 2 runtime/Thread/Fibonacci.java 8.430 > 3 runtime/Thread/TestThreadDumpMonitorContention.java 34.322 > 4 runtime/Thread/ThreadPriorities.java 13.064 > 5 runtime/Thread/TooSmallStackSize.java 10.086 > > And 32-bit linux-arm: > > 1 runtime/Thread/CancellableThreadTest.java 9.359 > 2 runtime/Thread/Fibonacci.java 11.744 > 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:04.370 > 4 runtime/Thread/ThreadPriorities.java 18.140 > 5 runtime/Thread/TooSmallStackSize.java 14.919 > > And windows-x64: > > 1 runtime/Thread/CancellableThreadTest.java 8.074 > 2 runtime/Thread/Fibonacci.java 10.238 > 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:21.404 > 4 runtime/Thread/ThreadPriorities.java 23.134 > 5 runtime/Thread/TooSmallStackSize.java 24.160 Maybe I'm missing it, but I don't see results for the new test (TestThreadStackSizes.java)... Dan > >> >> Thumbs up! I don't need to see another webrev if you choose to >> fix these minor typos... > They've all been fixed. > > thanks for the review, > > Chris > >> >> Dan >> >> >> On 3/20/17 5:29 PM, Chris Plummer wrote: >>> On 3/17/17 11:37 PM, Chris Plummer wrote: >>>> On 3/17/17 8:17 PM, Chris Plummer wrote: >>>>> On 3/17/17 7:01 PM, David Holmes wrote: >>>>>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>>>>> Looks like this will need some more work since the added asserts >>>>>>> are >>>>>>> triggering on mac os x (which is the only place we'd currently >>>>>>> expect >>>>>>> them to assert). >>>>>>> >>>>>>> The problem is that the code I found that rounds up to the page >>>>>>> size is >>>>>>> only applied to java threads created by the VM for which the >>>>>>> java user >>>>>>> specified no stack size. The VM and Compiler thread sizes are not >>>>>>> rounded. The failure I saw was with >>>>>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>>>> when is >>>>>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit >>>>>>> the assert >>>>>>> with an EINVAL. The size is not aligned, but it could also be >>>>>>> complaining because it is too big. I haven't tried aligning it >>>>>>> yet to see. >>>>>>> >>>>>>> On Linux we do the following: >>>>>>> >>>>>>> stack_size = align_size_up(stack_size + >>>>>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>>>>> >>>>>>> We don't do this on BSD. I think the same on BSD would solve this >>>>>>> problem. I'm not sure about adding the guard size. I'll need to >>>>>>> see if >>>>>>> mac os x has the same pthread bug as linux does. >>>>>> >>>>>> At this stage I would only deal with alignment issues. IIRC the >>>>>> guard issue only affected Linux. >>>>> Yes, that's what I eventually concluded. I put the fix in >>>>> os::Posix::get_initial_stack_size() in os_posix.cpp, and only did >>>>> the page aligning, not add the guard page. That way all Posix >>>>> ports are fixed in one place. It seems to be working. >>>>>> >>>>>>> BTW, did you know java users can specify the size of the a new >>>>>>> thread's >>>>>>> stack: >>>>>> >>>>>> Yes I mentioned that in another reply - wondering whether we >>>>>> suitably check and aligned such requests. >>>>> No we don't. Below I mentioned I was able to trigger the assert >>>>> with a 257k stack size. I guess I wasn't clear that I did that >>>>> from Java. I have a new test to add that will be testing this, >>>>> plus the 9007199254740991 stack size (which fails to create the >>>>> thread with an OOME, but that's acceptable). The fix I mention >>>>> above in os::Posix::get_initial_stack_size() takes care of this >>>>> issue also. >>>> Rounding up triggers a new assert, this time on 32-bit x86 and arm. >>>> >>>> I should have clarified it's 9007199254740991 "K", which is >>>> 9223372036854774784. Unfortunately on 32bit systems that is >>>> asserting with EINVAL. I think we need to do a better job of >>>> dealing with 32-bit size_t values: >>>> >>>> jlong java_lang_Thread::stackSize(oop java_thread) { >>>> if (_stackSize_offset > 0) { >>>> return java_thread->long_field(_stackSize_offset); >>>> } else { >>>> return 0; >>>> } >>>> } >>>> >>>> jlong size = >>>> java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); >>>> // Allocate the C++ Thread structure and create the native >>>> thread. The >>>> // stack size retrieved from java is signed, but the >>>> constructor takes >>>> // size_t (an unsigned type), so avoid passing negative >>>> values which would >>>> // result in really large stacks. >>>> size_t sz = size > 0 ? (size_t) size : 0; >>>> native_thread = new JavaThread(&thread_entry, sz); >>>> >>>> 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit >>>> MAX_INT), which is 0xfffffc00 when cast to a size_t on a 32-bit >>>> system (close to 32-bit MAX_UINT). Round it up to the 4k page size >>>> and you get 0, which I guess pthread_attr_setstacksize() doesn't >>>> like. So I think more processing of the size is needed here. Maybe >>>> in os::create_thread() we should check for 0 after rounding up, and >>>> subtract the os page size if it is 0. However, I think we should >>>> also avoid truncating on 32-bit to what is basically some random >>>> number. Maybe if "size" (a jlong) is greater than UINT_MAX, then >>>> just set "sz" (a size_t) it to UINT_MAX. >>>> >>> Ok, I think I have this all worked out now. I've added fixes for >>> unaligned stack sizes, 32-bit truncating of stack size, and the >>> "aligning up to 0" problem. I also added a test. Here's the latest >>> webrev: >>> >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>> >>> Here's what's changed since webrev.01: >>> >>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round up >>> the stack size to be paged aligned. This fixes issues on Mac OS X >>> (other platforms seem to be immune to this). Then check if the size >>> is zero after rounding up to the page size. Subtract the page size >>> in this case to produce the maximum stack size allowed. Surprisingly >>> I got no complaint from gcc for subtracting from an unsigned value >>> that is known to be 0. >>> >>> os_linux.cpp: In os::create_thread(), I also check here to make sure >>> the size is not 0 after adding the guard page and aligning up, and >>> subtract the os page size if it is 0. >>> >>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is >>> greater than UINT_MAX, then I set the size to UINT_MAX. Note it will >>> later be rounded up to 0 in os::Posix::get_initial_stack_size(), >>> which will result in subtracting the os page size to get the actual >>> maximum allowed stack size. >>> >>> TooSmallStackSize.java: added test case for unaligned stack sizes. >>> >>> TestThreadStackSizes.java: New test. Creates new threads with every >>> size up to 320k in 1k increments. Then creates threads with a few >>> other sizes that can be problematic. >>> >>> thanks, >>> >>> Chris >>>> Chris >>>> >>>> >>>>>> >>>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>>> long stackSize) { >>>>>>> init(group, target, name, stackSize); >>>>>>> } >>>>>>> >>>>>>> Fortunately we already force the stackSize to be at least >>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>>> assert by creating a thread with size 257k. >>>>>> >>>>>> Note this means that OSX stack logic is broken because it will >>>>>> currently be silently failing due to EINVAL! >>>>> Yes, that is correct. >>>>> >>>>> Chris >>>>>> >>>>>>> I'll get another webrev out once I've made the needed fixes. I >>>>>>> also have >>>>>>> a new test I'd like to add. >>>>>> >>>>>> Ok. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Chris >>>>>>> >>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>> Ok, time for a new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>> >>>>>>>> >>>>>>>> The only thing that has changed since the first webrev are the >>>>>>>> asserts >>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>>> discuss >>>>>>>> already: >>>>>>>> >>>>>>>> - The assert should never happen due to the stack size being too >>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>>> because we always align it to the page size. >>>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>>> error. >>>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>>> that is >>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>> specifies a >>>>>>>> stack size that is large enough but not page aligned, then we just >>>>>>>> ignore any error (if the platform doth protest) and the user >>>>>>>> gets a >>>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>>> >>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), but >>>>>>>> figure >>>>>>>> getting agreement on the changes first would be best before I >>>>>>>> bog down >>>>>>>> our testing resources. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review the following: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>> >>>>>>>>> >>>>>>>>> While working on 8175342 I noticed our stack size on xgene was >>>>>>>>> 8mb >>>>>>>>> even though I was specifying -Xss72k. It turns out the >>>>>>>>> following code >>>>>>>>> was failing: >>>>>>>>> >>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>> >>>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>>> should >>>>>>>>> be fine, pthreads on this platform requires the stack be at least >>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>> thread's >>>>>>>>> stack size, and we ended up with the platform default of 8mb. >>>>>>>>> The fix >>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN after >>>>>>>>> computing their new values: >>>>>>>>> >>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>> >>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>>> need to >>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>> #defined, and this needs to be done before including OS header >>>>>>>>> files. >>>>>>>>> I noticed that on solaris we were using thr_min_stack() elsewhere >>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same with >>>>>>>>> this >>>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>>> >>>>>>>>> And speaking of the existing use of thr_min_stack(), I deleted >>>>>>>>> it. It >>>>>>>>> was being applied before any adjustments to the stack sizes >>>>>>>>> had been >>>>>>>>> made (rounding and adding red, yellow, and shadow zones). This >>>>>>>>> mean >>>>>>>>> the stack ended up being larger than necessary. With the above >>>>>>>>> fix in >>>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>>> minimum stack sizes. If for any reason one of those stack >>>>>>>>> sizes is >>>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>>> sizes, >>>>>>>>> not apply thr_min_stack() to the initial stack sizes. However, it >>>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>>> turned up. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>> >>> >> > From chris.plummer at oracle.com Tue Mar 21 22:08:15 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 21 Mar 2017 15:08:15 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <3c64eedb-155b-a015-089d-e7bb9afffacf@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <04d47957-77df-642f-8d96-1584725eea38@oracle.com> <3c64eedb-155b-a015-089d-e7bb9afffacf@oracle.com> Message-ID: <3c0031eb-d1b5-77d9-1a40-895a54d56048@oracle.com> On 3/21/17 2:57 PM, Daniel D. Daugherty wrote: > On 3/21/17 3:20 PM, Chris Plummer wrote: >> On 3/21/17 12:51 PM, Daniel D. Daugherty wrote: >>> > >>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>> >>> src/os/aix/vm/os_aix.cpp >>> No comments. >>> >>> src/os/bsd/vm/os_bsd.cpp >>> No comments. >>> >>> src/os/linux/vm/os_linux.cpp >>> L729: // In that cast just subract the page size to get the >>> maximum possible stack size. >>> Typo: 'cast' -> 'case' >>> Typo: 'subract' -> 'subtract' (Thomas also commented on it) >>> >>> src/os/posix/vm/os_posix.cpp >>> L263: // aligning up could have resulted in the size being 0. >>> In that case just subract the >>> Nit: 'aligning' -> 'Aligning' (since it's a sentence) >>> Typo: 'subract' -> 'subtract' >>> >>> src/os/solaris/vm/os_solaris.cpp >>> No comments. >>> >>> src/share/vm/prims/jvm.cpp >>> L2812: // -Avoid truncating on 32-bit platforms if size is >>> greater than UINT_MAX >>> Nit: needs a period at the end like L2813. >>> >>> test/runtime/Thread/TooSmallStackSize.java >>> No comments. >>> >>> test/runtime/Thread/TestThreadStackSizes.java >>> L26: * @summary Test user threads with various stacks sizes. >>> Typo?: "stacks sizes" -> "stack sizes" >>> >>> L36: super(null, null, "TestThreadStackSizes"+stackSize, >>> stackSize); >>> Nit: spaces around the "+". >>> >>> L46: TestThreadStackSizes testThreadStackSize = new >>> TestThreadStackSizes(stackSize); >>> Nit: extra space before '='. >>> >>> So this test makes 326 createThread() calls... how long does >>> it take to run? >>> >> This is from the results page on Mac OS x log for all the tests in >> runtime/Thread >> >> 1 runtime/Thread/CancellableThreadTest.java 7.033 >> 2 runtime/Thread/Fibonacci.java 8.430 >> 3 runtime/Thread/TestThreadDumpMonitorContention.java 34.322 >> 4 runtime/Thread/ThreadPriorities.java 13.064 >> 5 runtime/Thread/TooSmallStackSize.java 10.086 >> >> And 32-bit linux-arm: >> >> 1 runtime/Thread/CancellableThreadTest.java 9.359 >> 2 runtime/Thread/Fibonacci.java 11.744 >> 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:04.370 >> 4 runtime/Thread/ThreadPriorities.java 18.140 >> 5 runtime/Thread/TooSmallStackSize.java 14.919 >> >> And windows-x64: >> >> 1 runtime/Thread/CancellableThreadTest.java 8.074 >> 2 runtime/Thread/Fibonacci.java 10.238 >> 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:21.404 >> 4 runtime/Thread/ThreadPriorities.java 23.134 >> 5 runtime/Thread/TooSmallStackSize.java 24.160 > > Maybe I'm missing it, but I don't see results for the new test > (TestThreadStackSizes.java)... Sorry, that was from my attempt to rerun the tests earlier today after a minor tweak. Unfortunately I had temporarily switched to a clean jdk repo, so that's what got tested. Let me run the tests on the right repo this time. Results in a couple of hours probably. Chris > > Dan > > >> >>> >>> Thumbs up! I don't need to see another webrev if you choose to >>> fix these minor typos... >> They've all been fixed. >> >> thanks for the review, >> >> Chris >> >>> >>> Dan >>> >>> >>> On 3/20/17 5:29 PM, Chris Plummer wrote: >>>> On 3/17/17 11:37 PM, Chris Plummer wrote: >>>>> On 3/17/17 8:17 PM, Chris Plummer wrote: >>>>>> On 3/17/17 7:01 PM, David Holmes wrote: >>>>>>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>>>>>> Looks like this will need some more work since the added >>>>>>>> asserts are >>>>>>>> triggering on mac os x (which is the only place we'd currently >>>>>>>> expect >>>>>>>> them to assert). >>>>>>>> >>>>>>>> The problem is that the code I found that rounds up to the page >>>>>>>> size is >>>>>>>> only applied to java threads created by the VM for which the >>>>>>>> java user >>>>>>>> specified no stack size. The VM and Compiler thread sizes are not >>>>>>>> rounded. The failure I saw was with >>>>>>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>>>>> when is >>>>>>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit >>>>>>>> the assert >>>>>>>> with an EINVAL. The size is not aligned, but it could also be >>>>>>>> complaining because it is too big. I haven't tried aligning it >>>>>>>> yet to see. >>>>>>>> >>>>>>>> On Linux we do the following: >>>>>>>> >>>>>>>> stack_size = align_size_up(stack_size + >>>>>>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>>>>>> >>>>>>>> We don't do this on BSD. I think the same on BSD would solve this >>>>>>>> problem. I'm not sure about adding the guard size. I'll need to >>>>>>>> see if >>>>>>>> mac os x has the same pthread bug as linux does. >>>>>>> >>>>>>> At this stage I would only deal with alignment issues. IIRC the >>>>>>> guard issue only affected Linux. >>>>>> Yes, that's what I eventually concluded. I put the fix in >>>>>> os::Posix::get_initial_stack_size() in os_posix.cpp, and only did >>>>>> the page aligning, not add the guard page. That way all Posix >>>>>> ports are fixed in one place. It seems to be working. >>>>>>> >>>>>>>> BTW, did you know java users can specify the size of the a new >>>>>>>> thread's >>>>>>>> stack: >>>>>>> >>>>>>> Yes I mentioned that in another reply - wondering whether we >>>>>>> suitably check and aligned such requests. >>>>>> No we don't. Below I mentioned I was able to trigger the assert >>>>>> with a 257k stack size. I guess I wasn't clear that I did that >>>>>> from Java. I have a new test to add that will be testing this, >>>>>> plus the 9007199254740991 stack size (which fails to create the >>>>>> thread with an OOME, but that's acceptable). The fix I mention >>>>>> above in os::Posix::get_initial_stack_size() takes care of this >>>>>> issue also. >>>>> Rounding up triggers a new assert, this time on 32-bit x86 and arm. >>>>> >>>>> I should have clarified it's 9007199254740991 "K", which is >>>>> 9223372036854774784. Unfortunately on 32bit systems that is >>>>> asserting with EINVAL. I think we need to do a better job of >>>>> dealing with 32-bit size_t values: >>>>> >>>>> jlong java_lang_Thread::stackSize(oop java_thread) { >>>>> if (_stackSize_offset > 0) { >>>>> return java_thread->long_field(_stackSize_offset); >>>>> } else { >>>>> return 0; >>>>> } >>>>> } >>>>> >>>>> jlong size = >>>>> java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); >>>>> // Allocate the C++ Thread structure and create the native >>>>> thread. The >>>>> // stack size retrieved from java is signed, but the >>>>> constructor takes >>>>> // size_t (an unsigned type), so avoid passing negative >>>>> values which would >>>>> // result in really large stacks. >>>>> size_t sz = size > 0 ? (size_t) size : 0; >>>>> native_thread = new JavaThread(&thread_entry, sz); >>>>> >>>>> 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit >>>>> MAX_INT), which is 0xfffffc00 when cast to a size_t on a 32-bit >>>>> system (close to 32-bit MAX_UINT). Round it up to the 4k page size >>>>> and you get 0, which I guess pthread_attr_setstacksize() doesn't >>>>> like. So I think more processing of the size is needed here. Maybe >>>>> in os::create_thread() we should check for 0 after rounding up, >>>>> and subtract the os page size if it is 0. However, I think we >>>>> should also avoid truncating on 32-bit to what is basically some >>>>> random number. Maybe if "size" (a jlong) is greater than UINT_MAX, >>>>> then just set "sz" (a size_t) it to UINT_MAX. >>>>> >>>> Ok, I think I have this all worked out now. I've added fixes for >>>> unaligned stack sizes, 32-bit truncating of stack size, and the >>>> "aligning up to 0" problem. I also added a test. Here's the latest >>>> webrev: >>>> >>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>>> >>>> Here's what's changed since webrev.01: >>>> >>>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round >>>> up the stack size to be paged aligned. This fixes issues on Mac OS >>>> X (other platforms seem to be immune to this). Then check if the >>>> size is zero after rounding up to the page size. Subtract the page >>>> size in this case to produce the maximum stack size allowed. >>>> Surprisingly I got no complaint from gcc for subtracting from an >>>> unsigned value that is known to be 0. >>>> >>>> os_linux.cpp: In os::create_thread(), I also check here to make >>>> sure the size is not 0 after adding the guard page and aligning up, >>>> and subtract the os page size if it is 0. >>>> >>>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is >>>> greater than UINT_MAX, then I set the size to UINT_MAX. Note it >>>> will later be rounded up to 0 in >>>> os::Posix::get_initial_stack_size(), which will result in >>>> subtracting the os page size to get the actual maximum allowed >>>> stack size. >>>> >>>> TooSmallStackSize.java: added test case for unaligned stack sizes. >>>> >>>> TestThreadStackSizes.java: New test. Creates new threads with every >>>> size up to 320k in 1k increments. Then creates threads with a few >>>> other sizes that can be problematic. >>>> >>>> thanks, >>>> >>>> Chris >>>>> Chris >>>>> >>>>> >>>>>>> >>>>>>>> public Thread(ThreadGroup group, Runnable target, String name, >>>>>>>> long stackSize) { >>>>>>>> init(group, target, name, stackSize); >>>>>>>> } >>>>>>>> >>>>>>>> Fortunately we already force the stackSize to be at least >>>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>>> rounding on Mac OS X as noted above, and I was able to trigger the >>>>>>>> assert by creating a thread with size 257k. >>>>>>> >>>>>>> Note this means that OSX stack logic is broken because it will >>>>>>> currently be silently failing due to EINVAL! >>>>>> Yes, that is correct. >>>>>> >>>>>> Chris >>>>>>> >>>>>>>> I'll get another webrev out once I've made the needed fixes. I >>>>>>>> also have >>>>>>>> a new test I'd like to add. >>>>>>> >>>>>>> Ok. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>>> Ok, time for a new webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>>> >>>>>>>>> >>>>>>>>> The only thing that has changed since the first webrev are the >>>>>>>>> asserts >>>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what we >>>>>>>>> discuss >>>>>>>>> already: >>>>>>>>> >>>>>>>>> - The assert should never happen due to the stack size being too >>>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>>>> because we always align it to the page size. >>>>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>>>> error. >>>>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>>>> that is >>>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>>> specifies a >>>>>>>>> stack size that is large enough but not page aligned, then we >>>>>>>>> just >>>>>>>>> ignore any error (if the platform doth protest) and the user >>>>>>>>> gets a >>>>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>>>> >>>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), >>>>>>>>> but figure >>>>>>>>> getting agreement on the changes first would be best before I >>>>>>>>> bog down >>>>>>>>> our testing resources. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> >>>>>>>>> Chris >>>>>>>>> >>>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> Please review the following: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> While working on 8175342 I noticed our stack size on xgene >>>>>>>>>> was 8mb >>>>>>>>>> even though I was specifying -Xss72k. It turns out the >>>>>>>>>> following code >>>>>>>>>> was failing: >>>>>>>>>> >>>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>>> >>>>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>>>> should >>>>>>>>>> be fine, pthreads on this platform requires the stack be at >>>>>>>>>> least >>>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The end >>>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>>> thread's >>>>>>>>>> stack size, and we ended up with the platform default of 8mb. >>>>>>>>>> The fix >>>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN >>>>>>>>>> after >>>>>>>>>> computing their new values: >>>>>>>>>> >>>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>>> >>>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>>>> need to >>>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>>> #defined, and this needs to be done before including OS >>>>>>>>>> header files. >>>>>>>>>> I noticed that on solaris we were using thr_min_stack() >>>>>>>>>> elsewhere >>>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same >>>>>>>>>> with this >>>>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>>>> >>>>>>>>>> And speaking of the existing use of thr_min_stack(), I >>>>>>>>>> deleted it. It >>>>>>>>>> was being applied before any adjustments to the stack sizes >>>>>>>>>> had been >>>>>>>>>> made (rounding and adding red, yellow, and shadow zones). >>>>>>>>>> This mean >>>>>>>>>> the stack ended up being larger than necessary. With the >>>>>>>>>> above fix in >>>>>>>>>> place, we are now applying thr_min_stack() after recomputing the >>>>>>>>>> minimum stack sizes. If for any reason one of those stack >>>>>>>>>> sizes is >>>>>>>>>> now too small, the correct fix is to adjust the initial stack >>>>>>>>>> sizes, >>>>>>>>>> not apply thr_min_stack() to the initial stack sizes. >>>>>>>>>> However, it >>>>>>>>>> looks like no adjustment is needed. I did something close to our >>>>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>>>> turned up. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> >>>>>>>>>> Chris >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>>> >>> >> > From chris.plummer at oracle.com Tue Mar 21 23:13:51 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 21 Mar 2017 16:13:51 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <3c0031eb-d1b5-77d9-1a40-895a54d56048@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <04d47957-77df-642f-8d96-1584725eea38@oracle.com> <3c64eedb-155b-a015-089d-e7bb9afffacf@oracle.com> <3c0031eb-d1b5-77d9-1a40-895a54d56048@oracle.com> Message-ID: On 3/21/17 3:08 PM, Chris Plummer wrote: > On 3/21/17 2:57 PM, Daniel D. Daugherty wrote: >> On 3/21/17 3:20 PM, Chris Plummer wrote: >>> On 3/21/17 12:51 PM, Daniel D. Daugherty wrote: >>>> > >>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>>> >>>> src/os/aix/vm/os_aix.cpp >>>> No comments. >>>> >>>> src/os/bsd/vm/os_bsd.cpp >>>> No comments. >>>> >>>> src/os/linux/vm/os_linux.cpp >>>> L729: // In that cast just subract the page size to get the >>>> maximum possible stack size. >>>> Typo: 'cast' -> 'case' >>>> Typo: 'subract' -> 'subtract' (Thomas also commented on it) >>>> >>>> src/os/posix/vm/os_posix.cpp >>>> L263: // aligning up could have resulted in the size being 0. >>>> In that case just subract the >>>> Nit: 'aligning' -> 'Aligning' (since it's a sentence) >>>> Typo: 'subract' -> 'subtract' >>>> >>>> src/os/solaris/vm/os_solaris.cpp >>>> No comments. >>>> >>>> src/share/vm/prims/jvm.cpp >>>> L2812: // -Avoid truncating on 32-bit platforms if size >>>> is greater than UINT_MAX >>>> Nit: needs a period at the end like L2813. >>>> >>>> test/runtime/Thread/TooSmallStackSize.java >>>> No comments. >>>> >>>> test/runtime/Thread/TestThreadStackSizes.java >>>> L26: * @summary Test user threads with various stacks sizes. >>>> Typo?: "stacks sizes" -> "stack sizes" >>>> >>>> L36: super(null, null, >>>> "TestThreadStackSizes"+stackSize, stackSize); >>>> Nit: spaces around the "+". >>>> >>>> L46: TestThreadStackSizes testThreadStackSize = new >>>> TestThreadStackSizes(stackSize); >>>> Nit: extra space before '='. >>>> >>>> So this test makes 326 createThread() calls... how long does >>>> it take to run? >>>> >>> This is from the results page on Mac OS x log for all the tests in >>> runtime/Thread >>> >>> 1 runtime/Thread/CancellableThreadTest.java 7.033 >>> 2 runtime/Thread/Fibonacci.java 8.430 >>> 3 runtime/Thread/TestThreadDumpMonitorContention.java 34.322 >>> 4 runtime/Thread/ThreadPriorities.java 13.064 >>> 5 runtime/Thread/TooSmallStackSize.java 10.086 >>> >>> And 32-bit linux-arm: >>> >>> 1 runtime/Thread/CancellableThreadTest.java 9.359 >>> 2 runtime/Thread/Fibonacci.java 11.744 >>> 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:04.370 >>> 4 runtime/Thread/ThreadPriorities.java 18.140 >>> 5 runtime/Thread/TooSmallStackSize.java 14.919 >>> >>> And windows-x64: >>> >>> 1 runtime/Thread/CancellableThreadTest.java 8.074 >>> 2 runtime/Thread/Fibonacci.java 10.238 >>> 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:21.404 >>> 4 runtime/Thread/ThreadPriorities.java 23.134 >>> 5 runtime/Thread/TooSmallStackSize.java 24.160 >> >> Maybe I'm missing it, but I don't see results for the new test >> (TestThreadStackSizes.java)... > Sorry, that was from my attempt to rerun the tests earlier today after > a minor tweak. Unfortunately I had temporarily switched to a clean jdk > repo, so that's what got tested. Let me run the tests on the right > repo this time. Results in a couple of hours probably. > Updated numbers: Mac OS X: 1 runtime/Thread/CancellableThreadTest.java 9.111 2 runtime/Thread/Fibonacci.java 10.516 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:06.040 4 runtime/Thread/TestThreadStackSizes.java 9.313 5 runtime/Thread/ThreadPriorities.java 20.633 6 runtime/Thread/TooSmallStackSize.java 16.117 32-bit linux-arm: 1 runtime/Thread/CancellableThreadTest.java 10.654 2 runtime/Thread/Fibonacci.java 13.592 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:05.805 4 runtime/Thread/TestThreadStackSizes.java 8.846 5 runtime/Thread/ThreadPriorities.java 19.052 6 runtime/Thread/TooSmallStackSize.java 15.426 windows-x64: 1 runtime/Thread/CancellableThreadTest.java 8.418 2 runtime/Thread/Fibonacci.java 10.984 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:52.157 4 runtime/Thread/TestThreadStackSizes.java 13.241 5 runtime/Thread/ThreadPriorities.java 29.333 6 runtime/Thread/TooSmallStackSize.java 45.316 Chris > Chris >> >> Dan >> >> >>> >>>> >>>> Thumbs up! I don't need to see another webrev if you choose to >>>> fix these minor typos... >>> They've all been fixed. >>> >>> thanks for the review, >>> >>> Chris >>> >>>> >>>> Dan >>>> >>>> >>>> On 3/20/17 5:29 PM, Chris Plummer wrote: >>>>> On 3/17/17 11:37 PM, Chris Plummer wrote: >>>>>> On 3/17/17 8:17 PM, Chris Plummer wrote: >>>>>>> On 3/17/17 7:01 PM, David Holmes wrote: >>>>>>>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>>>>>>> Looks like this will need some more work since the added >>>>>>>>> asserts are >>>>>>>>> triggering on mac os x (which is the only place we'd currently >>>>>>>>> expect >>>>>>>>> them to assert). >>>>>>>>> >>>>>>>>> The problem is that the code I found that rounds up to the >>>>>>>>> page size is >>>>>>>>> only applied to java threads created by the VM for which the >>>>>>>>> java user >>>>>>>>> specified no stack size. The VM and Compiler thread sizes are not >>>>>>>>> rounded. The failure I saw was with >>>>>>>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>>>>>> when is >>>>>>>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit >>>>>>>>> the assert >>>>>>>>> with an EINVAL. The size is not aligned, but it could also be >>>>>>>>> complaining because it is too big. I haven't tried aligning it >>>>>>>>> yet to see. >>>>>>>>> >>>>>>>>> On Linux we do the following: >>>>>>>>> >>>>>>>>> stack_size = align_size_up(stack_size + >>>>>>>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>>>>>>> >>>>>>>>> We don't do this on BSD. I think the same on BSD would solve this >>>>>>>>> problem. I'm not sure about adding the guard size. I'll need >>>>>>>>> to see if >>>>>>>>> mac os x has the same pthread bug as linux does. >>>>>>>> >>>>>>>> At this stage I would only deal with alignment issues. IIRC the >>>>>>>> guard issue only affected Linux. >>>>>>> Yes, that's what I eventually concluded. I put the fix in >>>>>>> os::Posix::get_initial_stack_size() in os_posix.cpp, and only >>>>>>> did the page aligning, not add the guard page. That way all >>>>>>> Posix ports are fixed in one place. It seems to be working. >>>>>>>> >>>>>>>>> BTW, did you know java users can specify the size of the a new >>>>>>>>> thread's >>>>>>>>> stack: >>>>>>>> >>>>>>>> Yes I mentioned that in another reply - wondering whether we >>>>>>>> suitably check and aligned such requests. >>>>>>> No we don't. Below I mentioned I was able to trigger the assert >>>>>>> with a 257k stack size. I guess I wasn't clear that I did that >>>>>>> from Java. I have a new test to add that will be testing this, >>>>>>> plus the 9007199254740991 stack size (which fails to create the >>>>>>> thread with an OOME, but that's acceptable). The fix I mention >>>>>>> above in os::Posix::get_initial_stack_size() takes care of this >>>>>>> issue also. >>>>>> Rounding up triggers a new assert, this time on 32-bit x86 and arm. >>>>>> >>>>>> I should have clarified it's 9007199254740991 "K", which is >>>>>> 9223372036854774784. Unfortunately on 32bit systems that is >>>>>> asserting with EINVAL. I think we need to do a better job of >>>>>> dealing with 32-bit size_t values: >>>>>> >>>>>> jlong java_lang_Thread::stackSize(oop java_thread) { >>>>>> if (_stackSize_offset > 0) { >>>>>> return java_thread->long_field(_stackSize_offset); >>>>>> } else { >>>>>> return 0; >>>>>> } >>>>>> } >>>>>> >>>>>> jlong size = >>>>>> java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); >>>>>> // Allocate the C++ Thread structure and create the native >>>>>> thread. The >>>>>> // stack size retrieved from java is signed, but the >>>>>> constructor takes >>>>>> // size_t (an unsigned type), so avoid passing negative >>>>>> values which would >>>>>> // result in really large stacks. >>>>>> size_t sz = size > 0 ? (size_t) size : 0; >>>>>> native_thread = new JavaThread(&thread_entry, sz); >>>>>> >>>>>> 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit >>>>>> MAX_INT), which is 0xfffffc00 when cast to a size_t on a 32-bit >>>>>> system (close to 32-bit MAX_UINT). Round it up to the 4k page >>>>>> size and you get 0, which I guess pthread_attr_setstacksize() >>>>>> doesn't like. So I think more processing of the size is needed >>>>>> here. Maybe in os::create_thread() we should check for 0 after >>>>>> rounding up, and subtract the os page size if it is 0. However, I >>>>>> think we should also avoid truncating on 32-bit to what is >>>>>> basically some random number. Maybe if "size" (a jlong) is >>>>>> greater than UINT_MAX, then just set "sz" (a size_t) it to UINT_MAX. >>>>>> >>>>> Ok, I think I have this all worked out now. I've added fixes for >>>>> unaligned stack sizes, 32-bit truncating of stack size, and the >>>>> "aligning up to 0" problem. I also added a test. Here's the latest >>>>> webrev: >>>>> >>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>>>> >>>>> >>>>> Here's what's changed since webrev.01: >>>>> >>>>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round >>>>> up the stack size to be paged aligned. This fixes issues on Mac OS >>>>> X (other platforms seem to be immune to this). Then check if the >>>>> size is zero after rounding up to the page size. Subtract the page >>>>> size in this case to produce the maximum stack size allowed. >>>>> Surprisingly I got no complaint from gcc for subtracting from an >>>>> unsigned value that is known to be 0. >>>>> >>>>> os_linux.cpp: In os::create_thread(), I also check here to make >>>>> sure the size is not 0 after adding the guard page and aligning >>>>> up, and subtract the os page size if it is 0. >>>>> >>>>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is >>>>> greater than UINT_MAX, then I set the size to UINT_MAX. Note it >>>>> will later be rounded up to 0 in >>>>> os::Posix::get_initial_stack_size(), which will result in >>>>> subtracting the os page size to get the actual maximum allowed >>>>> stack size. >>>>> >>>>> TooSmallStackSize.java: added test case for unaligned stack sizes. >>>>> >>>>> TestThreadStackSizes.java: New test. Creates new threads with >>>>> every size up to 320k in 1k increments. Then creates threads with >>>>> a few other sizes that can be problematic. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>>> Chris >>>>>> >>>>>> >>>>>>>> >>>>>>>>> public Thread(ThreadGroup group, Runnable target, String >>>>>>>>> name, >>>>>>>>> long stackSize) { >>>>>>>>> init(group, target, name, stackSize); >>>>>>>>> } >>>>>>>>> >>>>>>>>> Fortunately we already force the stackSize to be at least >>>>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>>>> rounding on Mac OS X as noted above, and I was able to trigger >>>>>>>>> the >>>>>>>>> assert by creating a thread with size 257k. >>>>>>>> >>>>>>>> Note this means that OSX stack logic is broken because it will >>>>>>>> currently be silently failing due to EINVAL! >>>>>>> Yes, that is correct. >>>>>>> >>>>>>> Chris >>>>>>>> >>>>>>>>> I'll get another webrev out once I've made the needed fixes. I >>>>>>>>> also have >>>>>>>>> a new test I'd like to add. >>>>>>>> >>>>>>>> Ok. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> Chris >>>>>>>>> >>>>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>>>> Ok, time for a new webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The only thing that has changed since the first webrev are >>>>>>>>>> the asserts >>>>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what >>>>>>>>>> we discuss >>>>>>>>>> already: >>>>>>>>>> >>>>>>>>>> - The assert should never happen due to the stack size being >>>>>>>>>> too >>>>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>>>> - The assert should never happen due to an unaligned stack size >>>>>>>>>> because we always align it to the page size. >>>>>>>>>> - Any assert would therefore be a VM bug and not due to user >>>>>>>>>> error. >>>>>>>>>> - No fixing the java launcher. If the user specifies a stack >>>>>>>>>> that is >>>>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>>>> specifies a >>>>>>>>>> stack size that is large enough but not page aligned, then we >>>>>>>>>> just >>>>>>>>>> ignore any error (if the platform doth protest) and the user >>>>>>>>>> gets a >>>>>>>>>> main thread with a stack size set to whatever the OS default is. >>>>>>>>>> >>>>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), >>>>>>>>>> but figure >>>>>>>>>> getting agreement on the changes first would be best before I >>>>>>>>>> bog down >>>>>>>>>> our testing resources. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> >>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> Please review the following: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> While working on 8175342 I noticed our stack size on xgene >>>>>>>>>>> was 8mb >>>>>>>>>>> even though I was specifying -Xss72k. It turns out the >>>>>>>>>>> following code >>>>>>>>>>> was failing: >>>>>>>>>>> >>>>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>>>> >>>>>>>>>>> Although we computed a minimum stack size of 72k, so -Xss72k >>>>>>>>>>> should >>>>>>>>>>> be fine, pthreads on this platform requires the stack be at >>>>>>>>>>> least >>>>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. The >>>>>>>>>>> end >>>>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>>>> thread's >>>>>>>>>>> stack size, and we ended up with the platform default of >>>>>>>>>>> 8mb. The fix >>>>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN >>>>>>>>>>> after >>>>>>>>>>> computing their new values: >>>>>>>>>>> >>>>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>>>> >>>>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. You >>>>>>>>>>> need to >>>>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>>>> #defined, and this needs to be done before including OS >>>>>>>>>>> header files. >>>>>>>>>>> I noticed that on solaris we were using thr_min_stack() >>>>>>>>>>> elsewhere >>>>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same >>>>>>>>>>> with this >>>>>>>>>>> fix. Either way is ugly (the #define or using thr_min_stack()). >>>>>>>>>>> >>>>>>>>>>> And speaking of the existing use of thr_min_stack(), I >>>>>>>>>>> deleted it. It >>>>>>>>>>> was being applied before any adjustments to the stack sizes >>>>>>>>>>> had been >>>>>>>>>>> made (rounding and adding red, yellow, and shadow zones). >>>>>>>>>>> This mean >>>>>>>>>>> the stack ended up being larger than necessary. With the >>>>>>>>>>> above fix in >>>>>>>>>>> place, we are now applying thr_min_stack() after recomputing >>>>>>>>>>> the >>>>>>>>>>> minimum stack sizes. If for any reason one of those stack >>>>>>>>>>> sizes is >>>>>>>>>>> now too small, the correct fix is to adjust the initial >>>>>>>>>>> stack sizes, >>>>>>>>>>> not apply thr_min_stack() to the initial stack sizes. >>>>>>>>>>> However, it >>>>>>>>>>> looks like no adjustment is needed. I did something close to >>>>>>>>>>> our >>>>>>>>>>> nightly testing on all affect platforms, and no new problems >>>>>>>>>>> turned up. >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> >>>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> > From daniel.daugherty at oracle.com Tue Mar 21 23:19:48 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 21 Mar 2017 17:19:48 -0600 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <04d47957-77df-642f-8d96-1584725eea38@oracle.com> <3c64eedb-155b-a015-089d-e7bb9afffacf@oracle.com> <3c0031eb-d1b5-77d9-1a40-895a54d56048@oracle.com> Message-ID: <86d8a8f8-cb68-302a-d93a-9b92de07e378@oracle.com> On 3/21/17 5:13 PM, Chris Plummer wrote: > On 3/21/17 3:08 PM, Chris Plummer wrote: >> On 3/21/17 2:57 PM, Daniel D. Daugherty wrote: >>> On 3/21/17 3:20 PM, Chris Plummer wrote: >>>> On 3/21/17 12:51 PM, Daniel D. Daugherty wrote: >>>>> > >>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>>>> >>>>> >>>>> src/os/aix/vm/os_aix.cpp >>>>> No comments. >>>>> >>>>> src/os/bsd/vm/os_bsd.cpp >>>>> No comments. >>>>> >>>>> src/os/linux/vm/os_linux.cpp >>>>> L729: // In that cast just subract the page size to get the >>>>> maximum possible stack size. >>>>> Typo: 'cast' -> 'case' >>>>> Typo: 'subract' -> 'subtract' (Thomas also commented on it) >>>>> >>>>> src/os/posix/vm/os_posix.cpp >>>>> L263: // aligning up could have resulted in the size being >>>>> 0. In that case just subract the >>>>> Nit: 'aligning' -> 'Aligning' (since it's a sentence) >>>>> Typo: 'subract' -> 'subtract' >>>>> >>>>> src/os/solaris/vm/os_solaris.cpp >>>>> No comments. >>>>> >>>>> src/share/vm/prims/jvm.cpp >>>>> L2812: // -Avoid truncating on 32-bit platforms if size >>>>> is greater than UINT_MAX >>>>> Nit: needs a period at the end like L2813. >>>>> >>>>> test/runtime/Thread/TooSmallStackSize.java >>>>> No comments. >>>>> >>>>> test/runtime/Thread/TestThreadStackSizes.java >>>>> L26: * @summary Test user threads with various stacks sizes. >>>>> Typo?: "stacks sizes" -> "stack sizes" >>>>> >>>>> L36: super(null, null, >>>>> "TestThreadStackSizes"+stackSize, stackSize); >>>>> Nit: spaces around the "+". >>>>> >>>>> L46: TestThreadStackSizes testThreadStackSize = >>>>> new TestThreadStackSizes(stackSize); >>>>> Nit: extra space before '='. >>>>> >>>>> So this test makes 326 createThread() calls... how long does >>>>> it take to run? >>>>> >>>> This is from the results page on Mac OS x log for all the tests in >>>> runtime/Thread >>>> >>>> 1 runtime/Thread/CancellableThreadTest.java 7.033 >>>> 2 runtime/Thread/Fibonacci.java 8.430 >>>> 3 runtime/Thread/TestThreadDumpMonitorContention.java 34.322 >>>> 4 runtime/Thread/ThreadPriorities.java 13.064 >>>> 5 runtime/Thread/TooSmallStackSize.java 10.086 >>>> >>>> And 32-bit linux-arm: >>>> >>>> 1 runtime/Thread/CancellableThreadTest.java 9.359 >>>> 2 runtime/Thread/Fibonacci.java 11.744 >>>> 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:04.370 >>>> 4 runtime/Thread/ThreadPriorities.java 18.140 >>>> 5 runtime/Thread/TooSmallStackSize.java 14.919 >>>> >>>> And windows-x64: >>>> >>>> 1 runtime/Thread/CancellableThreadTest.java 8.074 >>>> 2 runtime/Thread/Fibonacci.java 10.238 >>>> 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:21.404 >>>> 4 runtime/Thread/ThreadPriorities.java 23.134 >>>> 5 runtime/Thread/TooSmallStackSize.java 24.160 >>> > >>> Maybe I'm missing it, but I don't see results for the new test >>> (TestThreadStackSizes.java)... >> Sorry, that was from my attempt to rerun the tests earlier today >> after a minor tweak. Unfortunately I had temporarily switched to a >> clean jdk repo, so that's what got tested. Let me run the tests on >> the right repo this time. Results in a couple of hours probably. >> > Updated numbers: > > Mac OS X: > > 1 runtime/Thread/CancellableThreadTest.java 9.111 > 2 runtime/Thread/Fibonacci.java 10.516 > 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:06.040 > 4 runtime/Thread/TestThreadStackSizes.java 9.313 > 5 runtime/Thread/ThreadPriorities.java 20.633 > 6 runtime/Thread/TooSmallStackSize.java 16.117 > > 32-bit linux-arm: > > 1 runtime/Thread/CancellableThreadTest.java 10.654 > 2 runtime/Thread/Fibonacci.java 13.592 > 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:05.805 > 4 runtime/Thread/TestThreadStackSizes.java 8.846 > 5 runtime/Thread/ThreadPriorities.java 19.052 > 6 runtime/Thread/TooSmallStackSize.java 15.426 > > windows-x64: > > 1 runtime/Thread/CancellableThreadTest.java 8.418 > 2 runtime/Thread/Fibonacci.java 10.984 > 3 runtime/Thread/TestThreadDumpMonitorContention.java 00:01:52.157 > 4 runtime/Thread/TestThreadStackSizes.java 13.241 > 5 runtime/Thread/ThreadPriorities.java 29.333 > 6 runtime/Thread/TooSmallStackSize.java 45.316 Thanks. Those times look fine. Dan > > Chris >> Chris >>> >>> Dan >>> >>> >>>> >>>>> >>>>> Thumbs up! I don't need to see another webrev if you choose to >>>>> fix these minor typos... >>>> They've all been fixed. >>>> >>>> thanks for the review, >>>> >>>> Chris >>>> >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 3/20/17 5:29 PM, Chris Plummer wrote: >>>>>> On 3/17/17 11:37 PM, Chris Plummer wrote: >>>>>>> On 3/17/17 8:17 PM, Chris Plummer wrote: >>>>>>>> On 3/17/17 7:01 PM, David Holmes wrote: >>>>>>>>> On 18/03/2017 9:11 AM, Chris Plummer wrote: >>>>>>>>>> Looks like this will need some more work since the added >>>>>>>>>> asserts are >>>>>>>>>> triggering on mac os x (which is the only place we'd >>>>>>>>>> currently expect >>>>>>>>>> them to assert). >>>>>>>>>> >>>>>>>>>> The problem is that the code I found that rounds up to the >>>>>>>>>> page size is >>>>>>>>>> only applied to java threads created by the VM for which the >>>>>>>>>> java user >>>>>>>>>> specified no stack size. The VM and Compiler thread sizes are >>>>>>>>>> not >>>>>>>>>> rounded. The failure I saw was with >>>>>>>>>> runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>>>>>>> when is >>>>>>>>>> specified -XX:CompilerThreadStackSize=9007199254740991. I hit >>>>>>>>>> the assert >>>>>>>>>> with an EINVAL. The size is not aligned, but it could also be >>>>>>>>>> complaining because it is too big. I haven't tried aligning >>>>>>>>>> it yet to see. >>>>>>>>>> >>>>>>>>>> On Linux we do the following: >>>>>>>>>> >>>>>>>>>> stack_size = align_size_up(stack_size + >>>>>>>>>> os::Linux::default_guard_size(thr_type), vm_page_size()); >>>>>>>>>> >>>>>>>>>> We don't do this on BSD. I think the same on BSD would solve >>>>>>>>>> this >>>>>>>>>> problem. I'm not sure about adding the guard size. I'll need >>>>>>>>>> to see if >>>>>>>>>> mac os x has the same pthread bug as linux does. >>>>>>>>> >>>>>>>>> At this stage I would only deal with alignment issues. IIRC >>>>>>>>> the guard issue only affected Linux. >>>>>>>> Yes, that's what I eventually concluded. I put the fix in >>>>>>>> os::Posix::get_initial_stack_size() in os_posix.cpp, and only >>>>>>>> did the page aligning, not add the guard page. That way all >>>>>>>> Posix ports are fixed in one place. It seems to be working. >>>>>>>>> >>>>>>>>>> BTW, did you know java users can specify the size of the a >>>>>>>>>> new thread's >>>>>>>>>> stack: >>>>>>>>> >>>>>>>>> Yes I mentioned that in another reply - wondering whether we >>>>>>>>> suitably check and aligned such requests. >>>>>>>> No we don't. Below I mentioned I was able to trigger the assert >>>>>>>> with a 257k stack size. I guess I wasn't clear that I did that >>>>>>>> from Java. I have a new test to add that will be testing this, >>>>>>>> plus the 9007199254740991 stack size (which fails to create the >>>>>>>> thread with an OOME, but that's acceptable). The fix I mention >>>>>>>> above in os::Posix::get_initial_stack_size() takes care of this >>>>>>>> issue also. >>>>>>> Rounding up triggers a new assert, this time on 32-bit x86 and arm. >>>>>>> >>>>>>> I should have clarified it's 9007199254740991 "K", which is >>>>>>> 9223372036854774784. Unfortunately on 32bit systems that is >>>>>>> asserting with EINVAL. I think we need to do a better job of >>>>>>> dealing with 32-bit size_t values: >>>>>>> >>>>>>> jlong java_lang_Thread::stackSize(oop java_thread) { >>>>>>> if (_stackSize_offset > 0) { >>>>>>> return java_thread->long_field(_stackSize_offset); >>>>>>> } else { >>>>>>> return 0; >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> jlong size = >>>>>>> java_lang_Thread::stackSize(JNIHandles::resolve_non_null(jthread)); >>>>>>> // Allocate the C++ Thread structure and create the native >>>>>>> thread. The >>>>>>> // stack size retrieved from java is signed, but the >>>>>>> constructor takes >>>>>>> // size_t (an unsigned type), so avoid passing negative >>>>>>> values which would >>>>>>> // result in really large stacks. >>>>>>> size_t sz = size > 0 ? (size_t) size : 0; >>>>>>> native_thread = new JavaThread(&thread_entry, sz); >>>>>>> >>>>>>> 9223372036854774784 is 0x7ffffffffffffc00 (close to 64 bit >>>>>>> MAX_INT), which is 0xfffffc00 when cast to a size_t on a 32-bit >>>>>>> system (close to 32-bit MAX_UINT). Round it up to the 4k page >>>>>>> size and you get 0, which I guess pthread_attr_setstacksize() >>>>>>> doesn't like. So I think more processing of the size is needed >>>>>>> here. Maybe in os::create_thread() we should check for 0 after >>>>>>> rounding up, and subtract the os page size if it is 0. However, >>>>>>> I think we should also avoid truncating on 32-bit to what is >>>>>>> basically some random number. Maybe if "size" (a jlong) is >>>>>>> greater than UINT_MAX, then just set "sz" (a size_t) it to >>>>>>> UINT_MAX. >>>>>>> >>>>>> Ok, I think I have this all worked out now. I've added fixes for >>>>>> unaligned stack sizes, 32-bit truncating of stack size, and the >>>>>> "aligning up to 0" problem. I also added a test. Here's the >>>>>> latest webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.02/webrev.hotspot >>>>>> >>>>>> >>>>>> Here's what's changed since webrev.01: >>>>>> >>>>>> os_posix.cpp: In os::Posix::get_initial_stack_size(), first round >>>>>> up the stack size to be paged aligned. This fixes issues on Mac >>>>>> OS X (other platforms seem to be immune to this). Then check if >>>>>> the size is zero after rounding up to the page size. Subtract the >>>>>> page size in this case to produce the maximum stack size allowed. >>>>>> Surprisingly I got no complaint from gcc for subtracting from an >>>>>> unsigned value that is known to be 0. >>>>>> >>>>>> os_linux.cpp: In os::create_thread(), I also check here to make >>>>>> sure the size is not 0 after adding the guard page and aligning >>>>>> up, and subtract the os page size if it is 0. >>>>>> >>>>>> jvm.c: In JVM_StartThread(), on 32-bit platforms if the size is >>>>>> greater than UINT_MAX, then I set the size to UINT_MAX. Note it >>>>>> will later be rounded up to 0 in >>>>>> os::Posix::get_initial_stack_size(), which will result in >>>>>> subtracting the os page size to get the actual maximum allowed >>>>>> stack size. >>>>>> >>>>>> TooSmallStackSize.java: added test case for unaligned stack sizes. >>>>>> >>>>>> TestThreadStackSizes.java: New test. Creates new threads with >>>>>> every size up to 320k in 1k increments. Then creates threads with >>>>>> a few other sizes that can be problematic. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Chris >>>>>>> Chris >>>>>>> >>>>>>> >>>>>>>>> >>>>>>>>>> public Thread(ThreadGroup group, Runnable target, String >>>>>>>>>> name, >>>>>>>>>> long stackSize) { >>>>>>>>>> init(group, target, name, stackSize); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> Fortunately we already force the stackSize to be at least >>>>>>>>>> _java_thread_min_stack_allowed. However, we don't do any OS page >>>>>>>>>> rounding on Mac OS X as noted above, and I was able to >>>>>>>>>> trigger the >>>>>>>>>> assert by creating a thread with size 257k. >>>>>>>>> >>>>>>>>> Note this means that OSX stack logic is broken because it will >>>>>>>>> currently be silently failing due to EINVAL! >>>>>>>> Yes, that is correct. >>>>>>>> >>>>>>>> Chris >>>>>>>>> >>>>>>>>>> I'll get another webrev out once I've made the needed fixes. >>>>>>>>>> I also have >>>>>>>>>> a new test I'd like to add. >>>>>>>>> >>>>>>>>> Ok. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>>> On 3/16/17 9:27 PM, Chris Plummer wrote: >>>>>>>>>>> Ok, time for a new webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.01/webrev.hotspot >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The only thing that has changed since the first webrev are >>>>>>>>>>> the asserts >>>>>>>>>>> added to os_linux.cpp and os_bsd.cpp. And to summarize what >>>>>>>>>>> we discuss >>>>>>>>>>> already: >>>>>>>>>>> >>>>>>>>>>> - The assert should never happen due to the stack size >>>>>>>>>>> being too >>>>>>>>>>> small since it will be at least PTHREAD_STACK_MIN. >>>>>>>>>>> - The assert should never happen due to an unaligned stack >>>>>>>>>>> size >>>>>>>>>>> because we always align it to the page size. >>>>>>>>>>> - Any assert would therefore be a VM bug and not due to >>>>>>>>>>> user error. >>>>>>>>>>> - No fixing the java launcher. If the user specifies a >>>>>>>>>>> stack that is >>>>>>>>>>> too small, hotspot will already detect this. If the user >>>>>>>>>>> specifies a >>>>>>>>>>> stack size that is large enough but not page aligned, then >>>>>>>>>>> we just >>>>>>>>>>> ignore any error (if the platform doth protest) and the user >>>>>>>>>>> gets a >>>>>>>>>>> main thread with a stack size set to whatever the OS default >>>>>>>>>>> is. >>>>>>>>>>> >>>>>>>>>>> I still need to retest (I only ran TooSmallStackSize.java), >>>>>>>>>>> but figure >>>>>>>>>>> getting agreement on the changes first would be best before >>>>>>>>>>> I bog down >>>>>>>>>>> our testing resources. >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> >>>>>>>>>>> Chris >>>>>>>>>>> >>>>>>>>>>> On 3/15/17 10:03 PM, Chris Plummer wrote: >>>>>>>>>>>> Hello, >>>>>>>>>>>> >>>>>>>>>>>> Please review the following: >>>>>>>>>>>> >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8176768 >>>>>>>>>>>> http://cr.openjdk.java.net/~cjplummer/8176768/webrev.00/webrev.hotspot >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> While working on 8175342 I noticed our stack size on xgene >>>>>>>>>>>> was 8mb >>>>>>>>>>>> even though I was specifying -Xss72k. It turns out the >>>>>>>>>>>> following code >>>>>>>>>>>> was failing: >>>>>>>>>>>> >>>>>>>>>>>> pthread_attr_setstacksize(&attr, stack_size); >>>>>>>>>>>> >>>>>>>>>>>> Although we computed a minimum stack size of 72k, so >>>>>>>>>>>> -Xss72k should >>>>>>>>>>>> be fine, pthreads on this platform requires the stack be at >>>>>>>>>>>> least >>>>>>>>>>>> 128k, so it failed the pthread_attr_setstacksize() call. >>>>>>>>>>>> The end >>>>>>>>>>>> result is pthread_attr_setstacksize() had no impact on the >>>>>>>>>>>> thread's >>>>>>>>>>>> stack size, and we ended up with the platform default of >>>>>>>>>>>> 8mb. The fix >>>>>>>>>>>> is to round up the following variables to PTHREAD_STACK_MIN >>>>>>>>>>>> after >>>>>>>>>>>> computing their new values: >>>>>>>>>>>> >>>>>>>>>>>> _java_thread_min_stack_allowed >>>>>>>>>>>> _compiler_thread_min_stack_allowed >>>>>>>>>>>> _vm_internal_thread_min_stack_allowed >>>>>>>>>>>> >>>>>>>>>>>> For solaris, there was an issue using PTHREAD_STACK_MIN. >>>>>>>>>>>> You need to >>>>>>>>>>>> #define _POSIX_C_SOURCE >= 199506L in order to get >>>>>>>>>>>> PTHREAD_STACK_MIN >>>>>>>>>>>> #defined, and this needs to be done before including OS >>>>>>>>>>>> header files. >>>>>>>>>>>> I noticed that on solaris we were using thr_min_stack() >>>>>>>>>>>> elsewhere >>>>>>>>>>>> instead of PTHREAD_STACK_MIN, so I decided to do the same >>>>>>>>>>>> with this >>>>>>>>>>>> fix. Either way is ugly (the #define or using >>>>>>>>>>>> thr_min_stack()). >>>>>>>>>>>> >>>>>>>>>>>> And speaking of the existing use of thr_min_stack(), I >>>>>>>>>>>> deleted it. It >>>>>>>>>>>> was being applied before any adjustments to the stack sizes >>>>>>>>>>>> had been >>>>>>>>>>>> made (rounding and adding red, yellow, and shadow zones). >>>>>>>>>>>> This mean >>>>>>>>>>>> the stack ended up being larger than necessary. With the >>>>>>>>>>>> above fix in >>>>>>>>>>>> place, we are now applying thr_min_stack() after >>>>>>>>>>>> recomputing the >>>>>>>>>>>> minimum stack sizes. If for any reason one of those stack >>>>>>>>>>>> sizes is >>>>>>>>>>>> now too small, the correct fix is to adjust the initial >>>>>>>>>>>> stack sizes, >>>>>>>>>>>> not apply thr_min_stack() to the initial stack sizes. >>>>>>>>>>>> However, it >>>>>>>>>>>> looks like no adjustment is needed. I did something close >>>>>>>>>>>> to our >>>>>>>>>>>> nightly testing on all affect platforms, and no new >>>>>>>>>>>> problems turned up. >>>>>>>>>>>> >>>>>>>>>>>> thanks, >>>>>>>>>>>> >>>>>>>>>>>> Chris >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Wed Mar 22 00:02:03 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Mar 2017 10:02:03 +1000 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> Message-ID: <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> I haven't been looking at the details of this but have been watching from afar. As per my comments in the bug report (now public) I'm quite concerned about the thread-non-safety issue here ... On 22/03/2017 4:47 AM, dean.long at oracle.com wrote: > On 3/21/17 9:37 AM, Vladimir Ivanov wrote: > >>> and webrev.2 with it removed: >>> >>> http://cr.openjdk.java.net/~dlong/8158168/webrev.2/ >> >> Thanks, Dean. I started with webrev.2 and tried to minimize the >> changes. I ended up with the following version: >> >> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/webrev.00/ >> > > Thanks. The reason I didn't go with that approach from the beginning is > because I couldn't convince myself that I could find all the missing > bounds checks, and I wanted an interface to test against. With the > bounds checks in AbstractStringBuilder, it is very hard to test all the > possible race conditions, because some of the race conditions only > happen when an ASB field changes half-way through the method. So are we convinced that the proposed changes will never lead to a crash due to a missing or incorrect bounds check, due to a racy use of an unsynchronized ASB instance e.g. StringBuilder? Thanks, David ----- >> Some clarifications: >> >> ============ >> src/java.base/share/classes/java/lang/String.java: >> >> The bounds check is needed only in String.nonSyncContentEquals when it >> extracts info from AbstractStringBuilder. I don't see how out of >> bounds access can happen in String.contentEquals: >> if (n != length()) { >> return false; >> } >> ... >> for (int i = 0; i < n; i++) { >> if (StringUTF16.getChar(val, i) != cs.charAt(i)) { >> > > OK. > >> ============ >> src/java.base/share/classes/java/lang/StringConcatHelper.java: >> >> I think bounds checks in StringConcatHelper.prepend() are skipped >> intentionally, since java.lang.invoke.StringConcatFactory constructs >> method handle chains which already contain bounds checks: array length >> is precomputed based on argument values and all accesses are >> guaranteed to be in bounds. >> > > This is calling the trusted version of getChars() with no bounds > checks. It was a little more obvious when I had the Trusted inner class. > >> ============ >> src/java.base/share/classes/java/lang/StringUTF16.java: >> >> + static void putChar(byte[] val, int index, int c) { >> + assert index >= 0 && index < length(val) : "Trusted caller >> missed bounds check"; >> >> Unfortunately, asserts can affect inlining decisions (since they >> increase bytecode size). In order to minimize possible performance >> impact, I suggest to remove them from the fix targeting 9. >> > > Sure. > >> ============ >> private static int indexOfSupplementary(byte[] value, int ch, int >> fromIndex, int max) { >> if (Character.isValidCodePoint(ch)) { >> final char hi = Character.highSurrogate(ch); >> final char lo = Character.lowSurrogate(ch); >> + checkBoundsBeginEnd(fromIndex, max, value); >> >> The check is redundant here. fromIndex & max are always inbounds by >> construction: >> >> public static int indexOf(byte[] value, int ch, int fromIndex) { >> int max = value.length >> 1; >> if (fromIndex < 0) { >> fromIndex = 0; >> } else if (fromIndex >= max) { >> // Note: fromIndex might be near -1>>>1. >> return -1; >> } >> ... >> return indexOfSupplementary(value, ch, fromIndex, max); >> > > OK. > >> ============ >> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >> ABS.value & ABS.count. After that, all accesses should be inbounds by >> construction (in String.indexOf/lastIndexOf): >> >> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >> static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, >> String tgtStr, int fromIndex) { >> >> int rightIndex = srcCount - tgtCount; >> if (fromIndex > rightIndex) { >> fromIndex = rightIndex; >> } >> if (fromIndex < 0) { >> return -1; >> } >> >> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >> public static int lastIndexOf(byte[] src, int srcCount, >> byte[] tgt, int tgtCount, int >> fromIndex) { >> int min = tgtCount - 1; >> int i = min + fromIndex; >> int strLastIndex = tgtCount - 1; >> char strLastChar = getChar(tgt, strLastIndex); >> >> startSearchForLastChar: >> while (true) { >> while (i >= min && getChar(src, i) != strLastChar) { >> >> There are 2 places: >> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - inbound >> >> * getChar(src, i); i in [ min; min+fromIndex ] >> min = tgtCount - 1 >> rightIndex = srcCount - tgtCount >> fromIndex <= rightIndex >> >> 0 <= min + fromIndex <= min + rightIndex == (tgtCount - 1) >> + (srcCount - tgtCount) == srcCount - 1 >> >> Hence, should be covered by the check on count & value: >> public int lastIndexOf(String str, int fromIndex) { >> + byte[] value = this.value; >> + int count = this.count; >> + byte coder = this.coder; >> + checkIndex(count, value.length >> coder); >> return String.lastIndexOf(value, coder, count, str, fromIndex); >> } >> > > OK, I will go with your version if it's OK with Sherman. > > dl > >> Best regards, >> Vladimir Ivanov >> >>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>> >>>>>> I have the same concern. Can we fix the immediate problem in 9 and >>>>>> integrate verification logic in 10? >>>>>> >>>>> >>>>> OK, Tobias is suggesting having verification logic only inside the >>>>> intrinsics. Are you suggesting removing that as well? >>>> >>>> Yes and put them back in 10. >>>> >>>>> I'm OK with removing all the verification, but that won't reduce the >>>>> library changes much. I could undo the renaming to >>>>> Trusted.getChar, but >>>>> we would still have the bounds checks moved into StringUTF16. >>>> >>>> I suggest to go with a point fix for 9: just add missing range checks. >>> > From dean.long at oracle.com Wed Mar 22 00:26:01 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 21 Mar 2017 17:26:01 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> Message-ID: On 3/21/17 5:02 PM, David Holmes wrote: > I haven't been looking at the details of this but have been watching > from afar. As per my comments in the bug report (now public) I'm quite > concerned about the thread-non-safety issue here ... > > On 22/03/2017 4:47 AM, dean.long at oracle.com wrote: >> On 3/21/17 9:37 AM, Vladimir Ivanov wrote: >> >>>> and webrev.2 with it removed: >>>> >>>> http://cr.openjdk.java.net/~dlong/8158168/webrev.2/ >>> >>> Thanks, Dean. I started with webrev.2 and tried to minimize the >>> changes. I ended up with the following version: >>> >>> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/webrev.00/ >>> >> >> Thanks. The reason I didn't go with that approach from the beginning is >> because I couldn't convince myself that I could find all the missing >> bounds checks, and I wanted an interface to test against. With the >> bounds checks in AbstractStringBuilder, it is very hard to test all the >> possible race conditions, because some of the race conditions only >> happen when an ASB field changes half-way through the method. > > So are we convinced that the proposed changes will never lead to a > crash due to a missing or incorrect bounds check, due to a racy use of > an unsynchronized ASB instance e.g. StringBuilder? > If only we had a static analysis tool that could tell us if the code is safe. Because we don't, in my initial changeset, we always take a snapshot of the ASB fields by passing those field values to StringUTF16 before doing checks on them. And I wrote a test to make sure that those StringUTF16 interfaces are catching all the underflows and overflows I could imagine, and I added verification code to detect when a check was missed. However, all the reviewers have requested to minimize the amount of changes. In Vladimir's version, if there is a missing check somewhere, then yes it could lead to a crash. dl > Thanks, > David > ----- > >>> Some clarifications: >>> >>> ============ >>> src/java.base/share/classes/java/lang/String.java: >>> >>> The bounds check is needed only in String.nonSyncContentEquals when it >>> extracts info from AbstractStringBuilder. I don't see how out of >>> bounds access can happen in String.contentEquals: >>> if (n != length()) { >>> return false; >>> } >>> ... >>> for (int i = 0; i < n; i++) { >>> if (StringUTF16.getChar(val, i) != cs.charAt(i)) { >>> >> >> OK. >> >>> ============ >>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>> >>> I think bounds checks in StringConcatHelper.prepend() are skipped >>> intentionally, since java.lang.invoke.StringConcatFactory constructs >>> method handle chains which already contain bounds checks: array length >>> is precomputed based on argument values and all accesses are >>> guaranteed to be in bounds. >>> >> >> This is calling the trusted version of getChars() with no bounds >> checks. It was a little more obvious when I had the Trusted inner >> class. >> >>> ============ >>> src/java.base/share/classes/java/lang/StringUTF16.java: >>> >>> + static void putChar(byte[] val, int index, int c) { >>> + assert index >= 0 && index < length(val) : "Trusted caller >>> missed bounds check"; >>> >>> Unfortunately, asserts can affect inlining decisions (since they >>> increase bytecode size). In order to minimize possible performance >>> impact, I suggest to remove them from the fix targeting 9. >>> >> >> Sure. >> >>> ============ >>> private static int indexOfSupplementary(byte[] value, int ch, int >>> fromIndex, int max) { >>> if (Character.isValidCodePoint(ch)) { >>> final char hi = Character.highSurrogate(ch); >>> final char lo = Character.lowSurrogate(ch); >>> + checkBoundsBeginEnd(fromIndex, max, value); >>> >>> The check is redundant here. fromIndex & max are always inbounds by >>> construction: >>> >>> public static int indexOf(byte[] value, int ch, int fromIndex) { >>> int max = value.length >> 1; >>> if (fromIndex < 0) { >>> fromIndex = 0; >>> } else if (fromIndex >= max) { >>> // Note: fromIndex might be near -1>>>1. >>> return -1; >>> } >>> ... >>> return indexOfSupplementary(value, ch, fromIndex, max); >>> >> >> OK. >> >>> ============ >>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >>> ABS.value & ABS.count. After that, all accesses should be inbounds by >>> construction (in String.indexOf/lastIndexOf): >>> >>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>> static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, >>> String tgtStr, int fromIndex) { >>> >>> int rightIndex = srcCount - tgtCount; >>> if (fromIndex > rightIndex) { >>> fromIndex = rightIndex; >>> } >>> if (fromIndex < 0) { >>> return -1; >>> } >>> >>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>> public static int lastIndexOf(byte[] src, int srcCount, >>> byte[] tgt, int tgtCount, int >>> fromIndex) { >>> int min = tgtCount - 1; >>> int i = min + fromIndex; >>> int strLastIndex = tgtCount - 1; >>> char strLastChar = getChar(tgt, strLastIndex); >>> >>> startSearchForLastChar: >>> while (true) { >>> while (i >= min && getChar(src, i) != strLastChar) { >>> >>> There are 2 places: >>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - inbound >>> >>> * getChar(src, i); i in [ min; min+fromIndex ] >>> min = tgtCount - 1 >>> rightIndex = srcCount - tgtCount >>> fromIndex <= rightIndex >>> >>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount - 1) >>> + (srcCount - tgtCount) == srcCount - 1 >>> >>> Hence, should be covered by the check on count & value: >>> public int lastIndexOf(String str, int fromIndex) { >>> + byte[] value = this.value; >>> + int count = this.count; >>> + byte coder = this.coder; >>> + checkIndex(count, value.length >> coder); >>> return String.lastIndexOf(value, coder, count, str, >>> fromIndex); >>> } >>> >> >> OK, I will go with your version if it's OK with Sherman. >> >> dl >> >>> Best regards, >>> Vladimir Ivanov >>> >>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>> >>>>>>> I have the same concern. Can we fix the immediate problem in 9 and >>>>>>> integrate verification logic in 10? >>>>>>> >>>>>> >>>>>> OK, Tobias is suggesting having verification logic only inside the >>>>>> intrinsics. Are you suggesting removing that as well? >>>>> >>>>> Yes and put them back in 10. >>>>> >>>>>> I'm OK with removing all the verification, but that won't reduce the >>>>>> library changes much. I could undo the renaming to >>>>>> Trusted.getChar, but >>>>>> we would still have the bounds checks moved into StringUTF16. >>>>> >>>>> I suggest to go with a point fix for 9: just add missing range >>>>> checks. >>>> >> From david.holmes at oracle.com Wed Mar 22 05:09:40 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Mar 2017 15:09:40 +1000 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> Message-ID: On 22/03/2017 10:26 AM, dean.long at oracle.com wrote: > On 3/21/17 5:02 PM, David Holmes wrote: > >> I haven't been looking at the details of this but have been watching >> from afar. As per my comments in the bug report (now public) I'm quite >> concerned about the thread-non-safety issue here ... >> >> On 22/03/2017 4:47 AM, dean.long at oracle.com wrote: >>> On 3/21/17 9:37 AM, Vladimir Ivanov wrote: >>> >>>>> and webrev.2 with it removed: >>>>> >>>>> http://cr.openjdk.java.net/~dlong/8158168/webrev.2/ >>>> >>>> Thanks, Dean. I started with webrev.2 and tried to minimize the >>>> changes. I ended up with the following version: >>>> >>>> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/webrev.00/ >>>> >>> >>> Thanks. The reason I didn't go with that approach from the beginning is >>> because I couldn't convince myself that I could find all the missing >>> bounds checks, and I wanted an interface to test against. With the >>> bounds checks in AbstractStringBuilder, it is very hard to test all the >>> possible race conditions, because some of the race conditions only >>> happen when an ASB field changes half-way through the method. >> >> So are we convinced that the proposed changes will never lead to a >> crash due to a missing or incorrect bounds check, due to a racy use of >> an unsynchronized ASB instance e.g. StringBuilder? >> > > If only we had a static analysis tool that could tell us if the code is > safe. Because we don't, in my initial changeset, we always take a > snapshot of the ASB fields by passing those field values to StringUTF16 > before doing checks on them. And I wrote a test to make sure that those > StringUTF16 interfaces are catching all the underflows and overflows I > could imagine, and I added verification code to detect when a check was > missed. > > However, all the reviewers have requested to minimize the amount of > changes. In Vladimir's version, if there is a missing check somewhere, > then yes it could lead to a crash. I wonder if the reviewers have fully realized the potential impact here? This has exposed a flaw in the way intrinsics are used from core classes. Thanks, David ----- > > dl > >> Thanks, >> David >> ----- >> >>>> Some clarifications: >>>> >>>> ============ >>>> src/java.base/share/classes/java/lang/String.java: >>>> >>>> The bounds check is needed only in String.nonSyncContentEquals when it >>>> extracts info from AbstractStringBuilder. I don't see how out of >>>> bounds access can happen in String.contentEquals: >>>> if (n != length()) { >>>> return false; >>>> } >>>> ... >>>> for (int i = 0; i < n; i++) { >>>> if (StringUTF16.getChar(val, i) != cs.charAt(i)) { >>>> >>> >>> OK. >>> >>>> ============ >>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>> >>>> I think bounds checks in StringConcatHelper.prepend() are skipped >>>> intentionally, since java.lang.invoke.StringConcatFactory constructs >>>> method handle chains which already contain bounds checks: array length >>>> is precomputed based on argument values and all accesses are >>>> guaranteed to be in bounds. >>>> >>> >>> This is calling the trusted version of getChars() with no bounds >>> checks. It was a little more obvious when I had the Trusted inner >>> class. >>> >>>> ============ >>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>> >>>> + static void putChar(byte[] val, int index, int c) { >>>> + assert index >= 0 && index < length(val) : "Trusted caller >>>> missed bounds check"; >>>> >>>> Unfortunately, asserts can affect inlining decisions (since they >>>> increase bytecode size). In order to minimize possible performance >>>> impact, I suggest to remove them from the fix targeting 9. >>>> >>> >>> Sure. >>> >>>> ============ >>>> private static int indexOfSupplementary(byte[] value, int ch, int >>>> fromIndex, int max) { >>>> if (Character.isValidCodePoint(ch)) { >>>> final char hi = Character.highSurrogate(ch); >>>> final char lo = Character.lowSurrogate(ch); >>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>> >>>> The check is redundant here. fromIndex & max are always inbounds by >>>> construction: >>>> >>>> public static int indexOf(byte[] value, int ch, int fromIndex) { >>>> int max = value.length >> 1; >>>> if (fromIndex < 0) { >>>> fromIndex = 0; >>>> } else if (fromIndex >= max) { >>>> // Note: fromIndex might be near -1>>>1. >>>> return -1; >>>> } >>>> ... >>>> return indexOfSupplementary(value, ch, fromIndex, max); >>>> >>> >>> OK. >>> >>>> ============ >>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >>>> ABS.value & ABS.count. After that, all accesses should be inbounds by >>>> construction (in String.indexOf/lastIndexOf): >>>> >>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>> static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, >>>> String tgtStr, int fromIndex) { >>>> >>>> int rightIndex = srcCount - tgtCount; >>>> if (fromIndex > rightIndex) { >>>> fromIndex = rightIndex; >>>> } >>>> if (fromIndex < 0) { >>>> return -1; >>>> } >>>> >>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>> public static int lastIndexOf(byte[] src, int srcCount, >>>> byte[] tgt, int tgtCount, int >>>> fromIndex) { >>>> int min = tgtCount - 1; >>>> int i = min + fromIndex; >>>> int strLastIndex = tgtCount - 1; >>>> char strLastChar = getChar(tgt, strLastIndex); >>>> >>>> startSearchForLastChar: >>>> while (true) { >>>> while (i >= min && getChar(src, i) != strLastChar) { >>>> >>>> There are 2 places: >>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - inbound >>>> >>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>> min = tgtCount - 1 >>>> rightIndex = srcCount - tgtCount >>>> fromIndex <= rightIndex >>>> >>>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount - 1) >>>> + (srcCount - tgtCount) == srcCount - 1 >>>> >>>> Hence, should be covered by the check on count & value: >>>> public int lastIndexOf(String str, int fromIndex) { >>>> + byte[] value = this.value; >>>> + int count = this.count; >>>> + byte coder = this.coder; >>>> + checkIndex(count, value.length >> coder); >>>> return String.lastIndexOf(value, coder, count, str, >>>> fromIndex); >>>> } >>>> >>> >>> OK, I will go with your version if it's OK with Sherman. >>> >>> dl >>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>> >>>>>>>> I have the same concern. Can we fix the immediate problem in 9 and >>>>>>>> integrate verification logic in 10? >>>>>>>> >>>>>>> >>>>>>> OK, Tobias is suggesting having verification logic only inside the >>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>> >>>>>> Yes and put them back in 10. >>>>>> >>>>>>> I'm OK with removing all the verification, but that won't reduce the >>>>>>> library changes much. I could undo the renaming to >>>>>>> Trusted.getChar, but >>>>>>> we would still have the bounds checks moved into StringUTF16. >>>>>> >>>>>> I suggest to go with a point fix for 9: just add missing range >>>>>> checks. >>>>> >>> > From tobias.hartmann at oracle.com Wed Mar 22 07:26:23 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 22 Mar 2017 08:26:23 +0100 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> Message-ID: Hi, On 22.03.2017 06:09, David Holmes wrote: > I wonder if the reviewers have fully realized the potential impact here? This has exposed a flaw in the way intrinsics are used from core classes. For the record, I'm fine with both the initial fix as well as the simplified version. If we don't add the debug runtime checks in the intrinsics, we should definitely add them with JDK 10 as additional safety net (we should consider other intrinsics and C1 as well). Thanks, Tobias From mikael.gerdin at oracle.com Wed Mar 22 14:21:34 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 22 Mar 2017 15:21:34 +0100 Subject: C/C++ IDE support for HotSpot Message-ID: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Hi all, I've finally grown tired of manually setting up a hand crafted Eclipse CDT configuration for the JVM sources and decided to share my progress towards improving the overall situation for JVM developers. To achieve better IDE support without having to add project generators for all different kinds of past or future IDEs I've decided to try to leverage CMake to do project generation. The idea is to have the JDK build system generate a CMakeLists.txt describing all the include paths and definitions required by an IDE to interpret the sources correctly. Several modern IDEs natively support CMake but we can also rely on the fact that the CMake build system has the ability to generate projects for a number of different IDEs. For information about which generators CMake supports see https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html for your CMake version. To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in the jdk10/sandbox forest: http://hg.openjdk.java.net/jdk10/sandbox/branches So far I've only made changes in the toplevel and hotspot repositories. I've written a short readme in the repo: http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html It would be great if people tried this out to see if it works on their setup but I don't expect it to work on Windows without changing the makefile to do path conversion. If we can make this work properly then perhaps we can get rid of the Visual Studio generator and rely on CMake to generate VS projects. It would also be great if someone from build-dev could give me some hints about how to do the file writing and "vardeps" thing properly. Thanks /Mikael From stanislav.smirnov at oracle.com Wed Mar 22 15:02:25 2017 From: stanislav.smirnov at oracle.com (Stanislav Smirnov) Date: Wed, 22 Mar 2017 18:02:25 +0300 Subject: C/C++ IDE support for HotSpot In-Reply-To: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Message-ID: Hi Mikael, why do not you try NetBeans that has openjdk project support out of the box? common/nb_native/nbproject Best regards, Stanislav Smirnov > On 22 Mar 2017, at 17:21, Mikael Gerdin wrote: > > Hi all, > > I've finally grown tired of manually setting up a hand crafted Eclipse CDT configuration for the JVM sources and decided to share my progress towards improving the overall situation for JVM developers. > > To achieve better IDE support without having to add project generators for all different kinds of past or future IDEs I've decided to try to leverage CMake to do project generation. > The idea is to have the JDK build system generate a CMakeLists.txt describing all the include paths and definitions required by an IDE to interpret the sources correctly. > > Several modern IDEs natively support CMake but we can also rely on the fact that the CMake build system has the ability to generate projects for a number of different IDEs. For information about which generators CMake supports see > https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html > for your CMake version. > > To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in the jdk10/sandbox forest: > http://hg.openjdk.java.net/jdk10/sandbox/branches > So far I've only made changes in the toplevel and hotspot repositories. > I've written a short readme in the repo: > http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html > > It would be great if people tried this out to see if it works on their setup but I don't expect it to work on Windows without changing the makefile to do path conversion. > If we can make this work properly then perhaps we can get rid of the Visual Studio generator and rely on CMake to generate VS projects. > > It would also be great if someone from build-dev could give me some hints about how to do the file writing and "vardeps" thing properly. > > Thanks > /Mikael From vladimir.x.ivanov at oracle.com Wed Mar 22 15:35:35 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 22 Mar 2017 18:35:35 +0300 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> Message-ID: <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> >>> So are we convinced that the proposed changes will never lead to a >>> crash due to a missing or incorrect bounds check, due to a racy use of >>> an unsynchronized ASB instance e.g. StringBuilder? >> >> If only we had a static analysis tool that could tell us if the code is >> safe. Because we don't, in my initial changeset, we always take a >> snapshot of the ASB fields by passing those field values to StringUTF16 >> before doing checks on them. And I wrote a test to make sure that those >> StringUTF16 interfaces are catching all the underflows and overflows I >> could imagine, and I added verification code to detect when a check was >> missed. >> >> However, all the reviewers have requested to minimize the amount of >> changes. In Vladimir's version, if there is a missing check somewhere, >> then yes it could lead to a crash. I'd like to point out that asserts and verification code are disabled by default. They are invaluable during problem diagnosis, but don't help at all from defence-in-depth perspective. But I agree that it's easier to reason about and test the initial version of the fix. > I wonder if the reviewers have fully realized the potential impact here? > This has exposed a flaw in the way intrinsics are used from core classes. FTR here are the checks I omitted in the minimized version (modulo separation of indexOf/lastIndexOf for trusted/non-trusted callers): http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ Other than that, the difference is mainly about undoing refactorings and removing verification logic (asserts + checks in the JVM). There are still unsafe accesses which are considered safe in both versions (see StringUTF16.Trusted usages in the initial version [1]). We used to provide safe wrappers for unsafe intrinsics which makes it much easier to reason about code correctness. I'd like to see compact string code refactored that way and IMO the initial version by Dean is a big step in the right direction. I still prefer to see a point fix in 9 and major refactoring happening in 10, but I'll leave the decision on how to proceed with the fix to core-libs folks. After finishing the exercise minimizing the fix, I'm much more comfortable with the initial fix [1] (though there are changes I consider excessive). Best regards, Vladimir Ivanov [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 >>>>> Some clarifications: >>>>> >>>>> ============ >>>>> src/java.base/share/classes/java/lang/String.java: >>>>> >>>>> The bounds check is needed only in String.nonSyncContentEquals when it >>>>> extracts info from AbstractStringBuilder. I don't see how out of >>>>> bounds access can happen in String.contentEquals: >>>>> if (n != length()) { >>>>> return false; >>>>> } >>>>> ... >>>>> for (int i = 0; i < n; i++) { >>>>> if (StringUTF16.getChar(val, i) != cs.charAt(i)) { >>>>> >>>> >>>> OK. >>>> >>>>> ============ >>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>> >>>>> I think bounds checks in StringConcatHelper.prepend() are skipped >>>>> intentionally, since java.lang.invoke.StringConcatFactory constructs >>>>> method handle chains which already contain bounds checks: array length >>>>> is precomputed based on argument values and all accesses are >>>>> guaranteed to be in bounds. >>>>> >>>> >>>> This is calling the trusted version of getChars() with no bounds >>>> checks. It was a little more obvious when I had the Trusted inner >>>> class. >>>> >>>>> ============ >>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>> >>>>> + static void putChar(byte[] val, int index, int c) { >>>>> + assert index >= 0 && index < length(val) : "Trusted caller >>>>> missed bounds check"; >>>>> >>>>> Unfortunately, asserts can affect inlining decisions (since they >>>>> increase bytecode size). In order to minimize possible performance >>>>> impact, I suggest to remove them from the fix targeting 9. >>>>> >>>> >>>> Sure. >>>> >>>>> ============ >>>>> private static int indexOfSupplementary(byte[] value, int ch, int >>>>> fromIndex, int max) { >>>>> if (Character.isValidCodePoint(ch)) { >>>>> final char hi = Character.highSurrogate(ch); >>>>> final char lo = Character.lowSurrogate(ch); >>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>> >>>>> The check is redundant here. fromIndex & max are always inbounds by >>>>> construction: >>>>> >>>>> public static int indexOf(byte[] value, int ch, int fromIndex) { >>>>> int max = value.length >> 1; >>>>> if (fromIndex < 0) { >>>>> fromIndex = 0; >>>>> } else if (fromIndex >= max) { >>>>> // Note: fromIndex might be near -1>>>1. >>>>> return -1; >>>>> } >>>>> ... >>>>> return indexOfSupplementary(value, ch, fromIndex, max); >>>>> >>>> >>>> OK. >>>> >>>>> ============ >>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >>>>> ABS.value & ABS.count. After that, all accesses should be inbounds by >>>>> construction (in String.indexOf/lastIndexOf): >>>>> >>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>> static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, >>>>> String tgtStr, int fromIndex) { >>>>> >>>>> int rightIndex = srcCount - tgtCount; >>>>> if (fromIndex > rightIndex) { >>>>> fromIndex = rightIndex; >>>>> } >>>>> if (fromIndex < 0) { >>>>> return -1; >>>>> } >>>>> >>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>> byte[] tgt, int tgtCount, int >>>>> fromIndex) { >>>>> int min = tgtCount - 1; >>>>> int i = min + fromIndex; >>>>> int strLastIndex = tgtCount - 1; >>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>> >>>>> startSearchForLastChar: >>>>> while (true) { >>>>> while (i >= min && getChar(src, i) != strLastChar) { >>>>> >>>>> There are 2 places: >>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - inbound >>>>> >>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>> min = tgtCount - 1 >>>>> rightIndex = srcCount - tgtCount >>>>> fromIndex <= rightIndex >>>>> >>>>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount - 1) >>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>> >>>>> Hence, should be covered by the check on count & value: >>>>> public int lastIndexOf(String str, int fromIndex) { >>>>> + byte[] value = this.value; >>>>> + int count = this.count; >>>>> + byte coder = this.coder; >>>>> + checkIndex(count, value.length >> coder); >>>>> return String.lastIndexOf(value, coder, count, str, >>>>> fromIndex); >>>>> } >>>>> >>>> >>>> OK, I will go with your version if it's OK with Sherman. >>>> >>>> dl >>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>> >>>>>>>>> I have the same concern. Can we fix the immediate problem in 9 and >>>>>>>>> integrate verification logic in 10? >>>>>>>>> >>>>>>>> >>>>>>>> OK, Tobias is suggesting having verification logic only inside the >>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>> >>>>>>> Yes and put them back in 10. >>>>>>> >>>>>>>> I'm OK with removing all the verification, but that won't reduce >>>>>>>> the >>>>>>>> library changes much. I could undo the renaming to >>>>>>>> Trusted.getChar, but >>>>>>>> we would still have the bounds checks moved into StringUTF16. >>>>>>> >>>>>>> I suggest to go with a point fix for 9: just add missing range >>>>>>> checks. >>>>>> >>>> >> From erik.osterlund at oracle.com Wed Mar 22 16:20:06 2017 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 22 Mar 2017 17:20:06 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Message-ID: <58D2A436.4040203@oracle.com> Hi Mikael, In my workflow I use emacs with rtags for snytax awareness. With your cmake patch, I could easily generate the cmake files and run cmake with -DCMAKE_EXPORT_COMPILE_COMMANDS to generate the compiler_commands.json file used by rtags to understand how the code fits together, and feed it to rdm. Everything worked out of the box. With these changes it was faster than ever to setup a good emacs-based project for hotspot development. So I encourage these changes! Thanks, /Erik On 2017-03-22 15:21, Mikael Gerdin wrote: > Hi all, > > I've finally grown tired of manually setting up a hand crafted Eclipse > CDT configuration for the JVM sources and decided to share my progress > towards improving the overall situation for JVM developers. > > To achieve better IDE support without having to add project generators > for all different kinds of past or future IDEs I've decided to try to > leverage CMake to do project generation. > The idea is to have the JDK build system generate a CMakeLists.txt > describing all the include paths and definitions required by an IDE to > interpret the sources correctly. > > Several modern IDEs natively support CMake but we can also rely on the > fact that the CMake build system has the ability to generate projects > for a number of different IDEs. For information about which generators > CMake supports see > https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html > for your CMake version. > > To try this out check out (heh) my branch "JDK-8177329-cmake-branch" > in the jdk10/sandbox forest: > http://hg.openjdk.java.net/jdk10/sandbox/branches > So far I've only made changes in the toplevel and hotspot repositories. > I've written a short readme in the repo: > http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html > > > It would be great if people tried this out to see if it works on their > setup but I don't expect it to work on Windows without changing the > makefile to do path conversion. > If we can make this work properly then perhaps we can get rid of the > Visual Studio generator and rely on CMake to generate VS projects. > > It would also be great if someone from build-dev could give me some > hints about how to do the file writing and "vardeps" thing properly. > > Thanks > /Mikael From mikael.vidstedt at oracle.com Wed Mar 22 16:21:55 2017 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 22 Mar 2017 17:21:55 +0100 Subject: Merging jdk9/hs and jdk9/dev In-Reply-To: <41F7F709-1613-4ECB-BD05-F41ED2543CA7@oracle.com> References: <41F7F709-1613-4ECB-BD05-F41ED2543CA7@oracle.com> Message-ID: <7AADBB39-78A8-4D73-B30D-00CC40C63317@oracle.com> Having heard no objections we?re looking to move forward with this. Jesper, can you please provide some details on what the process is etc.? Cheers, Mikael > On Mar 9, 2017, at 7:43 PM, Mikael Vidstedt wrote: > > > All, > > As you all know JDK 9 development is ramping down, and according to the JDK 9 release schedule [1] we are entering Ramp-down Phase 2 (RDP2) starting next week (March 16). While there is a small number of open issues it is reasonable to believe that the inflow of changes will approach zero over the next few days/weeks. Any changes going forward will effectively be for show stopper type bugs, and must as such be well tested and vetted before they are integrated. > > Given the state we?re in with the release, and in line with earlier efforts to streamline our integration processes, we propose closing down jdk9/hs [2] and moving the remaining JDK 9 (hotspot) work directly to jdk9/dev [3]. In addition to a reduction in integration overhead, this will also shorten the lead times of getting changes into master in the final latency-sensitive phase of the release. > > We propose that the final integration from jdk9/hs to jdk9/dev takes place on Thursday, March 23 after which jdk9/hs will be locked down/marked read-only to avoid any accidental integrations. Any work in flight would have to be rebased on jdk9/dev. Just like today, Hotspot integrations will still be made using JPRT. > > Earlier consolidation projects have gone smoothly, but we will be keeping an extra eye on things and ask for your patience as we work through any issues. > > Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Thursday, March 16, we will go ahead with the forest consolidation as described. > > [1] http://openjdk.java.net/projects/jdk9/ > [2] http://hg.openjdk.java.net/jdk9/hs > [3] http://hg.openjdk.java.net/jdk9/dev > From jesper.wilhelmsson at oracle.com Wed Mar 22 16:37:26 2017 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Wed, 22 Mar 2017 17:37:26 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Message-ID: <520E90B8-BCA9-415E-A198-E646F543C8F5@oracle.com> Hi, I use the OpenJDK NetBeans project most of the time and am very happy with it. I still think Mikael's initiative is a very good one since I don't believe in forcing developers to use some specific tools, but rather think that we should make the OpenJDK code available to as many developers as possible regardless of their IDE preference. The technology should adapt to humans, humans should not adapt to technology. Thanks, /Jesper > On 22 Mar 2017, at 16:02, Stanislav Smirnov wrote: > > Hi Mikael, > > why do not you try NetBeans that has openjdk project support out of the box? > common/nb_native/nbproject > > Best regards, > Stanislav Smirnov > > >> On 22 Mar 2017, at 17:21, Mikael Gerdin wrote: >> >> Hi all, >> >> I've finally grown tired of manually setting up a hand crafted Eclipse CDT configuration for the JVM sources and decided to share my progress towards improving the overall situation for JVM developers. >> >> To achieve better IDE support without having to add project generators for all different kinds of past or future IDEs I've decided to try to leverage CMake to do project generation. >> The idea is to have the JDK build system generate a CMakeLists.txt describing all the include paths and definitions required by an IDE to interpret the sources correctly. >> >> Several modern IDEs natively support CMake but we can also rely on the fact that the CMake build system has the ability to generate projects for a number of different IDEs. For information about which generators CMake supports see >> https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html >> for your CMake version. >> >> To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in the jdk10/sandbox forest: >> http://hg.openjdk.java.net/jdk10/sandbox/branches >> So far I've only made changes in the toplevel and hotspot repositories. >> I've written a short readme in the repo: >> http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html >> >> It would be great if people tried this out to see if it works on their setup but I don't expect it to work on Windows without changing the makefile to do path conversion. >> If we can make this work properly then perhaps we can get rid of the Visual Studio generator and rely on CMake to generate VS projects. >> >> It would also be great if someone from build-dev could give me some hints about how to do the file writing and "vardeps" thing properly. >> >> Thanks >> /Mikael > From jesper.wilhelmsson at oracle.com Wed Mar 22 17:01:50 2017 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Wed, 22 Mar 2017 18:01:50 +0100 Subject: Merging jdk9/hs and jdk9/dev In-Reply-To: <7AADBB39-78A8-4D73-B30D-00CC40C63317@oracle.com> References: <41F7F709-1613-4ECB-BD05-F41ED2543CA7@oracle.com> <7AADBB39-78A8-4D73-B30D-00CC40C63317@oracle.com> Message-ID: <66C7B4D5-BDB7-4AD5-89A1-BCB754397E4C@oracle.com> What will happen is the following: 1. On Thursday evening (PT) we will close jdk9/hs for pushes. This will be done before the Thursday nightly is started. To get some margin I'd say that if you haven't started your push before 5 pm PT, rebase and push to jdk9/dev instead. 2. On Friday I'll look at the nightly results and if there are no integration blockers I'll start the last PIT from jdk9/hs. 2.5. If there is anything blocking integration I will work with the relevant engineers to get the problem resolved and start the PIT asap. 3. Once the PIT is analysed and approved I'll push the last set of changes from jdk9/hs to jdk9/dev and ask for jdk9/hs to be made read only. As mentioned below, going forward all pushes that modifies JDK 9 Hotspot code should go to jdk9/dev but should still use JPRT. Nightly Hotspot testing will be moved to run on jdk9/dev. Thanks, /Jesper > On 22 Mar 2017, at 17:21, Mikael Vidstedt wrote: > > > Having heard no objections we?re looking to move forward with this. > > Jesper, can you please provide some details on what the process is etc.? > > Cheers, > Mikael > > >> On Mar 9, 2017, at 7:43 PM, Mikael Vidstedt wrote: >> >> >> All, >> >> As you all know JDK 9 development is ramping down, and according to the JDK 9 release schedule [1] we are entering Ramp-down Phase 2 (RDP2) starting next week (March 16). While there is a small number of open issues it is reasonable to believe that the inflow of changes will approach zero over the next few days/weeks. Any changes going forward will effectively be for show stopper type bugs, and must as such be well tested and vetted before they are integrated. >> >> Given the state we?re in with the release, and in line with earlier efforts to streamline our integration processes, we propose closing down jdk9/hs [2] and moving the remaining JDK 9 (hotspot) work directly to jdk9/dev [3]. In addition to a reduction in integration overhead, this will also shorten the lead times of getting changes into master in the final latency-sensitive phase of the release. >> >> We propose that the final integration from jdk9/hs to jdk9/dev takes place on Thursday, March 23 after which jdk9/hs will be locked down/marked read-only to avoid any accidental integrations. Any work in flight would have to be rebased on jdk9/dev. Just like today, Hotspot integrations will still be made using JPRT. >> >> Earlier consolidation projects have gone smoothly, but we will be keeping an extra eye on things and ask for your patience as we work through any issues. >> >> Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Thursday, March 16, we will go ahead with the forest consolidation as described. >> >> [1] http://openjdk.java.net/projects/jdk9/ >> [2] http://hg.openjdk.java.net/jdk9/hs >> [3] http://hg.openjdk.java.net/jdk9/dev >> > From ch-hagedorn at hispeed.ch Wed Mar 22 16:32:52 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Wed, 22 Mar 2017 17:32:52 +0100 Subject: Adding native method in custom jdk package Message-ID: <000c01d2a329$f4a11630$dde34290$@hispeed.ch> Hi, I have created a class Test.java in an own package test.lang inside the jdk ("jdk/src/share/classes/test/lang/Test.java"). Now I wanted to add a native method. I found out that my header was created inside" build/linux./jdk/gensrc_headers/test_lang_Test.h". So I have tried to just create a new file inside a new folder "jdk/src/share/native/test/lang/Test.c" including "test_lang_Test.h". >From there I do my JNICALL to the VM like it is done for example in "jdk/src/share/native/java/lang/System.c". However this still results in a UnsatisfiedLinkError. I am sure I have to do some more steps to link and add it to the build properly. What am I missing? Best regards, Christian From daniel.daugherty at oracle.com Wed Mar 22 17:18:01 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 22 Mar 2017 11:18:01 -0600 Subject: Merging jdk9/hs and jdk9/dev In-Reply-To: <66C7B4D5-BDB7-4AD5-89A1-BCB754397E4C@oracle.com> References: <41F7F709-1613-4ECB-BD05-F41ED2543CA7@oracle.com> <7AADBB39-78A8-4D73-B30D-00CC40C63317@oracle.com> <66C7B4D5-BDB7-4AD5-89A1-BCB754397E4C@oracle.com> Message-ID: <2d60d56e-2254-3124-7fcb-47176a4052ea@oracle.com> The plan sounds wonderful. When JDK9-hs is made read-only, can we also remove: http://hg.openjdk.java.net/jdk9/hs-rt The last push to that repo was done on 2016.04.14 so we're coming up on a year with the JDK9-hs-rt repo being read-only... Dan P.S. http://hg.openjdk.java.net/jdk9/hs-gc has been read-only for longer and http://hg.openjdk.java.net/jdk9/hs-comp has been read-only for less time... Might want to retire those repos also... On 3/22/17 11:01 AM, jesper.wilhelmsson at oracle.com wrote: > What will happen is the following: > > 1. On Thursday evening (PT) we will close jdk9/hs for pushes. This will be done before the Thursday nightly is started. To get some margin I'd say that if you haven't started your push before 5 pm PT, rebase and push to jdk9/dev instead. > > 2. On Friday I'll look at the nightly results and if there are no integration blockers I'll start the last PIT from jdk9/hs. > 2.5. If there is anything blocking integration I will work with the relevant engineers to get the problem resolved and start the PIT asap. > > 3. Once the PIT is analysed and approved I'll push the last set of changes from jdk9/hs to jdk9/dev and ask for jdk9/hs to be made read only. > > As mentioned below, going forward all pushes that modifies JDK 9 Hotspot code should go to jdk9/dev but should still use JPRT. > Nightly Hotspot testing will be moved to run on jdk9/dev. > > Thanks, > /Jesper > >> On 22 Mar 2017, at 17:21, Mikael Vidstedt wrote: >> >> >> Having heard no objections we?re looking to move forward with this. >> >> Jesper, can you please provide some details on what the process is etc.? >> >> Cheers, >> Mikael >> >> >>> On Mar 9, 2017, at 7:43 PM, Mikael Vidstedt wrote: >>> >>> >>> All, >>> >>> As you all know JDK 9 development is ramping down, and according to the JDK 9 release schedule [1] we are entering Ramp-down Phase 2 (RDP2) starting next week (March 16). While there is a small number of open issues it is reasonable to believe that the inflow of changes will approach zero over the next few days/weeks. Any changes going forward will effectively be for show stopper type bugs, and must as such be well tested and vetted before they are integrated. >>> >>> Given the state we?re in with the release, and in line with earlier efforts to streamline our integration processes, we propose closing down jdk9/hs [2] and moving the remaining JDK 9 (hotspot) work directly to jdk9/dev [3]. In addition to a reduction in integration overhead, this will also shorten the lead times of getting changes into master in the final latency-sensitive phase of the release. >>> >>> We propose that the final integration from jdk9/hs to jdk9/dev takes place on Thursday, March 23 after which jdk9/hs will be locked down/marked read-only to avoid any accidental integrations. Any work in flight would have to be rebased on jdk9/dev. Just like today, Hotspot integrations will still be made using JPRT. >>> >>> Earlier consolidation projects have gone smoothly, but we will be keeping an extra eye on things and ask for your patience as we work through any issues. >>> >>> Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Thursday, March 16, we will go ahead with the forest consolidation as described. >>> >>> [1] http://openjdk.java.net/projects/jdk9/ >>> [2] http://hg.openjdk.java.net/jdk9/hs >>> [3] http://hg.openjdk.java.net/jdk9/dev >>> > From bob.vandette at oracle.com Wed Mar 22 17:22:44 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 22 Mar 2017 13:22:44 -0400 Subject: Adding native method in custom jdk package In-Reply-To: <000c01d2a329$f4a11630$dde34290$@hispeed.ch> References: <000c01d2a329$f4a11630$dde34290$@hispeed.ch> Message-ID: Checkout this blog that provides a simple example of how to create and call a native function in a shared library. https://blogs.oracle.com/moonocean/entry/a_simple_example_of_jni Let me know if you have any questions, Bob. > On Mar 22, 2017, at 12:32 PM, Christian Hagedorn wrote: > > Hi, > > > > I have created a class Test.java in an own package test.lang inside the jdk > ("jdk/src/share/classes/test/lang/Test.java"). Now I wanted to add a native > method. > > I found out that my header was created inside" > build/linux./jdk/gensrc_headers/test_lang_Test.h". > > So I have tried to just create a new file inside a new folder > "jdk/src/share/native/test/lang/Test.c" including "test_lang_Test.h". > > From there I do my JNICALL to the VM like it is done for example in > "jdk/src/share/native/java/lang/System.c". > > However this still results in a UnsatisfiedLinkError. I am sure I have to do > some more steps to link and add it to the build properly. What am I missing? > > > > Best regards, > > Christian > > > From jesper.wilhelmsson at oracle.com Wed Mar 22 17:32:44 2017 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Wed, 22 Mar 2017 18:32:44 +0100 Subject: Merging jdk9/hs and jdk9/dev In-Reply-To: <2d60d56e-2254-3124-7fcb-47176a4052ea@oracle.com> References: <41F7F709-1613-4ECB-BD05-F41ED2543CA7@oracle.com> <7AADBB39-78A8-4D73-B30D-00CC40C63317@oracle.com> <66C7B4D5-BDB7-4AD5-89A1-BCB754397E4C@oracle.com> <2d60d56e-2254-3124-7fcb-47176a4052ea@oracle.com> Message-ID: <00B5F6EF-CA22-480D-A71D-4D3427E32BC3@oracle.com> I think that is a good idea and I'll be happy to include removal of the three former team repos when sending the request to lock hs. Thanks, /Jesper > On 22 Mar 2017, at 18:18, Daniel D. Daugherty wrote: > > The plan sounds wonderful. > > When JDK9-hs is made read-only, can we also remove: > > http://hg.openjdk.java.net/jdk9/hs-rt > > The last push to that repo was done on 2016.04.14 so we're coming > up on a year with the JDK9-hs-rt repo being read-only... > > Dan > > P.S. > http://hg.openjdk.java.net/jdk9/hs-gc has been read-only for longer > and http://hg.openjdk.java.net/jdk9/hs-comp has been read-only for > less time... Might want to retire those repos also... > > > On 3/22/17 11:01 AM, jesper.wilhelmsson at oracle.com wrote: >> What will happen is the following: >> >> 1. On Thursday evening (PT) we will close jdk9/hs for pushes. This will be done before the Thursday nightly is started. To get some margin I'd say that if you haven't started your push before 5 pm PT, rebase and push to jdk9/dev instead. >> >> 2. On Friday I'll look at the nightly results and if there are no integration blockers I'll start the last PIT from jdk9/hs. >> 2.5. If there is anything blocking integration I will work with the relevant engineers to get the problem resolved and start the PIT asap. >> >> 3. Once the PIT is analysed and approved I'll push the last set of changes from jdk9/hs to jdk9/dev and ask for jdk9/hs to be made read only. >> >> As mentioned below, going forward all pushes that modifies JDK 9 Hotspot code should go to jdk9/dev but should still use JPRT. >> Nightly Hotspot testing will be moved to run on jdk9/dev. >> >> Thanks, >> /Jesper >> >>> On 22 Mar 2017, at 17:21, Mikael Vidstedt wrote: >>> >>> >>> Having heard no objections we?re looking to move forward with this. >>> >>> Jesper, can you please provide some details on what the process is etc.? >>> >>> Cheers, >>> Mikael >>> >>> >>>> On Mar 9, 2017, at 7:43 PM, Mikael Vidstedt wrote: >>>> >>>> >>>> All, >>>> >>>> As you all know JDK 9 development is ramping down, and according to the JDK 9 release schedule [1] we are entering Ramp-down Phase 2 (RDP2) starting next week (March 16). While there is a small number of open issues it is reasonable to believe that the inflow of changes will approach zero over the next few days/weeks. Any changes going forward will effectively be for show stopper type bugs, and must as such be well tested and vetted before they are integrated. >>>> >>>> Given the state we?re in with the release, and in line with earlier efforts to streamline our integration processes, we propose closing down jdk9/hs [2] and moving the remaining JDK 9 (hotspot) work directly to jdk9/dev [3]. In addition to a reduction in integration overhead, this will also shorten the lead times of getting changes into master in the final latency-sensitive phase of the release. >>>> >>>> We propose that the final integration from jdk9/hs to jdk9/dev takes place on Thursday, March 23 after which jdk9/hs will be locked down/marked read-only to avoid any accidental integrations. Any work in flight would have to be rebased on jdk9/dev. Just like today, Hotspot integrations will still be made using JPRT. >>>> >>>> Earlier consolidation projects have gone smoothly, but we will be keeping an extra eye on things and ask for your patience as we work through any issues. >>>> >>>> Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Thursday, March 16, we will go ahead with the forest consolidation as described. >>>> >>>> [1] http://openjdk.java.net/projects/jdk9/ >>>> [2] http://hg.openjdk.java.net/jdk9/hs >>>> [3] http://hg.openjdk.java.net/jdk9/dev >>>> >> > From thomas.stuefe at gmail.com Wed Mar 22 17:34:41 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 22 Mar 2017 18:34:41 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Message-ID: Hi Stanislav, last time I checked, there was no support for netbeans on Windows, is that still the case? Thanks, Thomas On Wed, Mar 22, 2017 at 4:02 PM, Stanislav Smirnov < stanislav.smirnov at oracle.com> wrote: > Hi Mikael, > > why do not you try NetBeans that has openjdk project support out of the > box? > common/nb_native/nbproject > > Best regards, > Stanislav Smirnov > > > > > > > On 22 Mar 2017, at 17:21, Mikael Gerdin > wrote: > > > > Hi all, > > > > I've finally grown tired of manually setting up a hand crafted Eclipse > CDT configuration for the JVM sources and decided to share my progress > towards improving the overall situation for JVM developers. > > > > To achieve better IDE support without having to add project generators > for all different kinds of past or future IDEs I've decided to try to > leverage CMake to do project generation. > > The idea is to have the JDK build system generate a CMakeLists.txt > describing all the include paths and definitions required by an IDE to > interpret the sources correctly. > > > > Several modern IDEs natively support CMake but we can also rely on the > fact that the CMake build system has the ability to generate projects for a > number of different IDEs. For information about which generators CMake > supports see > > https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html > > for your CMake version. > > > > To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in > the jdk10/sandbox forest: > > http://hg.openjdk.java.net/jdk10/sandbox/branches > > So far I've only made changes in the toplevel and hotspot repositories. > > I've written a short readme in the repo: > > http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK- > 8177329-cmake-branch/README-cmake.html > > > > It would be great if people tried this out to see if it works on their > setup but I don't expect it to work on Windows without changing the > makefile to do path conversion. > > If we can make this work properly then perhaps we can get rid of the > Visual Studio generator and rely on CMake to generate VS projects. > > > > It would also be great if someone from build-dev could give me some > hints about how to do the file writing and "vardeps" thing properly. > > > > Thanks > > /Mikael > > From stanislav.smirnov at oracle.com Wed Mar 22 17:40:47 2017 From: stanislav.smirnov at oracle.com (Stanislav Smirnov) Date: Wed, 22 Mar 2017 20:40:47 +0300 Subject: C/C++ IDE support for HotSpot In-Reply-To: References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Message-ID: <04E9C8CE-F189-44D8-9AC8-E19B70508399@oracle.com> Hi Thomas, according to https://netbeans.org/downloads/ there is Windows platform with C/C++ support if I understood you correctly. Best regards, Stanislav Smirnov > On 22 Mar 2017, at 20:34, Thomas St?fe wrote: > > Hi Stanislav, > > last time I checked, there was no support for netbeans on Windows, is that still the case? > > Thanks, Thomas > > On Wed, Mar 22, 2017 at 4:02 PM, Stanislav Smirnov > wrote: > Hi Mikael, > > why do not you try NetBeans that has openjdk project support out of the box? > common/nb_native/nbproject > > Best regards, > Stanislav Smirnov > > > > > > > On 22 Mar 2017, at 17:21, Mikael Gerdin > wrote: > > > > Hi all, > > > > I've finally grown tired of manually setting up a hand crafted Eclipse CDT configuration for the JVM sources and decided to share my progress towards improving the overall situation for JVM developers. > > > > To achieve better IDE support without having to add project generators for all different kinds of past or future IDEs I've decided to try to leverage CMake to do project generation. > > The idea is to have the JDK build system generate a CMakeLists.txt describing all the include paths and definitions required by an IDE to interpret the sources correctly. > > > > Several modern IDEs natively support CMake but we can also rely on the fact that the CMake build system has the ability to generate projects for a number of different IDEs. For information about which generators CMake supports see > > https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html > > for your CMake version. > > > > To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in the jdk10/sandbox forest: > > http://hg.openjdk.java.net/jdk10/sandbox/branches > > So far I've only made changes in the toplevel and hotspot repositories. > > I've written a short readme in the repo: > > http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html > > > > It would be great if people tried this out to see if it works on their setup but I don't expect it to work on Windows without changing the makefile to do path conversion. > > If we can make this work properly then perhaps we can get rid of the Visual Studio generator and rely on CMake to generate VS projects. > > > > It would also be great if someone from build-dev could give me some hints about how to do the file writing and "vardeps" thing properly. > > > > Thanks > > /Mikael > > From jesper.wilhelmsson at oracle.com Wed Mar 22 17:45:08 2017 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Wed, 22 Mar 2017 18:45:08 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Message-ID: <4D364C90-F069-4113-84ED-18BCC591F600@oracle.com> Hi, The project in common/nb_native/nbproject does not have a build configuration for Windows. The project itself should still be usable for browsing the code, but it might require some changes to make it build correctly. If you have a NetBeans / SolarisStudio installation on Windows and want to make the changes required to add a build configuration for Windows I'd be happy to sponsor the change. Thanks, /Jesper > On 22 Mar 2017, at 18:34, Thomas St?fe wrote: > > Hi Stanislav, > > last time I checked, there was no support for netbeans on Windows, is that > still the case? > > Thanks, Thomas > > On Wed, Mar 22, 2017 at 4:02 PM, Stanislav Smirnov < > stanislav.smirnov at oracle.com> wrote: > >> Hi Mikael, >> >> why do not you try NetBeans that has openjdk project support out of the >> box? >> common/nb_native/nbproject >> >> Best regards, >> Stanislav Smirnov >> >> >> >> >> >>> On 22 Mar 2017, at 17:21, Mikael Gerdin >> wrote: >>> >>> Hi all, >>> >>> I've finally grown tired of manually setting up a hand crafted Eclipse >> CDT configuration for the JVM sources and decided to share my progress >> towards improving the overall situation for JVM developers. >>> >>> To achieve better IDE support without having to add project generators >> for all different kinds of past or future IDEs I've decided to try to >> leverage CMake to do project generation. >>> The idea is to have the JDK build system generate a CMakeLists.txt >> describing all the include paths and definitions required by an IDE to >> interpret the sources correctly. >>> >>> Several modern IDEs natively support CMake but we can also rely on the >> fact that the CMake build system has the ability to generate projects for a >> number of different IDEs. For information about which generators CMake >> supports see >>> https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html >>> for your CMake version. >>> >>> To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in >> the jdk10/sandbox forest: >>> http://hg.openjdk.java.net/jdk10/sandbox/branches >>> So far I've only made changes in the toplevel and hotspot repositories. >>> I've written a short readme in the repo: >>> http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK- >> 8177329-cmake-branch/README-cmake.html >>> >>> It would be great if people tried this out to see if it works on their >> setup but I don't expect it to work on Windows without changing the >> makefile to do path conversion. >>> If we can make this work properly then perhaps we can get rid of the >> Visual Studio generator and rely on CMake to generate VS projects. >>> >>> It would also be great if someone from build-dev could give me some >> hints about how to do the file writing and "vardeps" thing properly. >>> >>> Thanks >>> /Mikael >> >> From thomas.stuefe at gmail.com Wed Mar 22 17:51:30 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 22 Mar 2017 18:51:30 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: <4D364C90-F069-4113-84ED-18BCC591F600@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> <4D364C90-F069-4113-84ED-18BCC591F600@oracle.com> Message-ID: Hi Jesper, thank you. Yes, this was my experience last time I tried it, that the windows build configuration was missing and that the Indexer did not work as well as expected (did, of course, not find any system headers etc). I did not feel like putting work into it, so I went back to CDT. I might look at it again though. Kind Regards, Thomas On Wed, Mar 22, 2017 at 6:45 PM, wrote: > Hi, > > The project in common/nb_native/nbproject does not have a build > configuration for Windows. The project itself should still be usable for > browsing the code, but it might require some changes to make it build > correctly. If you have a NetBeans / SolarisStudio installation on Windows > and want to make the changes required to add a build configuration for > Windows I'd be happy to sponsor the change. > > Thanks, > /Jesper > > > On 22 Mar 2017, at 18:34, Thomas St?fe wrote: > > > > Hi Stanislav, > > > > last time I checked, there was no support for netbeans on Windows, is > that > > still the case? > > > > Thanks, Thomas > > > > On Wed, Mar 22, 2017 at 4:02 PM, Stanislav Smirnov < > > stanislav.smirnov at oracle.com> wrote: > > > >> Hi Mikael, > >> > >> why do not you try NetBeans that has openjdk project support out of the > >> box? > >> common/nb_native/nbproject > >> > >> Best regards, > >> Stanislav Smirnov > >> > >> > >> > >> > >> > >>> On 22 Mar 2017, at 17:21, Mikael Gerdin > >> wrote: > >>> > >>> Hi all, > >>> > >>> I've finally grown tired of manually setting up a hand crafted Eclipse > >> CDT configuration for the JVM sources and decided to share my progress > >> towards improving the overall situation for JVM developers. > >>> > >>> To achieve better IDE support without having to add project generators > >> for all different kinds of past or future IDEs I've decided to try to > >> leverage CMake to do project generation. > >>> The idea is to have the JDK build system generate a CMakeLists.txt > >> describing all the include paths and definitions required by an IDE to > >> interpret the sources correctly. > >>> > >>> Several modern IDEs natively support CMake but we can also rely on the > >> fact that the CMake build system has the ability to generate projects > for a > >> number of different IDEs. For information about which generators CMake > >> supports see > >>> https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html > >>> for your CMake version. > >>> > >>> To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in > >> the jdk10/sandbox forest: > >>> http://hg.openjdk.java.net/jdk10/sandbox/branches > >>> So far I've only made changes in the toplevel and hotspot repositories. > >>> I've written a short readme in the repo: > >>> http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK- > >> 8177329-cmake-branch/README-cmake.html > >>> > >>> It would be great if people tried this out to see if it works on their > >> setup but I don't expect it to work on Windows without changing the > >> makefile to do path conversion. > >>> If we can make this work properly then perhaps we can get rid of the > >> Visual Studio generator and rely on CMake to generate VS projects. > >>> > >>> It would also be great if someone from build-dev could give me some > >> hints about how to do the file writing and "vardeps" thing properly. > >>> > >>> Thanks > >>> /Mikael > >> > >> > > From thomas.stuefe at gmail.com Wed Mar 22 17:52:21 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 22 Mar 2017 18:52:21 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: <04E9C8CE-F189-44D8-9AC8-E19B70508399@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> <04E9C8CE-F189-44D8-9AC8-E19B70508399@oracle.com> Message-ID: Hi Stanislav, thanks. See my answer to Jesper. I was using Netbeans on Windows and was missing Windows Configuration files. Sorry for being unclear. Kind Regards, Thomas On Wed, Mar 22, 2017 at 6:40 PM, Stanislav Smirnov < stanislav.smirnov at oracle.com> wrote: > Hi Thomas, > > according to https://netbeans.org/downloads/ there is Windows platform > with C/C++ support if I understood you correctly. > > Best regards, > Stanislav Smirnov > > > > > > On 22 Mar 2017, at 20:34, Thomas St?fe wrote: > > Hi Stanislav, > > last time I checked, there was no support for netbeans on Windows, is that > still the case? > > Thanks, Thomas > > On Wed, Mar 22, 2017 at 4:02 PM, Stanislav Smirnov < > stanislav.smirnov at oracle.com> wrote: > >> Hi Mikael, >> >> why do not you try NetBeans that has openjdk project support out of the >> box? >> common/nb_native/nbproject >> >> Best regards, >> Stanislav Smirnov >> >> >> >> >> >> > On 22 Mar 2017, at 17:21, Mikael Gerdin >> wrote: >> > >> > Hi all, >> > >> > I've finally grown tired of manually setting up a hand crafted Eclipse >> CDT configuration for the JVM sources and decided to share my progress >> towards improving the overall situation for JVM developers. >> > >> > To achieve better IDE support without having to add project generators >> for all different kinds of past or future IDEs I've decided to try to >> leverage CMake to do project generation. >> > The idea is to have the JDK build system generate a CMakeLists.txt >> describing all the include paths and definitions required by an IDE to >> interpret the sources correctly. >> > >> > Several modern IDEs natively support CMake but we can also rely on the >> fact that the CMake build system has the ability to generate projects for a >> number of different IDEs. For information about which generators CMake >> supports see >> > https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html >> >> > for your CMake version. >> > >> > To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in >> the jdk10/sandbox forest: >> > http://hg.openjdk.java.net/jdk10/sandbox/branches >> > So far I've only made changes in the toplevel and hotspot repositories. >> > I've written a short readme in the repo: >> > http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-817732 >> 9-cmake-branch/README-cmake.html >> > >> > It would be great if people tried this out to see if it works on their >> setup but I don't expect it to work on Windows without changing the >> makefile to do path conversion. >> > If we can make this work properly then perhaps we can get rid of the >> Visual Studio generator and rely on CMake to generate VS projects. >> > >> > It would also be great if someone from build-dev could give me some >> hints about how to do the file writing and "vardeps" thing properly. >> > >> > Thanks >> > /Mikael >> >> > > From dean.long at oracle.com Wed Mar 22 18:01:55 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 22 Mar 2017 11:01:55 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> Message-ID: Vladimir, don't you need to replace checkIndex with checkOffset in indexOf and lastIndexOf, so that we allow count == length? dl On 3/22/17 8:35 AM, Vladimir Ivanov wrote: >>>> So are we convinced that the proposed changes will never lead to a >>>> crash due to a missing or incorrect bounds check, due to a racy use of >>>> an unsynchronized ASB instance e.g. StringBuilder? >>> >>> If only we had a static analysis tool that could tell us if the code is >>> safe. Because we don't, in my initial changeset, we always take a >>> snapshot of the ASB fields by passing those field values to StringUTF16 >>> before doing checks on them. And I wrote a test to make sure that >>> those >>> StringUTF16 interfaces are catching all the underflows and overflows I >>> could imagine, and I added verification code to detect when a check was >>> missed. >>> >>> However, all the reviewers have requested to minimize the amount of >>> changes. In Vladimir's version, if there is a missing check somewhere, >>> then yes it could lead to a crash. > > I'd like to point out that asserts and verification code are disabled > by default. They are invaluable during problem diagnosis, but don't > help at all from defence-in-depth perspective. > > But I agree that it's easier to reason about and test the initial > version of the fix. > >> I wonder if the reviewers have fully realized the potential impact here? >> This has exposed a flaw in the way intrinsics are used from core >> classes. > > FTR here are the checks I omitted in the minimized version (modulo > separation of indexOf/lastIndexOf for trusted/non-trusted callers): > > http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ > > Other than that, the difference is mainly about undoing refactorings > and removing verification logic (asserts + checks in the JVM). > > There are still unsafe accesses which are considered safe in both > versions (see StringUTF16.Trusted usages in the initial version [1]). > > We used to provide safe wrappers for unsafe intrinsics which makes it > much easier to reason about code correctness. I'd like to see compact > string code refactored that way and IMO the initial version by Dean is > a big step in the right direction. > > I still prefer to see a point fix in 9 and major refactoring happening > in 10, but I'll leave the decision on how to proceed with the fix to > core-libs folks. After finishing the exercise minimizing the fix, I'm > much more comfortable with the initial fix [1] (though there are > changes I consider excessive). > > Best regards, > Vladimir Ivanov > > [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 > >>>>>> Some clarifications: >>>>>> >>>>>> ============ >>>>>> src/java.base/share/classes/java/lang/String.java: >>>>>> >>>>>> The bounds check is needed only in String.nonSyncContentEquals >>>>>> when it >>>>>> extracts info from AbstractStringBuilder. I don't see how out of >>>>>> bounds access can happen in String.contentEquals: >>>>>> if (n != length()) { >>>>>> return false; >>>>>> } >>>>>> ... >>>>>> for (int i = 0; i < n; i++) { >>>>>> if (StringUTF16.getChar(val, i) != cs.charAt(i)) { >>>>>> >>>>> >>>>> OK. >>>>> >>>>>> ============ >>>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>>> >>>>>> I think bounds checks in StringConcatHelper.prepend() are skipped >>>>>> intentionally, since java.lang.invoke.StringConcatFactory constructs >>>>>> method handle chains which already contain bounds checks: array >>>>>> length >>>>>> is precomputed based on argument values and all accesses are >>>>>> guaranteed to be in bounds. >>>>>> >>>>> >>>>> This is calling the trusted version of getChars() with no bounds >>>>> checks. It was a little more obvious when I had the Trusted inner >>>>> class. >>>>> >>>>>> ============ >>>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>> >>>>>> + static void putChar(byte[] val, int index, int c) { >>>>>> + assert index >= 0 && index < length(val) : "Trusted caller >>>>>> missed bounds check"; >>>>>> >>>>>> Unfortunately, asserts can affect inlining decisions (since they >>>>>> increase bytecode size). In order to minimize possible performance >>>>>> impact, I suggest to remove them from the fix targeting 9. >>>>>> >>>>> >>>>> Sure. >>>>> >>>>>> ============ >>>>>> private static int indexOfSupplementary(byte[] value, int >>>>>> ch, int >>>>>> fromIndex, int max) { >>>>>> if (Character.isValidCodePoint(ch)) { >>>>>> final char hi = Character.highSurrogate(ch); >>>>>> final char lo = Character.lowSurrogate(ch); >>>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>>> >>>>>> The check is redundant here. fromIndex & max are always inbounds by >>>>>> construction: >>>>>> >>>>>> public static int indexOf(byte[] value, int ch, int fromIndex) { >>>>>> int max = value.length >> 1; >>>>>> if (fromIndex < 0) { >>>>>> fromIndex = 0; >>>>>> } else if (fromIndex >= max) { >>>>>> // Note: fromIndex might be near -1>>>1. >>>>>> return -1; >>>>>> } >>>>>> ... >>>>>> return indexOfSupplementary(value, ch, fromIndex, max); >>>>>> >>>>> >>>>> OK. >>>>> >>>>>> ============ >>>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >>>>>> ABS.value & ABS.count. After that, all accesses should be >>>>>> inbounds by >>>>>> construction (in String.indexOf/lastIndexOf): >>>>>> >>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>> static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, >>>>>> String tgtStr, int fromIndex) { >>>>>> >>>>>> int rightIndex = srcCount - tgtCount; >>>>>> if (fromIndex > rightIndex) { >>>>>> fromIndex = rightIndex; >>>>>> } >>>>>> if (fromIndex < 0) { >>>>>> return -1; >>>>>> } >>>>>> >>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>>> byte[] tgt, int tgtCount, int >>>>>> fromIndex) { >>>>>> int min = tgtCount - 1; >>>>>> int i = min + fromIndex; >>>>>> int strLastIndex = tgtCount - 1; >>>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>>> >>>>>> startSearchForLastChar: >>>>>> while (true) { >>>>>> while (i >= min && getChar(src, i) != strLastChar) { >>>>>> >>>>>> There are 2 places: >>>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - inbound >>>>>> >>>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>>> min = tgtCount - 1 >>>>>> rightIndex = srcCount - tgtCount >>>>>> fromIndex <= rightIndex >>>>>> >>>>>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount >>>>>> - 1) >>>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>>> >>>>>> Hence, should be covered by the check on count & value: >>>>>> public int lastIndexOf(String str, int fromIndex) { >>>>>> + byte[] value = this.value; >>>>>> + int count = this.count; >>>>>> + byte coder = this.coder; >>>>>> + checkIndex(count, value.length >> coder); >>>>>> return String.lastIndexOf(value, coder, count, str, >>>>>> fromIndex); >>>>>> } >>>>>> >>>>> >>>>> OK, I will go with your version if it's OK with Sherman. >>>>> >>>>> dl >>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>>> >>>>>>>>>> I have the same concern. Can we fix the immediate problem in >>>>>>>>>> 9 and >>>>>>>>>> integrate verification logic in 10? >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK, Tobias is suggesting having verification logic only inside >>>>>>>>> the >>>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>>> >>>>>>>> Yes and put them back in 10. >>>>>>>> >>>>>>>>> I'm OK with removing all the verification, but that won't reduce >>>>>>>>> the >>>>>>>>> library changes much. I could undo the renaming to >>>>>>>>> Trusted.getChar, but >>>>>>>>> we would still have the bounds checks moved into StringUTF16. >>>>>>>> >>>>>>>> I suggest to go with a point fix for 9: just add missing range >>>>>>>> checks. >>>>>>> >>>>> >>> From ch-hagedorn at hispeed.ch Wed Mar 22 18:07:32 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Wed, 22 Mar 2017 19:07:32 +0100 Subject: Adding native method in custom jdk package In-Reply-To: References: <000c01d2a329$f4a11630$dde34290$@hispeed.ch> Message-ID: <001c01d2a337$2e31a560$8a94f020$@hispeed.ch> Thanks for your quick help. That got me on the right track, it does work now. I added my path to the LIBJAVA_SRC_DIRS in the ?jdk/make/lib/CoreLibraries.gmk? file. Best regards, Christian Von: Bob Vandette [mailto:bob.vandette at oracle.com] Gesendet: Mittwoch, 22. M?rz 2017 18:23 An: Christian Hagedorn Cc: hotspot-dev at openjdk.java.net Betreff: Re: Adding native method in custom jdk package Checkout this blog that provides a simple example of how to create and call a native function in a shared library. https://blogs.oracle.com/moonocean/entry/a_simple_example_of_jni Let me know if you have any questions, Bob. On Mar 22, 2017, at 12:32 PM, Christian Hagedorn wrote: Hi, I have created a class Test.java in an own package test.lang inside the jdk ("jdk/src/share/classes/test/lang/Test.java"). Now I wanted to add a native method. I found out that my header was created inside" build/linux./jdk/gensrc_headers/test_lang_Test.h". So I have tried to just create a new file inside a new folder "jdk/src/share/native/test/lang/Test.c" including "test_lang_Test.h". >From there I do my JNICALL to the VM like it is done for example in "jdk/src/share/native/java/lang/System.c". However this still results in a UnsatisfiedLinkError. I am sure I have to do some more steps to link and add it to the build properly. What am I missing? Best regards, Christian From sgehwolf at redhat.com Wed Mar 22 18:15:34 2017 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 22 Mar 2017 19:15:34 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Message-ID: <1490206534.3715.14.camel@redhat.com> Hi, On Wed, 2017-03-22 at 15:21 +0100, Mikael Gerdin wrote: > Hi all, > > I've finally grown tired of manually setting up a hand crafted Eclipse? > CDT configuration for the JVM sources and decided to share my progress? > towards improving the overall situation for JVM developers. > > To achieve better IDE support without having to add project generators? > for all different kinds of past or future IDEs I've decided to try to? > leverage CMake to do project generation. > The idea is to have the JDK build system generate a CMakeLists.txt? > describing all the include paths and definitions required by an IDE to? > interpret the sources correctly. > > Several modern IDEs natively support CMake but we can also rely on the? > fact that the CMake build system has the ability to generate projects? > for a number of different IDEs. For information about which generators? > CMake supports see > https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html > for your CMake version. > > To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in? > the jdk10/sandbox forest: > http://hg.openjdk.java.net/jdk10/sandbox/branches > So far I've only made changes in the toplevel and hotspot repositories. > I've written a short readme in the repo: > http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html > > It would be great if people tried this out to see if it works on their? > setup but I don't expect it to work on Windows without changing the? > makefile to do path conversion. > If we can make this work properly then perhaps we can get rid of the? > Visual Studio generator and rely on CMake to generate VS projects. > > It would also be great if someone from build-dev could give me some? > hints about how to do the file writing and "vardeps" thing properly. I, for one, wholeheartedly support this effort. Hopefully this will get in. Kudos! Note: I didn't have time to test this yet. Thanks, Severin From dean.long at oracle.com Wed Mar 22 18:28:18 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 22 Mar 2017 11:28:18 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> Message-ID: <6a3d62b5-fbea-e982-d7d5-087f9badb028@oracle.com> Also, it looks like the changes I made to ASB.appendChars(char[] s, int off, int end) are not needed. dl On 3/22/17 11:01 AM, dean.long at oracle.com wrote: > Vladimir, don't you need to replace checkIndex with checkOffset in > indexOf and lastIndexOf, so that we allow count == length? > > dl > > > On 3/22/17 8:35 AM, Vladimir Ivanov wrote: >>>>> So are we convinced that the proposed changes will never lead to a >>>>> crash due to a missing or incorrect bounds check, due to a racy >>>>> use of >>>>> an unsynchronized ASB instance e.g. StringBuilder? >>>> >>>> If only we had a static analysis tool that could tell us if the >>>> code is >>>> safe. Because we don't, in my initial changeset, we always take a >>>> snapshot of the ASB fields by passing those field values to >>>> StringUTF16 >>>> before doing checks on them. And I wrote a test to make sure that >>>> those >>>> StringUTF16 interfaces are catching all the underflows and overflows I >>>> could imagine, and I added verification code to detect when a check >>>> was >>>> missed. >>>> >>>> However, all the reviewers have requested to minimize the amount of >>>> changes. In Vladimir's version, if there is a missing check >>>> somewhere, >>>> then yes it could lead to a crash. >> >> I'd like to point out that asserts and verification code are disabled >> by default. They are invaluable during problem diagnosis, but don't >> help at all from defence-in-depth perspective. >> >> But I agree that it's easier to reason about and test the initial >> version of the fix. >> >>> I wonder if the reviewers have fully realized the potential impact >>> here? >>> This has exposed a flaw in the way intrinsics are used from core >>> classes. >> >> FTR here are the checks I omitted in the minimized version (modulo >> separation of indexOf/lastIndexOf for trusted/non-trusted callers): >> >> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ >> >> Other than that, the difference is mainly about undoing refactorings >> and removing verification logic (asserts + checks in the JVM). >> >> There are still unsafe accesses which are considered safe in both >> versions (see StringUTF16.Trusted usages in the initial version [1]). >> >> We used to provide safe wrappers for unsafe intrinsics which makes it >> much easier to reason about code correctness. I'd like to see compact >> string code refactored that way and IMO the initial version by Dean >> is a big step in the right direction. >> >> I still prefer to see a point fix in 9 and major refactoring >> happening in 10, but I'll leave the decision on how to proceed with >> the fix to core-libs folks. After finishing the exercise minimizing >> the fix, I'm much more comfortable with the initial fix [1] (though >> there are changes I consider excessive). >> >> Best regards, >> Vladimir Ivanov >> >> [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 >> >>>>>>> Some clarifications: >>>>>>> >>>>>>> ============ >>>>>>> src/java.base/share/classes/java/lang/String.java: >>>>>>> >>>>>>> The bounds check is needed only in String.nonSyncContentEquals >>>>>>> when it >>>>>>> extracts info from AbstractStringBuilder. I don't see how out of >>>>>>> bounds access can happen in String.contentEquals: >>>>>>> if (n != length()) { >>>>>>> return false; >>>>>>> } >>>>>>> ... >>>>>>> for (int i = 0; i < n; i++) { >>>>>>> if (StringUTF16.getChar(val, i) != cs.charAt(i)) { >>>>>>> >>>>>> >>>>>> OK. >>>>>> >>>>>>> ============ >>>>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>>>> >>>>>>> I think bounds checks in StringConcatHelper.prepend() are skipped >>>>>>> intentionally, since java.lang.invoke.StringConcatFactory >>>>>>> constructs >>>>>>> method handle chains which already contain bounds checks: array >>>>>>> length >>>>>>> is precomputed based on argument values and all accesses are >>>>>>> guaranteed to be in bounds. >>>>>>> >>>>>> >>>>>> This is calling the trusted version of getChars() with no bounds >>>>>> checks. It was a little more obvious when I had the Trusted inner >>>>>> class. >>>>>> >>>>>>> ============ >>>>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>> >>>>>>> + static void putChar(byte[] val, int index, int c) { >>>>>>> + assert index >= 0 && index < length(val) : "Trusted caller >>>>>>> missed bounds check"; >>>>>>> >>>>>>> Unfortunately, asserts can affect inlining decisions (since they >>>>>>> increase bytecode size). In order to minimize possible performance >>>>>>> impact, I suggest to remove them from the fix targeting 9. >>>>>>> >>>>>> >>>>>> Sure. >>>>>> >>>>>>> ============ >>>>>>> private static int indexOfSupplementary(byte[] value, int >>>>>>> ch, int >>>>>>> fromIndex, int max) { >>>>>>> if (Character.isValidCodePoint(ch)) { >>>>>>> final char hi = Character.highSurrogate(ch); >>>>>>> final char lo = Character.lowSurrogate(ch); >>>>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>>>> >>>>>>> The check is redundant here. fromIndex & max are always inbounds by >>>>>>> construction: >>>>>>> >>>>>>> public static int indexOf(byte[] value, int ch, int >>>>>>> fromIndex) { >>>>>>> int max = value.length >> 1; >>>>>>> if (fromIndex < 0) { >>>>>>> fromIndex = 0; >>>>>>> } else if (fromIndex >= max) { >>>>>>> // Note: fromIndex might be near -1>>>1. >>>>>>> return -1; >>>>>>> } >>>>>>> ... >>>>>>> return indexOfSupplementary(value, ch, fromIndex, max); >>>>>>> >>>>>> >>>>>> OK. >>>>>> >>>>>>> ============ >>>>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >>>>>>> ABS.value & ABS.count. After that, all accesses should be >>>>>>> inbounds by >>>>>>> construction (in String.indexOf/lastIndexOf): >>>>>>> >>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>> static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, >>>>>>> String tgtStr, int fromIndex) { >>>>>>> >>>>>>> int rightIndex = srcCount - tgtCount; >>>>>>> if (fromIndex > rightIndex) { >>>>>>> fromIndex = rightIndex; >>>>>>> } >>>>>>> if (fromIndex < 0) { >>>>>>> return -1; >>>>>>> } >>>>>>> >>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>>>> byte[] tgt, int tgtCount, int >>>>>>> fromIndex) { >>>>>>> int min = tgtCount - 1; >>>>>>> int i = min + fromIndex; >>>>>>> int strLastIndex = tgtCount - 1; >>>>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>>>> >>>>>>> startSearchForLastChar: >>>>>>> while (true) { >>>>>>> while (i >= min && getChar(src, i) != strLastChar) { >>>>>>> >>>>>>> There are 2 places: >>>>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - >>>>>>> inbound >>>>>>> >>>>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>>>> min = tgtCount - 1 >>>>>>> rightIndex = srcCount - tgtCount >>>>>>> fromIndex <= rightIndex >>>>>>> >>>>>>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount >>>>>>> - 1) >>>>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>>>> >>>>>>> Hence, should be covered by the check on count & value: >>>>>>> public int lastIndexOf(String str, int fromIndex) { >>>>>>> + byte[] value = this.value; >>>>>>> + int count = this.count; >>>>>>> + byte coder = this.coder; >>>>>>> + checkIndex(count, value.length >> coder); >>>>>>> return String.lastIndexOf(value, coder, count, str, >>>>>>> fromIndex); >>>>>>> } >>>>>>> >>>>>> >>>>>> OK, I will go with your version if it's OK with Sherman. >>>>>> >>>>>> dl >>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>>>> >>>>>>>>>>> I have the same concern. Can we fix the immediate problem in >>>>>>>>>>> 9 and >>>>>>>>>>> integrate verification logic in 10? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, Tobias is suggesting having verification logic only >>>>>>>>>> inside the >>>>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>>>> >>>>>>>>> Yes and put them back in 10. >>>>>>>>> >>>>>>>>>> I'm OK with removing all the verification, but that won't reduce >>>>>>>>>> the >>>>>>>>>> library changes much. I could undo the renaming to >>>>>>>>>> Trusted.getChar, but >>>>>>>>>> we would still have the bounds checks moved into StringUTF16. >>>>>>>>> >>>>>>>>> I suggest to go with a point fix for 9: just add missing range >>>>>>>>> checks. >>>>>>>> >>>>>> >>>> > From igor.ignatyev at oracle.com Wed Mar 22 20:09:53 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 22 Mar 2017 13:09:53 -0700 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests Message-ID: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html > 40 lines changed: 26 ins; 13 del; 1 mod; Hi all, could you please review this changeset which fixes in a few jdk_svc tests which were missed by JDK-8176176[1]? testing: :jdk_svc tests webrev: http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8177374 [1] https://bugs.openjdk.java.net/browse/JDK-8176176 Thanks, -- Igor From serguei.spitsyn at oracle.com Wed Mar 22 20:16:11 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 22 Mar 2017 13:16:11 -0700 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests In-Reply-To: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> References: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> Message-ID: Hi Igor, It looks good. Thanks, Serguei On 3/22/17 13:09, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html >> 40 lines changed: 26 ins; 13 del; 1 mod; > Hi all, > > could you please review this changeset which fixes in a few jdk_svc tests which were missed by JDK-8176176[1]? > > testing: :jdk_svc tests > webrev: http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8177374 > > [1] https://bugs.openjdk.java.net/browse/JDK-8176176 > > Thanks, > -- Igor From vladimir.x.ivanov at oracle.com Wed Mar 22 20:49:07 2017 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 22 Mar 2017 23:49:07 +0300 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <6a3d62b5-fbea-e982-d7d5-087f9badb028@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> <6a3d62b5-fbea-e982-d7d5-087f9badb028@oracle.com> Message-ID: <3765035a-ebd6-c7ed-4c06-6d9e269bc87c@oracle.com> > Also, it looks like the changes I made to ASB.appendChars(char[] s, int > off, int end) are not needed. Agree. >> Vladimir, don't you need to replace checkIndex with checkOffset in >> indexOf and lastIndexOf, so that we allow count == length? Yes, my bad. Good catch. Updated webrev in place. FTR I haven't done any extensive testing of the minimized fix. If we agree to proceed with it, the regression test should be updated as well. I think the viable solution would be to construct broken SBs (using reflection) and invoke affected methods on them. Best regards, Vladimir Ivanov >> On 3/22/17 8:35 AM, Vladimir Ivanov wrote: >>>>>> So are we convinced that the proposed changes will never lead to a >>>>>> crash due to a missing or incorrect bounds check, due to a racy >>>>>> use of >>>>>> an unsynchronized ASB instance e.g. StringBuilder? >>>>> >>>>> If only we had a static analysis tool that could tell us if the >>>>> code is >>>>> safe. Because we don't, in my initial changeset, we always take a >>>>> snapshot of the ASB fields by passing those field values to >>>>> StringUTF16 >>>>> before doing checks on them. And I wrote a test to make sure that >>>>> those >>>>> StringUTF16 interfaces are catching all the underflows and overflows I >>>>> could imagine, and I added verification code to detect when a check >>>>> was >>>>> missed. >>>>> >>>>> However, all the reviewers have requested to minimize the amount of >>>>> changes. In Vladimir's version, if there is a missing check >>>>> somewhere, >>>>> then yes it could lead to a crash. >>> >>> I'd like to point out that asserts and verification code are disabled >>> by default. They are invaluable during problem diagnosis, but don't >>> help at all from defence-in-depth perspective. >>> >>> But I agree that it's easier to reason about and test the initial >>> version of the fix. >>> >>>> I wonder if the reviewers have fully realized the potential impact >>>> here? >>>> This has exposed a flaw in the way intrinsics are used from core >>>> classes. >>> >>> FTR here are the checks I omitted in the minimized version (modulo >>> separation of indexOf/lastIndexOf for trusted/non-trusted callers): >>> >>> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ >>> >>> Other than that, the difference is mainly about undoing refactorings >>> and removing verification logic (asserts + checks in the JVM). >>> >>> There are still unsafe accesses which are considered safe in both >>> versions (see StringUTF16.Trusted usages in the initial version [1]). >>> >>> We used to provide safe wrappers for unsafe intrinsics which makes it >>> much easier to reason about code correctness. I'd like to see compact >>> string code refactored that way and IMO the initial version by Dean >>> is a big step in the right direction. >>> >>> I still prefer to see a point fix in 9 and major refactoring >>> happening in 10, but I'll leave the decision on how to proceed with >>> the fix to core-libs folks. After finishing the exercise minimizing >>> the fix, I'm much more comfortable with the initial fix [1] (though >>> there are changes I consider excessive). >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 >>> >>>>>>>> Some clarifications: >>>>>>>> >>>>>>>> ============ >>>>>>>> src/java.base/share/classes/java/lang/String.java: >>>>>>>> >>>>>>>> The bounds check is needed only in String.nonSyncContentEquals >>>>>>>> when it >>>>>>>> extracts info from AbstractStringBuilder. I don't see how out of >>>>>>>> bounds access can happen in String.contentEquals: >>>>>>>> if (n != length()) { >>>>>>>> return false; >>>>>>>> } >>>>>>>> ... >>>>>>>> for (int i = 0; i < n; i++) { >>>>>>>> if (StringUTF16.getChar(val, i) != cs.charAt(i)) { >>>>>>>> >>>>>>> >>>>>>> OK. >>>>>>> >>>>>>>> ============ >>>>>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>>>>> >>>>>>>> I think bounds checks in StringConcatHelper.prepend() are skipped >>>>>>>> intentionally, since java.lang.invoke.StringConcatFactory >>>>>>>> constructs >>>>>>>> method handle chains which already contain bounds checks: array >>>>>>>> length >>>>>>>> is precomputed based on argument values and all accesses are >>>>>>>> guaranteed to be in bounds. >>>>>>>> >>>>>>> >>>>>>> This is calling the trusted version of getChars() with no bounds >>>>>>> checks. It was a little more obvious when I had the Trusted inner >>>>>>> class. >>>>>>> >>>>>>>> ============ >>>>>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>> >>>>>>>> + static void putChar(byte[] val, int index, int c) { >>>>>>>> + assert index >= 0 && index < length(val) : "Trusted caller >>>>>>>> missed bounds check"; >>>>>>>> >>>>>>>> Unfortunately, asserts can affect inlining decisions (since they >>>>>>>> increase bytecode size). In order to minimize possible performance >>>>>>>> impact, I suggest to remove them from the fix targeting 9. >>>>>>>> >>>>>>> >>>>>>> Sure. >>>>>>> >>>>>>>> ============ >>>>>>>> private static int indexOfSupplementary(byte[] value, int >>>>>>>> ch, int >>>>>>>> fromIndex, int max) { >>>>>>>> if (Character.isValidCodePoint(ch)) { >>>>>>>> final char hi = Character.highSurrogate(ch); >>>>>>>> final char lo = Character.lowSurrogate(ch); >>>>>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>>>>> >>>>>>>> The check is redundant here. fromIndex & max are always inbounds by >>>>>>>> construction: >>>>>>>> >>>>>>>> public static int indexOf(byte[] value, int ch, int >>>>>>>> fromIndex) { >>>>>>>> int max = value.length >> 1; >>>>>>>> if (fromIndex < 0) { >>>>>>>> fromIndex = 0; >>>>>>>> } else if (fromIndex >= max) { >>>>>>>> // Note: fromIndex might be near -1>>>1. >>>>>>>> return -1; >>>>>>>> } >>>>>>>> ... >>>>>>>> return indexOfSupplementary(value, ch, fromIndex, max); >>>>>>>> >>>>>>> >>>>>>> OK. >>>>>>> >>>>>>>> ============ >>>>>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >>>>>>>> ABS.value & ABS.count. After that, all accesses should be >>>>>>>> inbounds by >>>>>>>> construction (in String.indexOf/lastIndexOf): >>>>>>>> >>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>> static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, >>>>>>>> String tgtStr, int fromIndex) { >>>>>>>> >>>>>>>> int rightIndex = srcCount - tgtCount; >>>>>>>> if (fromIndex > rightIndex) { >>>>>>>> fromIndex = rightIndex; >>>>>>>> } >>>>>>>> if (fromIndex < 0) { >>>>>>>> return -1; >>>>>>>> } >>>>>>>> >>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>>>>> byte[] tgt, int tgtCount, int >>>>>>>> fromIndex) { >>>>>>>> int min = tgtCount - 1; >>>>>>>> int i = min + fromIndex; >>>>>>>> int strLastIndex = tgtCount - 1; >>>>>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>>>>> >>>>>>>> startSearchForLastChar: >>>>>>>> while (true) { >>>>>>>> while (i >= min && getChar(src, i) != strLastChar) { >>>>>>>> >>>>>>>> There are 2 places: >>>>>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - >>>>>>>> inbound >>>>>>>> >>>>>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>>>>> min = tgtCount - 1 >>>>>>>> rightIndex = srcCount - tgtCount >>>>>>>> fromIndex <= rightIndex >>>>>>>> >>>>>>>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount >>>>>>>> - 1) >>>>>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>>>>> >>>>>>>> Hence, should be covered by the check on count & value: >>>>>>>> public int lastIndexOf(String str, int fromIndex) { >>>>>>>> + byte[] value = this.value; >>>>>>>> + int count = this.count; >>>>>>>> + byte coder = this.coder; >>>>>>>> + checkIndex(count, value.length >> coder); >>>>>>>> return String.lastIndexOf(value, coder, count, str, >>>>>>>> fromIndex); >>>>>>>> } >>>>>>>> >>>>>>> >>>>>>> OK, I will go with your version if it's OK with Sherman. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>>> Best regards, >>>>>>>> Vladimir Ivanov >>>>>>>> >>>>>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>>>>> >>>>>>>>>>>> I have the same concern. Can we fix the immediate problem in >>>>>>>>>>>> 9 and >>>>>>>>>>>> integrate verification logic in 10? >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, Tobias is suggesting having verification logic only >>>>>>>>>>> inside the >>>>>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>>>>> >>>>>>>>>> Yes and put them back in 10. >>>>>>>>>> >>>>>>>>>>> I'm OK with removing all the verification, but that won't reduce >>>>>>>>>>> the >>>>>>>>>>> library changes much. I could undo the renaming to >>>>>>>>>>> Trusted.getChar, but >>>>>>>>>>> we would still have the bounds checks moved into StringUTF16. >>>>>>>>>> >>>>>>>>>> I suggest to go with a point fix for 9: just add missing range >>>>>>>>>> checks. >>>>>>>>> >>>>>>> >>>>> >> > From mandy.chung at oracle.com Wed Mar 22 21:01:47 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 22 Mar 2017 14:01:47 -0700 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests In-Reply-To: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> References: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> Message-ID: > On Mar 22, 2017, at 1:09 PM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html >> 40 lines changed: 26 ins; 13 del; 1 mod; > > Hi all, > > could you please review this changeset which fixes in a few jdk_svc tests which were missed by JDK-8176176[1]? > > testing: :jdk_svc tests > webrev: http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8177374 test/java/lang/management/PlatformLoggingMXBean/LoggingMXBeanTest.java This test should need java.management. Where is it declared? Does the webrev miss some new file? Can you keep @modules sorted? e.g. the following in a few files: 42 * @modules jdk.management.agent/jdk.internal.agent 43 * java.management 44 * jdk.attach Mandy From chris.plummer at oracle.com Wed Mar 22 21:38:34 2017 From: chris.plummer at oracle.com (Chris Plummer) Date: Wed, 22 Mar 2017 14:38:34 -0700 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> Message-ID: <05c3ae38-2291-29b9-4c71-d67ffe5f19f7@oracle.com> Hello, Sorry to beat a dead horse here, but in addition to Dan's recommended fixes, I also reworked how I check for overflow when aligning up thanks to a suggestion from David: http://cr.openjdk.java.net/~cjplummer/8176768/webrev.03/webrev.hotspot/ src/os/linux/vm/os_linux.cpp: 726 // behaviour. However, be careful not to end up with a size 727 // of zero due to overflow. Don't add the guard page in that case. 728 size_t guard_size = os::Linux::default_guard_size(thr_type); 729 if (stack_size <= SIZE_MAX - guard_size) { 730 stack_size += guard_size; 731 } 732 assert(is_size_aligned(stack_size, os::vm_page_size()), "stack_size not aligned"); src/os/posix/vm/os_posix.cpp: 1260 // pthread_attr_setstacksize() may require that the size be rounded up to the OS page size. 1261 // Be careful not to round up to 0. Align down in that case. 1262 if (stack_size <= SIZE_MAX - vm_page_size()) { 1263 stack_size = align_size_up(stack_size, vm_page_size()); 1264 } else { 1265 stack_size = align_size_down(stack_size, vm_page_size()); 1266 } Dan's comments below for reference: > src/os/aix/vm/os_aix.cpp > No comments. > > src/os/bsd/vm/os_bsd.cpp > No comments. > > src/os/linux/vm/os_linux.cpp > L729: // In that cast just subract the page size to get the > maximum possible stack size. > Typo: 'cast' -> 'case' > Typo: 'subract' -> 'subtract' (Thomas also commented on it) > > src/os/posix/vm/os_posix.cpp > L263: // aligning up could have resulted in the size being 0. In > that case just subract the > Nit: 'aligning' -> 'Aligning' (since it's a sentence) > Typo: 'subract' -> 'subtract' > > src/os/solaris/vm/os_solaris.cpp > No comments. > > src/share/vm/prims/jvm.cpp > L2812: // -Avoid truncating on 32-bit platforms if size is > greater than UINT_MAX > Nit: needs a period at the end like L2813. > > test/runtime/Thread/TooSmallStackSize.java > No comments. > > test/runtime/Thread/TestThreadStackSizes.java > L26: * @summary Test user threads with various stacks sizes. > Typo?: "stacks sizes" -> "stack sizes" > > L36: super(null, null, "TestThreadStackSizes"+stackSize, > stackSize); > Nit: spaces around the "+". > > L46: TestThreadStackSizes testThreadStackSize = new > TestThreadStackSizes(stackSize); > Nit: extra space before '='. thanks Chris From daniel.daugherty at oracle.com Wed Mar 22 22:10:05 2017 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 22 Mar 2017 16:10:05 -0600 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <05c3ae38-2291-29b9-4c71-d67ffe5f19f7@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <05c3ae38-2291-29b9-4c71-d67ffe5f19f7@oracle.com> Message-ID: <689b9754-55cd-4dbd-a32e-dd93f7be5325@oracle.com> On 3/22/17 3:38 PM, Chris Plummer wrote: > Hello, > > Sorry to beat a dead horse here, but in addition to Dan's recommended > fixes, I also reworked how I check for overflow when aligning up > thanks to a suggestion from David: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.03/webrev.hotspot/ src/os/linux/vm/os_linux.cpp No comments. src/os/posix/vm/os_posix.cpp No comments. Thumbs up! Dan > > src/os/linux/vm/os_linux.cpp: > > 726 // behaviour. However, be careful not to end up with a size > 727 // of zero due to overflow. Don't add the guard page in that case. > 728 size_t guard_size = os::Linux::default_guard_size(thr_type); > 729 if (stack_size <= SIZE_MAX - guard_size) { > 730 stack_size += guard_size; > 731 } > 732 assert(is_size_aligned(stack_size, os::vm_page_size()), > "stack_size not aligned"); > > src/os/posix/vm/os_posix.cpp: > > 1260 // pthread_attr_setstacksize() may require that the size be > rounded up to the OS page size. > 1261 // Be careful not to round up to 0. Align down in that case. > 1262 if (stack_size <= SIZE_MAX - vm_page_size()) { > 1263 stack_size = align_size_up(stack_size, vm_page_size()); > 1264 } else { > 1265 stack_size = align_size_down(stack_size, vm_page_size()); > 1266 } > > Dan's comments below for reference: >> src/os/aix/vm/os_aix.cpp >> No comments. >> >> src/os/bsd/vm/os_bsd.cpp >> No comments. >> >> src/os/linux/vm/os_linux.cpp >> L729: // In that cast just subract the page size to get the >> maximum possible stack size. >> Typo: 'cast' -> 'case' >> Typo: 'subract' -> 'subtract' (Thomas also commented on it) >> >> src/os/posix/vm/os_posix.cpp >> L263: // aligning up could have resulted in the size being 0. In >> that case just subract the >> Nit: 'aligning' -> 'Aligning' (since it's a sentence) >> Typo: 'subract' -> 'subtract' >> >> src/os/solaris/vm/os_solaris.cpp >> No comments. >> >> src/share/vm/prims/jvm.cpp >> L2812: // -Avoid truncating on 32-bit platforms if size is >> greater than UINT_MAX >> Nit: needs a period at the end like L2813. >> >> test/runtime/Thread/TooSmallStackSize.java >> No comments. >> >> test/runtime/Thread/TestThreadStackSizes.java >> L26: * @summary Test user threads with various stacks sizes. >> Typo?: "stacks sizes" -> "stack sizes" >> >> L36: super(null, null, "TestThreadStackSizes"+stackSize, >> stackSize); >> Nit: spaces around the "+". >> >> L46: TestThreadStackSizes testThreadStackSize = new >> TestThreadStackSizes(stackSize); >> Nit: extra space before '='. > thanks > > Chris From igor.ignatyev at oracle.com Wed Mar 22 23:09:57 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 22 Mar 2017 16:09:57 -0700 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests In-Reply-To: References: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> Message-ID: <3056CF52-47EB-4C5D-B650-81CDCFFD568F@oracle.com> Hi Mandy, I've (re)sorted @modules in all the tests which I touched -- http://cr.openjdk.java.net/~iignatyev/8177374/webrev.01/index.html > test/java/lang/management/PlatformLoggingMXBean/LoggingMXBeanTest.java > This test should need java.management. Where is it declared? it's declared in test/java/lang/management/TEST.properties which has been pushed by JDK-8176176. Thanks, -- Igor > On Mar 22, 2017, at 2:01 PM, Mandy Chung wrote: > > >> On Mar 22, 2017, at 1:09 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html >>> 40 lines changed: 26 ins; 13 del; 1 mod; >> >> Hi all, >> >> could you please review this changeset which fixes in a few jdk_svc tests which were missed by JDK-8176176[1]? >> >> testing: :jdk_svc tests >> webrev: http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177374 > > > test/java/lang/management/PlatformLoggingMXBean/LoggingMXBeanTest.java > This test should need java.management. Where is it declared? > Does the webrev miss some new file? > > Can you keep @modules sorted? e.g. the following in a few files: > 42 * @modules jdk.management.agent/jdk.internal.agent > 43 * java.management > 44 * jdk.attach > Mandy > From mandy.chung at oracle.com Wed Mar 22 23:29:11 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 22 Mar 2017 16:29:11 -0700 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests In-Reply-To: <3056CF52-47EB-4C5D-B650-81CDCFFD568F@oracle.com> References: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> <3056CF52-47EB-4C5D-B650-81CDCFFD568F@oracle.com> Message-ID: > On Mar 22, 2017, at 4:09 PM, Igor Ignatyev wrote: > > Hi Mandy, > > I've (re)sorted @modules in all the tests which I touched -- http://cr.openjdk.java.net/~iignatyev/8177374/webrev.01/index.html > +1 >> test/java/lang/management/PlatformLoggingMXBean/LoggingMXBeanTest.java >> This test should need java.management. Where is it declared? > it's declared in test/java/lang/management/TEST.properties which has been pushed by JDK-8176176. > Ah I see. this is in jdk9/hs/jdk repo and not yet integrated in jdk9/dev/jdk. Mandy From david.holmes at oracle.com Thu Mar 23 00:03:09 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Mar 2017 10:03:09 +1000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <05c3ae38-2291-29b9-4c71-d67ffe5f19f7@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <05c3ae38-2291-29b9-4c71-d67ffe5f19f7@oracle.com> Message-ID: <210ce0aa-2859-83df-46f1-6e22b0860c07@oracle.com> Looks good Chris! Thanks. David On 23/03/2017 7:38 AM, Chris Plummer wrote: > Hello, > > Sorry to beat a dead horse here, but in addition to Dan's recommended > fixes, I also reworked how I check for overflow when aligning up thanks > to a suggestion from David: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.03/webrev.hotspot/ > > src/os/linux/vm/os_linux.cpp: > > 726 // behaviour. However, be careful not to end up with a size > 727 // of zero due to overflow. Don't add the guard page in that case. > 728 size_t guard_size = os::Linux::default_guard_size(thr_type); > 729 if (stack_size <= SIZE_MAX - guard_size) { > 730 stack_size += guard_size; > 731 } > 732 assert(is_size_aligned(stack_size, os::vm_page_size()), > "stack_size not aligned"); > > src/os/posix/vm/os_posix.cpp: > > 1260 // pthread_attr_setstacksize() may require that the size be > rounded up to the OS page size. > 1261 // Be careful not to round up to 0. Align down in that case. > 1262 if (stack_size <= SIZE_MAX - vm_page_size()) { > 1263 stack_size = align_size_up(stack_size, vm_page_size()); > 1264 } else { > 1265 stack_size = align_size_down(stack_size, vm_page_size()); > 1266 } > > Dan's comments below for reference: >> src/os/aix/vm/os_aix.cpp >> No comments. >> >> src/os/bsd/vm/os_bsd.cpp >> No comments. >> >> src/os/linux/vm/os_linux.cpp >> L729: // In that cast just subract the page size to get the >> maximum possible stack size. >> Typo: 'cast' -> 'case' >> Typo: 'subract' -> 'subtract' (Thomas also commented on it) >> >> src/os/posix/vm/os_posix.cpp >> L263: // aligning up could have resulted in the size being 0. In >> that case just subract the >> Nit: 'aligning' -> 'Aligning' (since it's a sentence) >> Typo: 'subract' -> 'subtract' >> >> src/os/solaris/vm/os_solaris.cpp >> No comments. >> >> src/share/vm/prims/jvm.cpp >> L2812: // -Avoid truncating on 32-bit platforms if size is >> greater than UINT_MAX >> Nit: needs a period at the end like L2813. >> >> test/runtime/Thread/TooSmallStackSize.java >> No comments. >> >> test/runtime/Thread/TestThreadStackSizes.java >> L26: * @summary Test user threads with various stacks sizes. >> Typo?: "stacks sizes" -> "stack sizes" >> >> L36: super(null, null, "TestThreadStackSizes"+stackSize, >> stackSize); >> Nit: spaces around the "+". >> >> L46: TestThreadStackSizes testThreadStackSize = new >> TestThreadStackSizes(stackSize); >> Nit: extra space before '='. > thanks > > Chris From david.holmes at oracle.com Thu Mar 23 00:22:05 2017 From: david.holmes at oracle.com (David Holmes) Date: Thu, 23 Mar 2017 10:22:05 +1000 Subject: Adding native method in custom jdk package In-Reply-To: <001c01d2a337$2e31a560$8a94f020$@hispeed.ch> References: <000c01d2a329$f4a11630$dde34290$@hispeed.ch> <001c01d2a337$2e31a560$8a94f020$@hispeed.ch> Message-ID: <27efece2-e51c-78cf-86c3-a3ce935bd137@oracle.com> On 23/03/2017 4:07 AM, Christian Hagedorn wrote: > Thanks for your quick help. That got me on the right track, it does work > now. I added my path to the LIBJAVA_SRC_DIRS in the > ?jdk/make/lib/CoreLibraries.gmk? file. Yes that was the missing part. But this is a build question, or at best a JDK question, not a hotspot question. Regards, David > > > Best regards, > > Christian > > > > Von: Bob Vandette [mailto:bob.vandette at oracle.com] > Gesendet: Mittwoch, 22. M?rz 2017 18:23 > An: Christian Hagedorn > Cc: hotspot-dev at openjdk.java.net > Betreff: Re: Adding native method in custom jdk package > > > > Checkout this blog that provides a simple example of how to create and call > a native > > function in a shared library. > > > > https://blogs.oracle.com/moonocean/entry/a_simple_example_of_jni > > > > Let me know if you have any questions, > > Bob. > > > > On Mar 22, 2017, at 12:32 PM, Christian Hagedorn > wrote: > > > > Hi, > > > > I have created a class Test.java in an own package test.lang inside the jdk > ("jdk/src/share/classes/test/lang/Test.java"). Now I wanted to add a native > method. > > I found out that my header was created inside" > build/linux./jdk/gensrc_headers/test_lang_Test.h". > > So I have tried to just create a new file inside a new folder > "jdk/src/share/native/test/lang/Test.c" including "test_lang_Test.h". > > From there I do my JNICALL to the VM like it is done for example in > "jdk/src/share/native/java/lang/System.c". > > However this still results in a UnsatisfiedLinkError. I am sure I have to do > some more steps to link and add it to the build properly. What am I missing? > > > > Best regards, > > Christian > > > > > > From thomas.stuefe at gmail.com Thu Mar 23 05:01:47 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 23 Mar 2017 05:01:47 +0000 Subject: (RFR)(S)(10): 8176768: hotspot ignores PTHREAD_STACK_MIN when creating new threads In-Reply-To: <05c3ae38-2291-29b9-4c71-d67ffe5f19f7@oracle.com> References: <9901f6db-1932-f61d-89c7-e1cecc8f1a2f@oracle.com> <74ab50c7-b8e6-4f70-6909-6032a4471b03@oracle.com> <05c3ae38-2291-29b9-4c71-d67ffe5f19f7@oracle.com> Message-ID: Looks fine! On Wed, 22 Mar 2017 at 22:38, Chris Plummer wrote: > Hello, > > Sorry to beat a dead horse here, but in addition to Dan's recommended > fixes, I also reworked how I check for overflow when aligning up thanks > to a suggestion from David: > > http://cr.openjdk.java.net/~cjplummer/8176768/webrev.03/webrev.hotspot/ > > src/os/linux/vm/os_linux.cpp: > > 726 // behaviour. However, be careful not to end up with a size > 727 // of zero due to overflow. Don't add the guard page in that case. > 728 size_t guard_size = os::Linux::default_guard_size(thr_type); > 729 if (stack_size <= SIZE_MAX - guard_size) { > 730 stack_size += guard_size; > 731 } > 732 assert(is_size_aligned(stack_size, os::vm_page_size()), > "stack_size not aligned"); > > src/os/posix/vm/os_posix.cpp: > > 1260 // pthread_attr_setstacksize() may require that the size be > rounded up to the OS page size. > 1261 // Be careful not to round up to 0. Align down in that case. > 1262 if (stack_size <= SIZE_MAX - vm_page_size()) { > 1263 stack_size = align_size_up(stack_size, vm_page_size()); > 1264 } else { > 1265 stack_size = align_size_down(stack_size, vm_page_size()); > 1266 } > > Dan's comments below for reference: > > src/os/aix/vm/os_aix.cpp > > No comments. > > > > src/os/bsd/vm/os_bsd.cpp > > No comments. > > > > src/os/linux/vm/os_linux.cpp > > L729: // In that cast just subract the page size to get the > > maximum possible stack size. > > Typo: 'cast' -> 'case' > > Typo: 'subract' -> 'subtract' (Thomas also commented on it) > > > > src/os/posix/vm/os_posix.cpp > > L263: // aligning up could have resulted in the size being 0. In > > that case just subract the > > Nit: 'aligning' -> 'Aligning' (since it's a sentence) > > Typo: 'subract' -> 'subtract' > > > > src/os/solaris/vm/os_solaris.cpp > > No comments. > > > > src/share/vm/prims/jvm.cpp > > L2812: // -Avoid truncating on 32-bit platforms if size is > > greater than UINT_MAX > > Nit: needs a period at the end like L2813. > > > > test/runtime/Thread/TooSmallStackSize.java > > No comments. > > > > test/runtime/Thread/TestThreadStackSizes.java > > L26: * @summary Test user threads with various stacks sizes. > > Typo?: "stacks sizes" -> "stack sizes" > > > > L36: super(null, null, "TestThreadStackSizes"+stackSize, > > stackSize); > > Nit: spaces around the "+". > > > > L46: TestThreadStackSizes testThreadStackSize = new > > TestThreadStackSizes(stackSize); > > Nit: extra space before '='. > thanks > > Chris > From michael.rasmussen at zeroturnaround.com Thu Mar 23 09:30:45 2017 From: michael.rasmussen at zeroturnaround.com (Michael Rasmussen) Date: Thu, 23 Mar 2017 11:30:45 +0200 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: References: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> <1fef2eae-529f-4c01-2faf-30481a04e1d9@oracle.com> Message-ID: On Mar 14, 2017 23:03, "Vladimir Ivanov" wrote: Yes, you are right. Then the only option is: -XX:+UnlockDiagnosticVMOptions -XX:-CheckIntrinsics Best regards, Vladimir Ivanov Thinking about this a bit further, are these messages going to be enabled by default when jdk9 is released? These kind of messages are clearly only for the benefit of hotspot developers, and has zero value to end users, who wouldn't know anything about this annotation or the implications of it. For the average user, heck you have to be a pretty advanced user to even know what intrinsic means, getting warnings like this, I can only think it would do a lot more harm than good, thinking that their installation is broken or something. /Michael From patric.hedlin at oracle.com Thu Mar 23 11:04:47 2017 From: patric.hedlin at oracle.com (Patric Hedlin) Date: Thu, 23 Mar 2017 12:04:47 +0100 Subject: JDK10/RFR(XXS): 8011352: C1: TraceCodeBlobStacks crashes fastdebug solaris sparc Message-ID: <6821786d-3c0b-9219-0975-5d7c4e4e8e79@oracle.com> Hi, Please review this minor change/fix: https://bugs.openjdk.java.net/browse/JDK-8011352 Rationale: Replacing assumption on '_younger_window' being available when performing lookup on OUT-register part (in window), in order to handle such access in the same manner as other "out-of-scope" lookup (i.e. by returning NULL). Regards, Patric Patch below: -----8<----- --- old/src/cpu/sparc/vm/frame_sparc.cpp Wed Mar 22 16:47:13 2017 +++ new/src/cpu/sparc/vm/frame_sparc.cpp Wed Mar 22 16:47:12 2017 @@ -123,8 +123,8 @@ reg = regname->as_Register(); } if (reg->is_out()) { - assert(_younger_window != NULL, "Younger window should be available"); - return second_word + (address)&_younger_window[reg->after_save()->sp_offset_in_saved_window()]; + return _younger_window == NULL ? NULL : + second_word + (address)&_younger_window[reg->after_save()->sp_offset_in_saved_window()]; } if (reg->is_local() || reg->is_in()) { assert(_window != NULL, "Window should be available"); From volker.simonis at gmail.com Thu Mar 23 11:43:21 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 23 Mar 2017 12:43:21 +0100 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: References: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> <1fef2eae-529f-4c01-2faf-30481a04e1d9@oracle.com> Message-ID: On Thu, Mar 23, 2017 at 10:30 AM, Michael Rasmussen wrote: > On Mar 14, 2017 23:03, "Vladimir Ivanov" > wrote: > > Yes, you are right. Then the only option is: > -XX:+UnlockDiagnosticVMOptions -XX:-CheckIntrinsics > > Best regards, > Vladimir Ivanov > > > Thinking about this a bit further, are these messages going to be enabled > by default when jdk9 is released? > > These kind of messages are clearly only for the benefit of hotspot > developers, and has zero value to end users, who wouldn't know anything > about this annotation or the implications of it. > > For the average user, heck you have to be a pretty advanced user to even > know what intrinsic means, getting warnings like this, I can only think it > would do a lot more harm than good, thinking that their installation is > broken or something. > I think this issue is somehow a counterpart of and related to "JDK-8013579: Intrinsics for Java methods don't take class redefinition into account" :) So if you transform a non-native method which is subject to intrinsification you should be aware that your transformations will be lost, once the method will be intrinsified! That said, I agree that the warning message is hard to understand for the average Java developer but I think it has a value. As Vladimir pointed out, if you wrap a native method, which would be intrinisfied otherwise, through class transformation / SetNativeMethodPrefix, that method will not be intrinsified any more. So if you do the transformation for profiling for example, you'd profile a different thing. I actually somehow missed "8131326: Enable CheckIntrinsics in all types of builds" which enabled CheckIntrinsics in product builds by default (before it was only enabled in debug builds). Not sure what was the rational behind this change (CC'ed Zoltan who did the change) but I'm pretty sure that the effects of SetNativeMethodPrefix haven't been taken into account when that change was made. In the end it paid off, otherwise you wouldn't have found this issue :) Regards, Volker > /Michael From zoltan.majo at oracle.com Thu Mar 23 14:01:32 2017 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Thu, 23 Mar 2017 15:01:32 +0100 Subject: @HotSpotIntrinsicCandidate and native prefixes In-Reply-To: References: <14e87480-fd1e-7481-4c9f-d9939472e50b@oracle.com> <1fef2eae-529f-4c01-2faf-30481a04e1d9@oracle.com> Message-ID: Hi Volker, On 03/23/2017 12:43 PM, Volker Simonis wrote: > [...] > > > I actually somehow missed "8131326: Enable CheckIntrinsics in all > types of builds" which enabled CheckIntrinsics in product builds by > default (before it was only enabled in debug builds). Not sure what > was the rational behind this change (CC'ed Zoltan who did the change) > but I'm pretty sure that the effects of SetNativeMethodPrefix haven't > been taken into account when that change was made. In the end it paid > off, otherwise you wouldn't have found this issue :) I mentioned the following in the RFR for 8131326: "Problem: The CheckIntrinsics flag added by JDK-8076112 is currently enabled only in debug builds. As a result, users of product builds might easier oversee potential mismatches between VM-level and classfile-level intrinsics." http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2015-July/018425.html I don't recall any other reason for doing this change, although somebody might have asked me in a private message to do it (maybe with more/different justification). Best regards, Zoltan > > Regards, > Volker > >> /Michael From rickard.backman at oracle.com Thu Mar 23 14:04:57 2017 From: rickard.backman at oracle.com (Rickard =?iso-8859-1?Q?B=E4ckman?=) Date: Thu, 23 Mar 2017 15:04:57 +0100 Subject: JDK10/RFR(XXS): 8011352: C1: TraceCodeBlobStacks crashes fastdebug solaris sparc In-Reply-To: <6821786d-3c0b-9219-0975-5d7c4e4e8e79@oracle.com> References: <6821786d-3c0b-9219-0975-5d7c4e4e8e79@oracle.com> Message-ID: <20170323140456.GC8528@rbackman> Looks good. /R On 03/23, Patric Hedlin wrote: > Hi, > > Please review this minor change/fix: > > https://bugs.openjdk.java.net/browse/JDK-8011352 > > Rationale: > > Replacing assumption on '_younger_window' being available when > performing lookup on OUT-register part (in window), in order to > handle such access in the same manner as other "out-of-scope" lookup > (i.e. by returning NULL). > > Regards, > Patric > > Patch below: > > -----8<----- > > --- old/src/cpu/sparc/vm/frame_sparc.cpp Wed Mar 22 16:47:13 2017 > +++ new/src/cpu/sparc/vm/frame_sparc.cpp Wed Mar 22 16:47:12 2017 > @@ -123,8 +123,8 @@ > reg = regname->as_Register(); > } > if (reg->is_out()) { > - assert(_younger_window != NULL, "Younger window should be available"); > - return second_word + (address)&_younger_window[reg->after_save()->sp_offset_in_saved_window()]; > + return _younger_window == NULL ? NULL : > + second_word + (address)&_younger_window[reg->after_save()->sp_offset_in_saved_window()]; > } > if (reg->is_local() || reg->is_in()) { > assert(_window != NULL, "Window should be available"); > From daniel.fuchs at oracle.com Thu Mar 23 14:33:32 2017 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 23 Mar 2017 14:33:32 +0000 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests In-Reply-To: <3056CF52-47EB-4C5D-B650-81CDCFFD568F@oracle.com> References: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> <3056CF52-47EB-4C5D-B650-81CDCFFD568F@oracle.com> Message-ID: <9fb37d41-528b-e040-4114-a154e55c777b@oracle.com> Hi Igor, small nit: 42 * @modules java.management 43 * jdk.attach 44 * jdk.management.agent/jdk.internal.agent I don't think java.management needs to be specified as a dependency when the test requires jdk.management.agent, because jdk.management.agent already requires java.management. best regards, -- daniel On 22/03/2017 23:09, Igor Ignatyev wrote: > Hi Mandy, > > I've (re)sorted @modules in all the tests which I touched -- > http://cr.openjdk.java.net/~iignatyev/8177374/webrev.01/index.html > >> test/java/lang/management/PlatformLoggingMXBean/LoggingMXBeanTest.java >> This test should need java.management. Where is it declared? > it's declared in test/java/lang/management/TEST.properties which has > been pushed by JDK-8176176. > > Thanks, > -- Igor >> On Mar 22, 2017, at 2:01 PM, Mandy Chung > > wrote: >> >> >>> On Mar 22, 2017, at 1:09 PM, Igor Ignatyev >> > wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html >>>> 40 lines changed: 26 ins; 13 del; 1 mod; >>> >>> Hi all, >>> >>> could you please review this changeset which fixes in a few jdk_svc >>> tests which were missed by JDK-8176176[1]? >>> >>> testing: :jdk_svc tests >>> webrev: >>> http://cr.openjdk.java.net/~iignatyev/8177374/webrev.00/index.html >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177374 >> >> >> test/java/lang/management/PlatformLoggingMXBean/LoggingMXBeanTest.java >> This test should need java.management. Where is it declared? >> Does the webrev miss some new file? >> >> Can you keep @modules sorted? e.g. the following in a few files: >> 42 * @modules jdk.management.agent/jdk.internal.agent >> 43 * java.management >> 44 * jdk.attach >> Mandy >> > From mandy.chung at oracle.com Thu Mar 23 16:39:49 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 23 Mar 2017 09:39:49 -0700 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests In-Reply-To: <9fb37d41-528b-e040-4114-a154e55c777b@oracle.com> References: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> <3056CF52-47EB-4C5D-B650-81CDCFFD568F@oracle.com> <9fb37d41-528b-e040-4114-a154e55c777b@oracle.com> Message-ID: > On Mar 23, 2017, at 7:33 AM, Daniel Fuchs wrote: > > Hi Igor, > > small nit: > > 42 * @modules java.management > 43 * jdk.attach > 44 * jdk.management.agent/jdk.internal.agent > > I don't think java.management needs to be specified as > a dependency when the test requires jdk.management.agent, > because jdk.management.agent already requires java.management. That?s true. Igor - How do you analyze the dependency? Are you using jdeps ?-list-reduced-deps? Mandy From dean.long at oracle.com Thu Mar 23 18:25:47 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 23 Mar 2017 11:25:47 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <3765035a-ebd6-c7ed-4c06-6d9e269bc87c@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> <6a3d62b5-fbea-e982-d7d5-087f9badb028@oracle.com> <3765035a-ebd6-c7ed-4c06-6d9e269bc87c@oracle.com> Message-ID: <7ade4c8d-891c-618b-0fa3-4d641da236ba@oracle.com> On 3/22/17 1:49 PM, Vladimir Ivanov wrote: >> Also, it looks like the changes I made to ASB.appendChars(char[] s, int >> off, int end) are not needed. > > Agree. > >>> Vladimir, don't you need to replace checkIndex with checkOffset in >>> indexOf and lastIndexOf, so that we allow count == length? > > Yes, my bad. Good catch. Updated webrev in place. > > FTR I haven't done any extensive testing of the minimized fix. > > If we agree to proceed with it, the regression test should be updated > as well. I think the viable solution would be to construct broken SBs > (using reflection) and invoke affected methods on them. > We can construct broken SBs using the Helper class that gets patched into java.lang. I'll work on that. dl > Best regards, > Vladimir Ivanov > >>> On 3/22/17 8:35 AM, Vladimir Ivanov wrote: >>>>>>> So are we convinced that the proposed changes will never lead to a >>>>>>> crash due to a missing or incorrect bounds check, due to a racy >>>>>>> use of >>>>>>> an unsynchronized ASB instance e.g. StringBuilder? >>>>>> >>>>>> If only we had a static analysis tool that could tell us if the >>>>>> code is >>>>>> safe. Because we don't, in my initial changeset, we always take a >>>>>> snapshot of the ASB fields by passing those field values to >>>>>> StringUTF16 >>>>>> before doing checks on them. And I wrote a test to make sure that >>>>>> those >>>>>> StringUTF16 interfaces are catching all the underflows and >>>>>> overflows I >>>>>> could imagine, and I added verification code to detect when a check >>>>>> was >>>>>> missed. >>>>>> >>>>>> However, all the reviewers have requested to minimize the amount of >>>>>> changes. In Vladimir's version, if there is a missing check >>>>>> somewhere, >>>>>> then yes it could lead to a crash. >>>> >>>> I'd like to point out that asserts and verification code are disabled >>>> by default. They are invaluable during problem diagnosis, but don't >>>> help at all from defence-in-depth perspective. >>>> >>>> But I agree that it's easier to reason about and test the initial >>>> version of the fix. >>>> >>>>> I wonder if the reviewers have fully realized the potential impact >>>>> here? >>>>> This has exposed a flaw in the way intrinsics are used from core >>>>> classes. >>>> >>>> FTR here are the checks I omitted in the minimized version (modulo >>>> separation of indexOf/lastIndexOf for trusted/non-trusted callers): >>>> >>>> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ >>>> >>>> Other than that, the difference is mainly about undoing refactorings >>>> and removing verification logic (asserts + checks in the JVM). >>>> >>>> There are still unsafe accesses which are considered safe in both >>>> versions (see StringUTF16.Trusted usages in the initial version [1]). >>>> >>>> We used to provide safe wrappers for unsafe intrinsics which makes it >>>> much easier to reason about code correctness. I'd like to see compact >>>> string code refactored that way and IMO the initial version by Dean >>>> is a big step in the right direction. >>>> >>>> I still prefer to see a point fix in 9 and major refactoring >>>> happening in 10, but I'll leave the decision on how to proceed with >>>> the fix to core-libs folks. After finishing the exercise minimizing >>>> the fix, I'm much more comfortable with the initial fix [1] (though >>>> there are changes I consider excessive). >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 >>>> >>>>>>>>> Some clarifications: >>>>>>>>> >>>>>>>>> ============ >>>>>>>>> src/java.base/share/classes/java/lang/String.java: >>>>>>>>> >>>>>>>>> The bounds check is needed only in String.nonSyncContentEquals >>>>>>>>> when it >>>>>>>>> extracts info from AbstractStringBuilder. I don't see how out of >>>>>>>>> bounds access can happen in String.contentEquals: >>>>>>>>> if (n != length()) { >>>>>>>>> return false; >>>>>>>>> } >>>>>>>>> ... >>>>>>>>> for (int i = 0; i < n; i++) { >>>>>>>>> if (StringUTF16.getChar(val, i) != >>>>>>>>> cs.charAt(i)) { >>>>>>>>> >>>>>>>> >>>>>>>> OK. >>>>>>>> >>>>>>>>> ============ >>>>>>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>>>>>> >>>>>>>>> I think bounds checks in StringConcatHelper.prepend() are skipped >>>>>>>>> intentionally, since java.lang.invoke.StringConcatFactory >>>>>>>>> constructs >>>>>>>>> method handle chains which already contain bounds checks: array >>>>>>>>> length >>>>>>>>> is precomputed based on argument values and all accesses are >>>>>>>>> guaranteed to be in bounds. >>>>>>>>> >>>>>>>> >>>>>>>> This is calling the trusted version of getChars() with no bounds >>>>>>>> checks. It was a little more obvious when I had the Trusted inner >>>>>>>> class. >>>>>>>> >>>>>>>>> ============ >>>>>>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>> >>>>>>>>> + static void putChar(byte[] val, int index, int c) { >>>>>>>>> + assert index >= 0 && index < length(val) : "Trusted >>>>>>>>> caller >>>>>>>>> missed bounds check"; >>>>>>>>> >>>>>>>>> Unfortunately, asserts can affect inlining decisions (since they >>>>>>>>> increase bytecode size). In order to minimize possible >>>>>>>>> performance >>>>>>>>> impact, I suggest to remove them from the fix targeting 9. >>>>>>>>> >>>>>>>> >>>>>>>> Sure. >>>>>>>> >>>>>>>>> ============ >>>>>>>>> private static int indexOfSupplementary(byte[] value, int >>>>>>>>> ch, int >>>>>>>>> fromIndex, int max) { >>>>>>>>> if (Character.isValidCodePoint(ch)) { >>>>>>>>> final char hi = Character.highSurrogate(ch); >>>>>>>>> final char lo = Character.lowSurrogate(ch); >>>>>>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>>>>>> >>>>>>>>> The check is redundant here. fromIndex & max are always >>>>>>>>> inbounds by >>>>>>>>> construction: >>>>>>>>> >>>>>>>>> public static int indexOf(byte[] value, int ch, int >>>>>>>>> fromIndex) { >>>>>>>>> int max = value.length >> 1; >>>>>>>>> if (fromIndex < 0) { >>>>>>>>> fromIndex = 0; >>>>>>>>> } else if (fromIndex >= max) { >>>>>>>>> // Note: fromIndex might be near -1>>>1. >>>>>>>>> return -1; >>>>>>>>> } >>>>>>>>> ... >>>>>>>>> return indexOfSupplementary(value, ch, fromIndex, >>>>>>>>> max); >>>>>>>>> >>>>>>>> >>>>>>>> OK. >>>>>>>> >>>>>>>>> ============ >>>>>>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>>>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >>>>>>>>> ABS.value & ABS.count. After that, all accesses should be >>>>>>>>> inbounds by >>>>>>>>> construction (in String.indexOf/lastIndexOf): >>>>>>>>> >>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>> static int lastIndexOf(byte[] src, byte srcCoder, int >>>>>>>>> srcCount, >>>>>>>>> String tgtStr, int fromIndex) { >>>>>>>>> >>>>>>>>> int rightIndex = srcCount - tgtCount; >>>>>>>>> if (fromIndex > rightIndex) { >>>>>>>>> fromIndex = rightIndex; >>>>>>>>> } >>>>>>>>> if (fromIndex < 0) { >>>>>>>>> return -1; >>>>>>>>> } >>>>>>>>> >>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>>>>>> byte[] tgt, int tgtCount, int >>>>>>>>> fromIndex) { >>>>>>>>> int min = tgtCount - 1; >>>>>>>>> int i = min + fromIndex; >>>>>>>>> int strLastIndex = tgtCount - 1; >>>>>>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>>>>>> >>>>>>>>> startSearchForLastChar: >>>>>>>>> while (true) { >>>>>>>>> while (i >= min && getChar(src, i) != strLastChar) { >>>>>>>>> >>>>>>>>> There are 2 places: >>>>>>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - >>>>>>>>> inbound >>>>>>>>> >>>>>>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>>>>>> min = tgtCount - 1 >>>>>>>>> rightIndex = srcCount - tgtCount >>>>>>>>> fromIndex <= rightIndex >>>>>>>>> >>>>>>>>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount >>>>>>>>> - 1) >>>>>>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>>>>>> >>>>>>>>> Hence, should be covered by the check on count & value: >>>>>>>>> public int lastIndexOf(String str, int fromIndex) { >>>>>>>>> + byte[] value = this.value; >>>>>>>>> + int count = this.count; >>>>>>>>> + byte coder = this.coder; >>>>>>>>> + checkIndex(count, value.length >> coder); >>>>>>>>> return String.lastIndexOf(value, coder, count, str, >>>>>>>>> fromIndex); >>>>>>>>> } >>>>>>>>> >>>>>>>> >>>>>>>> OK, I will go with your version if it's OK with Sherman. >>>>>>>> >>>>>>>> dl >>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Vladimir Ivanov >>>>>>>>> >>>>>>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>>>>>> >>>>>>>>>>>>> I have the same concern. Can we fix the immediate problem in >>>>>>>>>>>>> 9 and >>>>>>>>>>>>> integrate verification logic in 10? >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, Tobias is suggesting having verification logic only >>>>>>>>>>>> inside the >>>>>>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>>>>>> >>>>>>>>>>> Yes and put them back in 10. >>>>>>>>>>> >>>>>>>>>>>> I'm OK with removing all the verification, but that won't >>>>>>>>>>>> reduce >>>>>>>>>>>> the >>>>>>>>>>>> library changes much. I could undo the renaming to >>>>>>>>>>>> Trusted.getChar, but >>>>>>>>>>>> we would still have the bounds checks moved into StringUTF16. >>>>>>>>>>> >>>>>>>>>>> I suggest to go with a point fix for 9: just add missing range >>>>>>>>>>> checks. >>>>>>>>>> >>>>>>>> >>>>>> >>> >> From bob.vandette at oracle.com Thu Mar 23 18:58:52 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 23 Mar 2017 14:58:52 -0400 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> <1FBB8985-1FE6-44E1-B338-7C6D49372D93@oracle.com> Message-ID: <595FE441-10E0-4AB6-B7EE-686132F2269C@oracle.com> What does your aarch64 build have as IMPLEMENTOR in it?s ?release? file. I wonder if we can use this difference for now? IMPLEMENTOR="Oracle Corporation? Bob. > On Mar 21, 2017, at 11:27 AM, Andrew Haley wrote: > > On 21/03/17 15:25, Bob Vandette wrote: >> I?d prefer to use something like these System properties. >> >> "java.vendor" JRE vendor name >> "java.vendor.url" JRE vendor URL >> >> or to add something to the ?release? file. >> >> HOTSPOT_ARCH_DIR=arm >> >> or >> >> HOTSPOT_ARCH_DIR=aarch64 > > OK, sounds good. I'll have a think. > > Andrew. > From dean.long at oracle.com Thu Mar 23 19:03:20 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 23 Mar 2017 12:03:20 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <7ade4c8d-891c-618b-0fa3-4d641da236ba@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> <6a3d62b5-fbea-e982-d7d5-087f9badb028@oracle.com> <3765035a-ebd6-c7ed-4c06-6d9e269bc87c@oracle.com> <7ade4c8d-891c-618b-0fa3-4d641da236ba@oracle.com> Message-ID: <7b319f81-2a48-1533-efe1-6fd86a3d7244@oracle.com> On 3/23/17 11:25 AM, dean.long at oracle.com wrote: > On 3/22/17 1:49 PM, Vladimir Ivanov wrote: > >>> Also, it looks like the changes I made to ASB.appendChars(char[] s, int >>> off, int end) are not needed. >> >> Agree. >> >>>> Vladimir, don't you need to replace checkIndex with checkOffset in >>>> indexOf and lastIndexOf, so that we allow count == length? >> >> Yes, my bad. Good catch. Updated webrev in place. >> >> FTR I haven't done any extensive testing of the minimized fix. >> >> If we agree to proceed with it, the regression test should be updated >> as well. I think the viable solution would be to construct broken SBs >> (using reflection) and invoke affected methods on them. >> > > We can construct broken SBs using the Helper class that gets patched > into java.lang. I'll work on that. > Nevermind. I forgot that some problems can only happen when the SB is changed half-way through the method. For example, in append(), we can't force an overflow unless we change the SB after ensureCapacityInternal() is called. I could do something like: 760 public AbstractStringBuilder append(int i) { 761 int count = this.count; 762 int spaceNeeded = count + Integer.stringSize(i); 763 ensureCapacityInternal(spaceNeeded); 764 if (isLatin1()) { >>>>>> Helper.fuzzValue(this); 765 Integer.getChars(i, spaceNeeded, value); 766 } else { 767 byte[] val = this.value; >>>>>> Helper.fuzzValue(this); 768 checkBoundsBeginEnd(count, spaceNeeded, val.length >> 1); 769 Integer.getCharsUTF16(i, spaceNeeded, val); 770 } 771 this.count = spaceNeeded; 772 return this; 773 } where the default Helper.fuzzValue() is an empty method, but the test would patch in its own version of Helper that changes the ASB field values. I like this less than refactoring the checks to StringUTF16. dl > dl > >> Best regards, >> Vladimir Ivanov >> >>>> On 3/22/17 8:35 AM, Vladimir Ivanov wrote: >>>>>>>> So are we convinced that the proposed changes will never lead to a >>>>>>>> crash due to a missing or incorrect bounds check, due to a racy >>>>>>>> use of >>>>>>>> an unsynchronized ASB instance e.g. StringBuilder? >>>>>>> >>>>>>> If only we had a static analysis tool that could tell us if the >>>>>>> code is >>>>>>> safe. Because we don't, in my initial changeset, we always take a >>>>>>> snapshot of the ASB fields by passing those field values to >>>>>>> StringUTF16 >>>>>>> before doing checks on them. And I wrote a test to make sure that >>>>>>> those >>>>>>> StringUTF16 interfaces are catching all the underflows and >>>>>>> overflows I >>>>>>> could imagine, and I added verification code to detect when a check >>>>>>> was >>>>>>> missed. >>>>>>> >>>>>>> However, all the reviewers have requested to minimize the amount of >>>>>>> changes. In Vladimir's version, if there is a missing check >>>>>>> somewhere, >>>>>>> then yes it could lead to a crash. >>>>> >>>>> I'd like to point out that asserts and verification code are disabled >>>>> by default. They are invaluable during problem diagnosis, but don't >>>>> help at all from defence-in-depth perspective. >>>>> >>>>> But I agree that it's easier to reason about and test the initial >>>>> version of the fix. >>>>> >>>>>> I wonder if the reviewers have fully realized the potential impact >>>>>> here? >>>>>> This has exposed a flaw in the way intrinsics are used from core >>>>>> classes. >>>>> >>>>> FTR here are the checks I omitted in the minimized version (modulo >>>>> separation of indexOf/lastIndexOf for trusted/non-trusted callers): >>>>> >>>>> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ >>>>> >>>>> Other than that, the difference is mainly about undoing refactorings >>>>> and removing verification logic (asserts + checks in the JVM). >>>>> >>>>> There are still unsafe accesses which are considered safe in both >>>>> versions (see StringUTF16.Trusted usages in the initial version [1]). >>>>> >>>>> We used to provide safe wrappers for unsafe intrinsics which makes it >>>>> much easier to reason about code correctness. I'd like to see compact >>>>> string code refactored that way and IMO the initial version by Dean >>>>> is a big step in the right direction. >>>>> >>>>> I still prefer to see a point fix in 9 and major refactoring >>>>> happening in 10, but I'll leave the decision on how to proceed with >>>>> the fix to core-libs folks. After finishing the exercise minimizing >>>>> the fix, I'm much more comfortable with the initial fix [1] (though >>>>> there are changes I consider excessive). >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>> [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 >>>>> >>>>>>>>>> Some clarifications: >>>>>>>>>> >>>>>>>>>> ============ >>>>>>>>>> src/java.base/share/classes/java/lang/String.java: >>>>>>>>>> >>>>>>>>>> The bounds check is needed only in String.nonSyncContentEquals >>>>>>>>>> when it >>>>>>>>>> extracts info from AbstractStringBuilder. I don't see how out of >>>>>>>>>> bounds access can happen in String.contentEquals: >>>>>>>>>> if (n != length()) { >>>>>>>>>> return false; >>>>>>>>>> } >>>>>>>>>> ... >>>>>>>>>> for (int i = 0; i < n; i++) { >>>>>>>>>> if (StringUTF16.getChar(val, i) != >>>>>>>>>> cs.charAt(i)) { >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK. >>>>>>>>> >>>>>>>>>> ============ >>>>>>>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>>>>>>> >>>>>>>>>> I think bounds checks in StringConcatHelper.prepend() are >>>>>>>>>> skipped >>>>>>>>>> intentionally, since java.lang.invoke.StringConcatFactory >>>>>>>>>> constructs >>>>>>>>>> method handle chains which already contain bounds checks: array >>>>>>>>>> length >>>>>>>>>> is precomputed based on argument values and all accesses are >>>>>>>>>> guaranteed to be in bounds. >>>>>>>>>> >>>>>>>>> >>>>>>>>> This is calling the trusted version of getChars() with no bounds >>>>>>>>> checks. It was a little more obvious when I had the Trusted >>>>>>>>> inner >>>>>>>>> class. >>>>>>>>> >>>>>>>>>> ============ >>>>>>>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>> >>>>>>>>>> + static void putChar(byte[] val, int index, int c) { >>>>>>>>>> + assert index >= 0 && index < length(val) : "Trusted >>>>>>>>>> caller >>>>>>>>>> missed bounds check"; >>>>>>>>>> >>>>>>>>>> Unfortunately, asserts can affect inlining decisions (since they >>>>>>>>>> increase bytecode size). In order to minimize possible >>>>>>>>>> performance >>>>>>>>>> impact, I suggest to remove them from the fix targeting 9. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Sure. >>>>>>>>> >>>>>>>>>> ============ >>>>>>>>>> private static int indexOfSupplementary(byte[] value, int >>>>>>>>>> ch, int >>>>>>>>>> fromIndex, int max) { >>>>>>>>>> if (Character.isValidCodePoint(ch)) { >>>>>>>>>> final char hi = Character.highSurrogate(ch); >>>>>>>>>> final char lo = Character.lowSurrogate(ch); >>>>>>>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>>>>>>> >>>>>>>>>> The check is redundant here. fromIndex & max are always >>>>>>>>>> inbounds by >>>>>>>>>> construction: >>>>>>>>>> >>>>>>>>>> public static int indexOf(byte[] value, int ch, int >>>>>>>>>> fromIndex) { >>>>>>>>>> int max = value.length >> 1; >>>>>>>>>> if (fromIndex < 0) { >>>>>>>>>> fromIndex = 0; >>>>>>>>>> } else if (fromIndex >= max) { >>>>>>>>>> // Note: fromIndex might be near -1>>>1. >>>>>>>>>> return -1; >>>>>>>>>> } >>>>>>>>>> ... >>>>>>>>>> return indexOfSupplementary(value, ch, fromIndex, >>>>>>>>>> max); >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK. >>>>>>>>> >>>>>>>>>> ============ >>>>>>>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>>>>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range >>>>>>>>>> check on >>>>>>>>>> ABS.value & ABS.count. After that, all accesses should be >>>>>>>>>> inbounds by >>>>>>>>>> construction (in String.indexOf/lastIndexOf): >>>>>>>>>> >>>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>> static int lastIndexOf(byte[] src, byte srcCoder, int >>>>>>>>>> srcCount, >>>>>>>>>> String tgtStr, int fromIndex) { >>>>>>>>>> >>>>>>>>>> int rightIndex = srcCount - tgtCount; >>>>>>>>>> if (fromIndex > rightIndex) { >>>>>>>>>> fromIndex = rightIndex; >>>>>>>>>> } >>>>>>>>>> if (fromIndex < 0) { >>>>>>>>>> return -1; >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>>>>>>> byte[] tgt, int tgtCount, int >>>>>>>>>> fromIndex) { >>>>>>>>>> int min = tgtCount - 1; >>>>>>>>>> int i = min + fromIndex; >>>>>>>>>> int strLastIndex = tgtCount - 1; >>>>>>>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>>>>>>> >>>>>>>>>> startSearchForLastChar: >>>>>>>>>> while (true) { >>>>>>>>>> while (i >= min && getChar(src, i) != strLastChar) { >>>>>>>>>> >>>>>>>>>> There are 2 places: >>>>>>>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - >>>>>>>>>> inbound >>>>>>>>>> >>>>>>>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>>>>>>> min = tgtCount - 1 >>>>>>>>>> rightIndex = srcCount - tgtCount >>>>>>>>>> fromIndex <= rightIndex >>>>>>>>>> >>>>>>>>>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount >>>>>>>>>> - 1) >>>>>>>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>>>>>>> >>>>>>>>>> Hence, should be covered by the check on count & value: >>>>>>>>>> public int lastIndexOf(String str, int fromIndex) { >>>>>>>>>> + byte[] value = this.value; >>>>>>>>>> + int count = this.count; >>>>>>>>>> + byte coder = this.coder; >>>>>>>>>> + checkIndex(count, value.length >> coder); >>>>>>>>>> return String.lastIndexOf(value, coder, count, str, >>>>>>>>>> fromIndex); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK, I will go with your version if it's OK with Sherman. >>>>>>>>> >>>>>>>>> dl >>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Vladimir Ivanov >>>>>>>>>> >>>>>>>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>>>>>>> >>>>>>>>>>>>>> I have the same concern. Can we fix the immediate problem in >>>>>>>>>>>>>> 9 and >>>>>>>>>>>>>> integrate verification logic in 10? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> OK, Tobias is suggesting having verification logic only >>>>>>>>>>>>> inside the >>>>>>>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>>>>>>> >>>>>>>>>>>> Yes and put them back in 10. >>>>>>>>>>>> >>>>>>>>>>>>> I'm OK with removing all the verification, but that won't >>>>>>>>>>>>> reduce >>>>>>>>>>>>> the >>>>>>>>>>>>> library changes much. I could undo the renaming to >>>>>>>>>>>>> Trusted.getChar, but >>>>>>>>>>>>> we would still have the bounds checks moved into StringUTF16. >>>>>>>>>>>> >>>>>>>>>>>> I suggest to go with a point fix for 9: just add missing range >>>>>>>>>>>> checks. >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >>> > From aph at redhat.com Thu Mar 23 19:05:33 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 23 Mar 2017 19:05:33 +0000 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: <595FE441-10E0-4AB6-B7EE-686132F2269C@oracle.com> References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> <1FBB8985-1FE6-44E1-B338-7C6D49372D93@oracle.com> <595FE441-10E0-4AB6-B7EE-686132F2269C@oracle.com> Message-ID: On 23/03/17 18:58, Bob Vandette wrote: > What does your aarch64 build have as IMPLEMENTOR in it?s ?release? file. > I wonder if we can use this difference for now? > > IMPLEMENTOR="Oracle Corporation? At the moment, SOURCE=".:d660c00d91bf corba:ff8cb43c07c0 hotspot:9bfa2cceba90+ jaxp:8c9a2a24752b jaxws:92aa05eff5d1 jdk:a2d3b7f65c95 langtools:69e2e4d7136c nashorn:4a07ebdf8b45" IMPLEMENTOR="N/A" I don't know where that file comes from or even what it means. It would be OK, but what would we put there? "OpenJDK"? They're both supposed to be OpenJDK projects. Andrew. From bob.vandette at oracle.com Thu Mar 23 19:27:27 2017 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 23 Mar 2017 15:27:27 -0400 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> <1FBB8985-1FE6-44E1-B338-7C6D49372D93@oracle.com> <595FE441-10E0-4AB6-B7EE-686132F2269C@oracle.com> Message-ID: <96A77F56-4F35-449D-83EB-CE19FDEEF768@oracle.com> I?m not sure of the exact meaning of these variables but the logic that generates this file is in jdk9/make/ReleaseFile.gmk. I?ll look into the intended purpose. Perhaps the fact that you have N/A and our build is Oracle might be good enough. Bob. > On Mar 23, 2017, at 3:05 PM, Andrew Haley wrote: > > On 23/03/17 18:58, Bob Vandette wrote: >> What does your aarch64 build have as IMPLEMENTOR in it?s ?release? file. >> I wonder if we can use this difference for now? >> >> IMPLEMENTOR="Oracle Corporation? > > At the moment, > > SOURCE=".:d660c00d91bf corba:ff8cb43c07c0 hotspot:9bfa2cceba90+ jaxp:8c9a2a24752b jaxws:92aa05eff5d1 jdk:a2d3b7f65c95 langtools:69e2e4d7136c nashorn:4a07ebdf8b45" > IMPLEMENTOR="N/A" > > I don't know where that file comes from or even what it means. > It would be OK, but what would we put there? "OpenJDK"? They're > both supposed to be OpenJDK projects. > > Andrew. > From igor.ignatyev at oracle.com Thu Mar 23 20:07:42 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 23 Mar 2017 13:07:42 -0700 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests In-Reply-To: References: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> <3056CF52-47EB-4C5D-B650-81CDCFFD568F@oracle.com> <9fb37d41-528b-e040-4114-a154e55c777b@oracle.com> Message-ID: <1FB79522-93C4-4FC5-B351-935D1355C67C@oracle.com> Mandy/Daniel, yes, jdk.management.agent does require java.management, but not transitively. Shura and I have discussed that and agreed that in such cases a test should declare dependency explicitly, otherwise it can start to fail when some of transitive requires (which are not a part of the contract) are changed. I used jdeps with the post-proceccing which makes reduction similar to list-reduced-deps. I have run 'jdeps --list-reduced-deps' on classes from sun/management/jmxremote/bootstrap/CustomLauncherTest.java test run, and it showed the same: > java.management > jdk.attach > jdk.management.agent/jdk.internal.agent > unnamed module: /tmp/run/jdk/sun/management/jmxremote/bootstrap/JTwork-sun-management-jmxremote-bootstrap-CustomLauncherTest-java_0/classes Thanks, -- Igor > On Mar 23, 2017, at 9:39 AM, Mandy Chung wrote: > > >> On Mar 23, 2017, at 7:33 AM, Daniel Fuchs wrote: >> >> Hi Igor, >> >> small nit: >> >> 42 * @modules java.management >> 43 * jdk.attach >> 44 * jdk.management.agent/jdk.internal.agent >> >> I don't think java.management needs to be specified as >> a dependency when the test requires jdk.management.agent, >> because jdk.management.agent already requires java.management. > > That?s true. > > Igor - How do you analyze the dependency? Are you using jdeps ?-list-reduced-deps? > > Mandy > From mandy.chung at oracle.com Thu Mar 23 20:28:20 2017 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 23 Mar 2017 13:28:20 -0700 Subject: RFR(S) : 8177374 : fix module dependency declaration in jdk_svc tests In-Reply-To: <1FB79522-93C4-4FC5-B351-935D1355C67C@oracle.com> References: <0DADE381-4D06-42B9-8BD4-28C274E87BC4@oracle.com> <3056CF52-47EB-4C5D-B650-81CDCFFD568F@oracle.com> <9fb37d41-528b-e040-4114-a154e55c777b@oracle.com> <1FB79522-93C4-4FC5-B351-935D1355C67C@oracle.com> Message-ID: <3F51C4BA-8557-43D1-8DE8-73FE159F1628@oracle.com> > On Mar 23, 2017, at 1:07 PM, Igor Ignatyev wrote: > > Mandy/Daniel, > > yes, jdk.management.agent does require java.management, but not transitively. Shura and I have discussed that and agreed that in such cases a test should declare dependency explicitly, otherwise it can start to fail when some of transitive requires (which are not a part of the contract) are changed. > > I used jdeps with the post-proceccing which makes reduction similar to list-reduced-deps. I have run 'jdeps --list-reduced-deps' on classes from sun/management/jmxremote/bootstrap/CustomLauncherTest.java test run, and it showed the same: >> java.management >> jdk.attach >> jdk.management.agent/jdk.internal.agent >> unnamed module: /tmp/run/jdk/sun/management/jmxremote/bootstrap/JTwork-sun-management-jmxremote-bootstrap-CustomLauncherTest-java_0/classes > It?s good you are using `jdeps --list-reduced-deps`. It includes java.management because the test references the exported APIs from java.management. I agree that @modules should list java.management even though the test selection would work properly without it. Mandy From gromero at linux.vnet.ibm.com Thu Mar 23 22:24:51 2017 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 23 Mar 2017 19:24:51 -0300 Subject: Request for clarification about changes in Hotspot across 8, 9, and 10 Message-ID: <58D44B33.6030507@linux.vnet.ibm.com> Hi, I understand the current status of OpenJDK 8, 9, and 10 regarding changes in Hotspot as: jdk8u: "We're open for fixes for 8u152 in the jdk8u-dev forest" [1] jdk9 : We passed Rampdown Phase 2 (2017/03/16) so "only showstopper bugs can be fixed" [2] jdk10: "As of today these forests are open for bug fixes and small enhancements" [3] Distros in general will repackage (update) OpenJDK following the CPU cycles [4]. The next CPU release is on 18 April. If a change is already in jdk8u when repackaging due to the CPU release occurs, then distros can - under request - take the change into the new updated OpenJDK 8 package. But to get a change into jdk8u I need to get it first into 10, then request backport to 9, and finally request a backport to 8u. However, jdk9 passed the Rampdown Phase 2, so the change can't get into 9 and hence can't get into jdk8u by now. So it looks like jdk9 current status is blocking the path to jdk8u. If the change (bug fix or enhancement) is PPC64-only (let's say a change in the ppc.ad file), is it possible to get it into 8u before 18 April (next CPU release) somehow? If not, what are the options? Is it necessary to wait 9 GA and so jdk9u? Thank you very much. Best regards, Gustavo [1] http://openjdk.java.net/projects/jdk8u/ [2] http://openjdk.java.net/projects/jdk9/ [3] http://mail.openjdk.java.net/pipermail/jdk10-dev/2017-January/000041.html [4] http://www.oracle.com/technetwork/es/topics/security/alerts-086861.html From david.holmes at oracle.com Fri Mar 24 01:36:23 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 24 Mar 2017 11:36:23 +1000 Subject: Request for clarification about changes in Hotspot across 8, 9, and 10 In-Reply-To: <58D44B33.6030507@linux.vnet.ibm.com> References: <58D44B33.6030507@linux.vnet.ibm.com> Message-ID: Hi Gustavo, This is really a question for jdk8u-dev - cc'd. On 24/03/2017 8:24 AM, Gustavo Romero wrote: > Hi, > > I understand the current status of OpenJDK 8, 9, and 10 regarding changes in > Hotspot as: > > jdk8u: "We're open for fixes for 8u152 in the jdk8u-dev forest" [1] > jdk9 : We passed Rampdown Phase 2 (2017/03/16) so "only showstopper bugs can be fixed" [2] > jdk10: "As of today these forests are open for bug fixes and small enhancements" [3] > > Distros in general will repackage (update) OpenJDK following the CPU cycles [4]. > The next CPU release is on 18 April. If a change is already in jdk8u when > repackaging due to the CPU release occurs, then distros can - under request - > take the change into the new updated OpenJDK 8 package. > > But to get a change into jdk8u I need to get it first into 10, then request > backport to 9, and finally request a backport to 8u. However, jdk9 passed the > Rampdown Phase 2, so the change can't get into 9 and hence can't get into jdk8u > by now. So it looks like jdk9 current status is blocking the path to jdk8u. I raised this recently in relation to a RFE that was being backported to 8u and thus could not now go into 9: http://mail.openjdk.java.net/pipermail/jdk8u-dev/2017-March/006486.html The "conclusion" was that this would be re-examined as the need arose. So if you have need ... > If the change (bug fix or enhancement) is PPC64-only (let's say a change in the > ppc.ad file), is it possible to get it into 8u before 18 April (next CPU release) > somehow? If not, what are the options? Is it necessary to wait 9 GA and so jdk9u? I don't know how you handle producing eg PPC64 builds to match a given CPU release. I would have presumed you would simply wait till the CPU changes are pushed to jdk8u-dev and then produce a build containing all CPU fixes along with any other fixes already accumulating in 8u-dev ? David ----- > Thank you very much. > > Best regards, > Gustavo > > [1] http://openjdk.java.net/projects/jdk8u/ > [2] http://openjdk.java.net/projects/jdk9/ > [3] http://mail.openjdk.java.net/pipermail/jdk10-dev/2017-January/000041.html > [4] http://www.oracle.com/technetwork/es/topics/security/alerts-086861.html > From david.holmes at oracle.com Fri Mar 24 01:42:31 2017 From: david.holmes at oracle.com (David Holmes) Date: Fri, 24 Mar 2017 11:42:31 +1000 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: <96A77F56-4F35-449D-83EB-CE19FDEEF768@oracle.com> References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> <1FBB8985-1FE6-44E1-B338-7C6D49372D93@oracle.com> <595FE441-10E0-4AB6-B7EE-686132F2269C@oracle.com> <96A77F56-4F35-449D-83EB-CE19FDEEF768@oracle.com> Message-ID: This issue seems to be fragmenting, given that Mario has put out a RFR on jdk9-dev based on the VENDOR_URL! We can easily change -Xinternalversion output, which would also affect hs-err log file. David On 24/03/2017 5:27 AM, Bob Vandette wrote: > I?m not sure of the exact meaning of these variables but the logic that > generates this file is in jdk9/make/ReleaseFile.gmk. I?ll look into > the intended purpose. > > Perhaps the fact that you have N/A and our build is Oracle might be good enough. > > Bob. > > >> On Mar 23, 2017, at 3:05 PM, Andrew Haley wrote: >> >> On 23/03/17 18:58, Bob Vandette wrote: >>> What does your aarch64 build have as IMPLEMENTOR in it?s ?release? file. >>> I wonder if we can use this difference for now? >>> >>> IMPLEMENTOR="Oracle Corporation? >> >> At the moment, >> >> SOURCE=".:d660c00d91bf corba:ff8cb43c07c0 hotspot:9bfa2cceba90+ jaxp:8c9a2a24752b jaxws:92aa05eff5d1 jdk:a2d3b7f65c95 langtools:69e2e4d7136c nashorn:4a07ebdf8b45" >> IMPLEMENTOR="N/A" >> >> I don't know where that file comes from or even what it means. >> It would be OK, but what would we put there? "OpenJDK"? They're >> both supposed to be OpenJDK projects. >> >> Andrew. >> > From robbin.ehn at oracle.com Fri Mar 24 07:58:48 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 24 Mar 2017 08:58:48 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: <58D2A436.4040203@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> <58D2A436.4040203@oracle.com> Message-ID: <12af8937-c462-3d1b-a7cb-f2cd113a7048@oracle.com> Hi Mikeal, On 03/22/2017 05:20 PM, Erik ?sterlund wrote: > Hi Mikael, > > In my workflow I use emacs with rtags for snytax awareness. > With your cmake patch, I could easily generate the cmake files and run cmake with -DCMAKE_EXPORT_COMPILE_COMMANDS to generate the compiler_commands.json file used by rtags > to understand how the code fits together, and feed it to rdm. Everything worked out of the box. I also use rtags, but with vim. > > With these changes it was faster than ever to setup a good emacs-based project for hotspot development. So I encourage these changes! I have not tested this, but it must be faster, since now I compile the source twice, once with gcc and once with clang for rtags. Can cmake do incremental updates to compiler_command.json file, would be awesome if so? (Erik?) Thanks for doing this! /Robbin > > Thanks, > /Erik > > On 2017-03-22 15:21, Mikael Gerdin wrote: >> Hi all, >> >> I've finally grown tired of manually setting up a hand crafted Eclipse CDT configuration for the JVM sources and decided to share my progress towards improving the >> overall situation for JVM developers. >> >> To achieve better IDE support without having to add project generators for all different kinds of past or future IDEs I've decided to try to leverage CMake to do project >> generation. >> The idea is to have the JDK build system generate a CMakeLists.txt describing all the include paths and definitions required by an IDE to interpret the sources correctly. >> >> Several modern IDEs natively support CMake but we can also rely on the fact that the CMake build system has the ability to generate projects for a number of different >> IDEs. For information about which generators CMake supports see >> https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html >> for your CMake version. >> >> To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in the jdk10/sandbox forest: >> http://hg.openjdk.java.net/jdk10/sandbox/branches >> So far I've only made changes in the toplevel and hotspot repositories. >> I've written a short readme in the repo: >> http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html >> >> It would be great if people tried this out to see if it works on their setup but I don't expect it to work on Windows without changing the makefile to do path conversion. >> If we can make this work properly then perhaps we can get rid of the Visual Studio generator and rely on CMake to generate VS projects. >> >> It would also be great if someone from build-dev could give me some hints about how to do the file writing and "vardeps" thing properly. >> >> Thanks >> /Mikael > From robbin.ehn at oracle.com Fri Mar 24 08:02:00 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 24 Mar 2017 09:02:00 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: <12af8937-c462-3d1b-a7cb-f2cd113a7048@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> <58D2A436.4040203@oracle.com> <12af8937-c462-3d1b-a7cb-f2cd113a7048@oracle.com> Message-ID: Hi, again, On 03/24/2017 08:58 AM, Robbin Ehn wrote: > Hi Mikeal, > > On 03/22/2017 05:20 PM, Erik ?sterlund wrote: >> Hi Mikael, >> >> In my workflow I use emacs with rtags for snytax awareness. >> With your cmake patch, I could easily generate the cmake files and run cmake with -DCMAKE_EXPORT_COMPILE_COMMANDS to generate the compiler_commands.json file used by rtags >> to understand how the code fits together, and feed it to rdm. Everything worked out of the box. > > I also use rtags, but with vim. > >> >> With these changes it was faster than ever to setup a good emacs-based project for hotspot development. So I encourage these changes! > > I have not tested this, but it must be faster, since now I compile the source twice, once with gcc and once with clang for rtags. I have still gotten my morning coffee, this step I have to do anyways of course. > > Can cmake do incremental updates to compiler_command.json file, would be awesome if so? (Erik?) This is what I need, now I can't do incremental builds if change any flags or add files. /Robbin > > Thanks for doing this! > > /Robbin > >> >> Thanks, >> /Erik >> >> On 2017-03-22 15:21, Mikael Gerdin wrote: >>> Hi all, >>> >>> I've finally grown tired of manually setting up a hand crafted Eclipse CDT configuration for the JVM sources and decided to share my progress towards improving the >>> overall situation for JVM developers. >>> >>> To achieve better IDE support without having to add project generators for all different kinds of past or future IDEs I've decided to try to leverage CMake to do project >>> generation. >>> The idea is to have the JDK build system generate a CMakeLists.txt describing all the include paths and definitions required by an IDE to interpret the sources correctly. >>> >>> Several modern IDEs natively support CMake but we can also rely on the fact that the CMake build system has the ability to generate projects for a number of different >>> IDEs. For information about which generators CMake supports see >>> https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html >>> for your CMake version. >>> >>> To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in the jdk10/sandbox forest: >>> http://hg.openjdk.java.net/jdk10/sandbox/branches >>> So far I've only made changes in the toplevel and hotspot repositories. >>> I've written a short readme in the repo: >>> http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html >>> >>> It would be great if people tried this out to see if it works on their setup but I don't expect it to work on Windows without changing the makefile to do path conversion. >>> If we can make this work properly then perhaps we can get rid of the Visual Studio generator and rely on CMake to generate VS projects. >>> >>> It would also be great if someone from build-dev could give me some hints about how to do the file writing and "vardeps" thing properly. >>> >>> Thanks >>> /Mikael >> From robbin.ehn at oracle.com Fri Mar 24 08:03:19 2017 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 24 Mar 2017 09:03:19 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> <58D2A436.4040203@oracle.com> <12af8937-c462-3d1b-a7cb-f2cd113a7048@oracle.com> Message-ID: <6f64c3df-f09a-8728-b2d8-dff5c0c957f4@oracle.com> Hmmz, brain not functioning. On 03/24/2017 09:02 AM, Robbin Ehn wrote: > I have still gotten my morning coffee, this step I have to do anyways of course. NOT /Robbin From aph at redhat.com Fri Mar 24 08:07:17 2017 From: aph at redhat.com (Andrew Haley) Date: Fri, 24 Mar 2017 08:07:17 +0000 Subject: [aarch64-port-dev ] RFR: 8168503 JEP 297: Unified arm32/arm64 Port In-Reply-To: References: <8BACE784-C921-4C0E-90E0-C7446859C367@oracle.com> <58421A35.5070706@oracle.com> <35F08BAA-BE09-4A63-A4AB-EA71F3FA808F@oracle.com> <84d6c7ff-cef6-c245-660a-7cb61074b1a4@oracle.com> <530D0083-BD1C-41CF-A782-42935ACCDCFB@oracle.com> <4082474e-c3bf-ec2b-99d6-dc8a995c16fe@redhat.com> <5f79ca1a-18a9-4f52-6332-205ffff2e1a0@redhat.com> <1FBB8985-1FE6-44E1-B338-7C6D49372D93@oracle.com> <595FE441-10E0-4AB6-B7EE-686132F2269C@oracle.com> <96A77F56-4F35-449D-83EB-CE19FDEEF768@oracle.com> Message-ID: <5eaabe83-75f0-221a-910e-b73954c4f250@redhat.com> On 24/03/17 01:42, David Holmes wrote: > This issue seems to be fragmenting, given that Mario has put out a RFR > on jdk9-dev based on the VENDOR_URL! > > We can easily change -Xinternalversion output, which would also affect > hs-err log file. It's not fragmenting at all: Mario's idea looks decent enough. I would love you to explain what to do with -Xinternalversion. That would be awesome. It'd be nice to have something that can easily be read programmatically. Andrew. From mikael.gerdin at oracle.com Fri Mar 24 08:47:05 2017 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Fri, 24 Mar 2017 09:47:05 +0100 Subject: C/C++ IDE support for HotSpot In-Reply-To: References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> <58D2A436.4040203@oracle.com> <12af8937-c462-3d1b-a7cb-f2cd113a7048@oracle.com> Message-ID: <58435167-6ff3-1493-a1cd-9940a6b5b856@oracle.com> Hi Robbin, On 2017-03-24 09:02, Robbin Ehn wrote: > Hi, again, > > On 03/24/2017 08:58 AM, Robbin Ehn wrote: >> Hi Mikeal, >> >> On 03/22/2017 05:20 PM, Erik ?sterlund wrote: >>> Hi Mikael, >>> >>> In my workflow I use emacs with rtags for snytax awareness. >>> With your cmake patch, I could easily generate the cmake files and >>> run cmake with -DCMAKE_EXPORT_COMPILE_COMMANDS to generate the >>> compiler_commands.json file used by rtags >>> to understand how the code fits together, and feed it to rdm. >>> Everything worked out of the box. >> >> I also use rtags, but with vim. >> >>> >>> With these changes it was faster than ever to setup a good >>> emacs-based project for hotspot development. So I encourage these >>> changes! >> >> I have not tested this, but it must be faster, since now I compile the >> source twice, once with gcc and once with clang for rtags. > > I have still gotten my morning coffee, this step I have to do anyways of > course. > >> >> Can cmake do incremental updates to compiler_command.json file, would >> be awesome if so? (Erik?) > > This is what I need, now I can't do incremental builds if change any > flags or add files. I'm not sure how the generation of compiler_command.json works but to refresh the CMakeLists.txt file you have to invoke the hotspot-cmake-project make target again. This should pick up new files or flags from the build system. There is a way to add dependencies on build system variables so that the refresh would only actuallly run when needed but I couldn't figure that out so for now the CMakeLists generation is .PHONY-target. /Mikael > > /Robbin > >> >> Thanks for doing this! >> >> /Robbin >> >>> >>> Thanks, >>> /Erik >>> >>> On 2017-03-22 15:21, Mikael Gerdin wrote: >>>> Hi all, >>>> >>>> I've finally grown tired of manually setting up a hand crafted >>>> Eclipse CDT configuration for the JVM sources and decided to share >>>> my progress towards improving the >>>> overall situation for JVM developers. >>>> >>>> To achieve better IDE support without having to add project >>>> generators for all different kinds of past or future IDEs I've >>>> decided to try to leverage CMake to do project >>>> generation. >>>> The idea is to have the JDK build system generate a CMakeLists.txt >>>> describing all the include paths and definitions required by an IDE >>>> to interpret the sources correctly. >>>> >>>> Several modern IDEs natively support CMake but we can also rely on >>>> the fact that the CMake build system has the ability to generate >>>> projects for a number of different >>>> IDEs. For information about which generators CMake supports see >>>> https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html >>>> for your CMake version. >>>> >>>> To try this out check out (heh) my branch "JDK-8177329-cmake-branch" >>>> in the jdk10/sandbox forest: >>>> http://hg.openjdk.java.net/jdk10/sandbox/branches >>>> So far I've only made changes in the toplevel and hotspot repositories. >>>> I've written a short readme in the repo: >>>> http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html >>>> >>>> >>>> It would be great if people tried this out to see if it works on >>>> their setup but I don't expect it to work on Windows without >>>> changing the makefile to do path conversion. >>>> If we can make this work properly then perhaps we can get rid of the >>>> Visual Studio generator and rely on CMake to generate VS projects. >>>> >>>> It would also be great if someone from build-dev could give me some >>>> hints about how to do the file writing and "vardeps" thing properly. >>>> >>>> Thanks >>>> /Mikael >>> From ch-hagedorn at hispeed.ch Fri Mar 24 09:34:11 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Fri, 24 Mar 2017 10:34:11 +0100 Subject: AW: Adding native method in custom jdk package In-Reply-To: <27efece2-e51c-78cf-86c3-a3ce935bd137@oracle.com> References: <000c01d2a329$f4a11630$dde34290$@hispeed.ch> <001c01d2a337$2e31a560$8a94f020$@hispeed.ch> <27efece2-e51c-78cf-86c3-a3ce935bd137@oracle.com> Message-ID: <000601d2a481$cc442480$64cc6d80$@hispeed.ch> Hi David, Yes you are right, now I see that the problem was just a build setting. I thought I miss some other steps, too. Best regards, Christian -----Urspr?ngliche Nachricht----- Von: David Holmes [mailto:david.holmes at oracle.com] Gesendet: Donnerstag, 23. M?rz 2017 01:22 An: Christian Hagedorn; hotspot-dev at openjdk.java.net Betreff: Re: Adding native method in custom jdk package On 23/03/2017 4:07 AM, Christian Hagedorn wrote: > Thanks for your quick help. That got me on the right track, it does > work now. I added my path to the LIBJAVA_SRC_DIRS in the > ?jdk/make/lib/CoreLibraries.gmk? file. Yes that was the missing part. But this is a build question, or at best a JDK question, not a hotspot question. Regards, David > > > Best regards, > > Christian > > > > Von: Bob Vandette [mailto:bob.vandette at oracle.com] > Gesendet: Mittwoch, 22. M?rz 2017 18:23 > An: Christian Hagedorn > Cc: hotspot-dev at openjdk.java.net > Betreff: Re: Adding native method in custom jdk package > > > > Checkout this blog that provides a simple example of how to create and > call a native > > function in a shared library. > > > > https://blogs.oracle.com/moonocean/entry/a_simple_example_of_jni > > > > Let me know if you have any questions, > > Bob. > > > > On Mar 22, 2017, at 12:32 PM, Christian Hagedorn > > wrote: > > > > Hi, > > > > I have created a class Test.java in an own package test.lang inside > the jdk ("jdk/src/share/classes/test/lang/Test.java"). Now I wanted to > add a native method. > > I found out that my header was created inside" > build/linux./jdk/gensrc_headers/test_lang_Test.h". > > So I have tried to just create a new file inside a new folder > "jdk/src/share/native/test/lang/Test.c" including "test_lang_Test.h". > > From there I do my JNICALL to the VM like it is done for example in > "jdk/src/share/native/java/lang/System.c". > > However this still results in a UnsatisfiedLinkError. I am sure I have > to do some more steps to link and add it to the build properly. What am I missing? > > > > Best regards, > > Christian > > > > > > From matthias.baesken at sap.com Fri Mar 24 11:28:01 2017 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 24 Mar 2017 11:28:01 +0000 Subject: [XXS] RFR : 8177531 libGetNamedModuleTest.c crash when printing NULL-pointer Message-ID: <2d15fa270c254b189027e337f34db5aa@sap.com> Hello, I noticed the hotspot test test/serviceability/jvmti/GetNamedModule/libGetNamedModuleTest.c crashes on Solaris (x86_64) , when in function get_module, printf(" ... %s ...", ....) on a NULL-pointer argument is called . The following very small fix adjusts this . Bug : https://bugs.openjdk.java.net/browse/JDK-8177531 change for 9 : http://cr.openjdk.java.net/~mbaesken/webrevs/8177531/ Please review this small test change (needed for 9 and 10 ). Thanks and regards, Matthias From thomas.stuefe at gmail.com Fri Mar 24 11:39:42 2017 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 24 Mar 2017 12:39:42 +0100 Subject: [XXS] RFR : 8177531 libGetNamedModuleTest.c crash when printing NULL-pointer In-Reply-To: <2d15fa270c254b189027e337f34db5aa@sap.com> References: <2d15fa270c254b189027e337f34db5aa@sap.com> Message-ID: Hi Matthias, looks fine. Thanks for fixing this. Kind Regards, Thomas On Fri, Mar 24, 2017 at 12:28 PM, Baesken, Matthias < matthias.baesken at sap.com> wrote: > Hello, I noticed the hotspot test test/serviceability/jvmti/ > GetNamedModule/libGetNamedModuleTest.c > > crashes on Solaris (x86_64) , when in function get_module, printf(" > ... %s ...", ....) on a NULL-pointer argument is called . > > The following very small fix adjusts this . > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8177531 > > change for 9 : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8177531/ > > > Please review this small test change (needed for 9 and 10 ). > > > Thanks and regards, Matthias > > > > From volker.simonis at gmail.com Fri Mar 24 12:31:36 2017 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 24 Mar 2017 13:31:36 +0100 Subject: [XXS] RFR : 8177531 libGetNamedModuleTest.c crash when printing NULL-pointer In-Reply-To: <2d15fa270c254b189027e337f34db5aa@sap.com> References: <2d15fa270c254b189027e337f34db5aa@sap.com> Message-ID: Hi Matthias, thanks for fixing this. The change looks good. I've added the 'noreg-self' label as required by the "JDK 9 Rampdown Phase Two" rules [1]. With that, this change is ready to be pushed and we're just waiting for a gracious sponsor :) Regards, Volker [1] http://openjdk.java.net/projects/jdk9/rdp-2 On Fri, Mar 24, 2017 at 12:28 PM, Baesken, Matthias wrote: > Hello, I noticed the hotspot test test/serviceability/jvmti/GetNamedModule/libGetNamedModuleTest.c > > crashes on Solaris (x86_64) , when in function get_module, printf(" ... %s ...", ....) on a NULL-pointer argument is called . > > The following very small fix adjusts this . > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8177531 > > change for 9 : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8177531/ > > > Please review this small test change (needed for 9 and 10 ). > > > Thanks and regards, Matthias > > > From jesper.wilhelmsson at oracle.com Fri Mar 24 16:36:36 2017 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Fri, 24 Mar 2017 17:36:36 +0100 Subject: Reminder and status update Message-ID: <92d0df69-1118-97c2-39f8-5afa8423af87@oracle.com> Hi, Just a reminder that jdk9/hs is now closed for any and all pushes. The last snapshot has already been taken and changes pushed after the snapshot will not be integrated to jdk9/dev. The Thursday nightly looks good and I'm preparing the PIT submission right now. With any luck we'll be able to make jdk9/hs read only mid next week. Thanks, /Jesper From igor.ignatyev at oracle.com Fri Mar 24 20:56:34 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 24 Mar 2017 13:56:34 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified Message-ID: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 > 295 lines changed: 176 ins; 15 del; 104 mod; Hi all, could you please review this fix for 8177507? due to their nature, some of jdi tests are line number sensitive. unfortunately different tests indicate that differently, so it's quite easy to overlook that and incidentally break tests, for example by changing module dependency declaration or license modification. this fix unifies the way line number sensitivity is indicated and also improves readability/maintainability of some tests by using constant fields instead of magic numbers. some of line number sensitive tests have been unexpectedly removed from execution because they had @test/nodynamiccopyright/ instead of @test tag. this changeset fixes and returns them to regular execution. webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 testing: test/com/sun/jdi Thanks, -- Igor From serguei.spitsyn at oracle.com Fri Mar 24 23:44:40 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 24 Mar 2017 16:44:40 -0700 Subject: [XXS] RFR : 8177531 libGetNamedModuleTest.c crash when printing NULL-pointer In-Reply-To: References: <2d15fa270c254b189027e337f34db5aa@sap.com> Message-ID: <1a6800a8-3bb2-93a3-f465-b27bf3a2c81c@oracle.com> Hi Matthias, The change looks good to me. Thank you for fixing it! I'll sponsor the push. Thanks, Serguei On 3/24/17 05:31, Volker Simonis wrote: > Hi Matthias, > > thanks for fixing this. The change looks good. > I've added the 'noreg-self' label as required by the "JDK 9 Rampdown > Phase Two" rules [1]. > > With that, this change is ready to be pushed and we're just waiting > for a gracious sponsor :) > > Regards, > Volker > > [1] http://openjdk.java.net/projects/jdk9/rdp-2 > > On Fri, Mar 24, 2017 at 12:28 PM, Baesken, Matthias > wrote: >> Hello, I noticed the hotspot test test/serviceability/jvmti/GetNamedModule/libGetNamedModuleTest.c >> >> crashes on Solaris (x86_64) , when in function get_module, printf(" ... %s ...", ....) on a NULL-pointer argument is called . >> >> The following very small fix adjusts this . >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8177531 >> >> change for 9 : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8177531/ >> >> >> Please review this small test change (needed for 9 and 10 ). >> >> >> Thanks and regards, Matthias >> >> >> From sean.coffey at oracle.com Sat Mar 25 10:22:26 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Sat, 25 Mar 2017 10:22:26 +0000 Subject: Request for clarification about changes in Hotspot across 8, 9, and 10 In-Reply-To: References: <58D44B33.6030507@linux.vnet.ibm.com> Message-ID: Hi David, we're aware of this issue and are taking steps to address it. See http://mail.openjdk.java.net/pipermail/jdk8u-dev/2017-March/006512.html regards, Sean. On 24/03/2017 01:36, David Holmes wrote: > Hi Gustavo, > > This is really a question for jdk8u-dev - cc'd. > > On 24/03/2017 8:24 AM, Gustavo Romero wrote: >> Hi, >> >> I understand the current status of OpenJDK 8, 9, and 10 regarding >> changes in >> Hotspot as: >> >> jdk8u: "We're open for fixes for 8u152 in the jdk8u-dev >> forest" [1] >> jdk9 : We passed Rampdown Phase 2 (2017/03/16) so "only showstopper >> bugs can be fixed" [2] >> jdk10: "As of today these forests are open for bug fixes and small >> enhancements" [3] >> >> Distros in general will repackage (update) OpenJDK following the CPU >> cycles [4]. >> The next CPU release is on 18 April. If a change is already in jdk8u >> when >> repackaging due to the CPU release occurs, then distros can - under >> request - >> take the change into the new updated OpenJDK 8 package. >> >> But to get a change into jdk8u I need to get it first into 10, then >> request >> backport to 9, and finally request a backport to 8u. However, jdk9 >> passed the >> Rampdown Phase 2, so the change can't get into 9 and hence can't get >> into jdk8u >> by now. So it looks like jdk9 current status is blocking the path to >> jdk8u. > > I raised this recently in relation to a RFE that was being backported > to 8u and thus could not now go into 9: > > http://mail.openjdk.java.net/pipermail/jdk8u-dev/2017-March/006486.html > > The "conclusion" was that this would be re-examined as the need arose. > So if you have need ... > >> If the change (bug fix or enhancement) is PPC64-only (let's say a >> change in the >> ppc.ad file), is it possible to get it into 8u before 18 April (next >> CPU release) >> somehow? If not, what are the options? Is it necessary to wait 9 GA >> and so jdk9u? > > I don't know how you handle producing eg PPC64 builds to match a given > CPU release. I would have presumed you would simply wait till the CPU > changes are pushed to jdk8u-dev and then produce a build containing > all CPU fixes along with any other fixes already accumulating in 8u-dev ? > > David > ----- > >> Thank you very much. >> >> Best regards, >> Gustavo >> >> [1] http://openjdk.java.net/projects/jdk8u/ >> [2] http://openjdk.java.net/projects/jdk9/ >> [3] >> http://mail.openjdk.java.net/pipermail/jdk10-dev/2017-January/000041.html >> [4] >> http://www.oracle.com/technetwork/es/topics/security/alerts-086861.html >> From igor.ignatyev at oracle.com Wed Mar 29 00:42:34 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 28 Mar 2017 17:42:34 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: Message-ID: Hi, still looking for a reviewer, anyone? -- Igor > On Mar 24, 2017, at 1:56 PM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >> 295 lines changed: 176 ins; 15 del; 104 mod; > > Hi all, > > could you please review this fix for 8177507? > > due to their nature, some of jdi tests are line number sensitive. unfortunately different tests indicate that differently, so it's quite easy to overlook that and incidentally break tests, for example by changing module dependency declaration or license modification. this fix unifies the way line number sensitivity is indicated and also improves readability/maintainability of some tests by using constant fields instead of magic numbers. > > some of line number sensitive tests have been unexpectedly removed from execution because they had @test/nodynamiccopyright/ instead of @test tag. this changeset fixes and returns them to regular execution. > > webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 > testing: test/com/sun/jdi > > Thanks, > -- Igor From david.holmes at oracle.com Wed Mar 29 01:37:38 2017 From: david.holmes at oracle.com (David Holmes) Date: Wed, 29 Mar 2017 11:37:38 +1000 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: Message-ID: Hi Igor, This cleanup looks good to me! One query: test/com/sun/jdi/LineNumberOnBraceTest.java The old version seems to have the wrong line numbers - was it failing? Two nits: - test/com/sun/jdi/FetchLocals.java - test/com/sun/jdi/LambdaBreakpointTest.java Second copyright year should be 2017. Thanks, David ----- On 29/03/2017 10:42 AM, Igor Ignatyev wrote: > Hi, > > still looking for a reviewer, anyone? > > -- Igor >> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>> 295 lines changed: 176 ins; 15 del; 104 mod; >> >> Hi all, >> >> could you please review this fix for 8177507? >> >> due to their nature, some of jdi tests are line number sensitive. unfortunately different tests indicate that differently, so it's quite easy to overlook that and incidentally break tests, for example by changing module dependency declaration or license modification. this fix unifies the way line number sensitivity is indicated and also improves readability/maintainability of some tests by using constant fields instead of magic numbers. >> >> some of line number sensitive tests have been unexpectedly removed from execution because they had @test/nodynamiccopyright/ instead of @test tag. this changeset fixes and returns them to regular execution. >> >> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >> testing: test/com/sun/jdi >> >> Thanks, >> -- Igor > From gromero at linux.vnet.ibm.com Wed Mar 29 02:05:05 2017 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 28 Mar 2017 23:05:05 -0300 Subject: Request for clarification about changes in Hotspot across 8, 9, and 10 In-Reply-To: References: <58D44B33.6030507@linux.vnet.ibm.com> Message-ID: <58DB1651.1050803@linux.vnet.ibm.com> Hi David, Sean Sorry for the delay on replaying... On 25-03-2017 07:22, Se?n Coffey wrote: > Hi David, > > we're aware of this issue and are taking steps to address it. See > > http://mail.openjdk.java.net/pipermail/jdk8u-dev/2017-March/006512.html > > regards, > Sean. Sean, thanks for pointing out your proposal, it's much appreciated. > > On 24/03/2017 01:36, David Holmes wrote: >> Hi Gustavo, >> >> This is really a question for jdk8u-dev - cc'd. >> >> On 24/03/2017 8:24 AM, Gustavo Romero wrote: >>> Hi, >>> >>> I understand the current status of OpenJDK 8, 9, and 10 regarding changes in >>> Hotspot as: >>> >>> jdk8u: "We're open for fixes for 8u152 in the jdk8u-dev forest" [1] >>> jdk9 : We passed Rampdown Phase 2 (2017/03/16) so "only showstopper bugs can be fixed" [2] >>> jdk10: "As of today these forests are open for bug fixes and small enhancements" [3] >>> >>> Distros in general will repackage (update) OpenJDK following the CPU cycles [4]. >>> The next CPU release is on 18 April. If a change is already in jdk8u when >>> repackaging due to the CPU release occurs, then distros can - under request - >>> take the change into the new updated OpenJDK 8 package. >>> >>> But to get a change into jdk8u I need to get it first into 10, then request >>> backport to 9, and finally request a backport to 8u. However, jdk9 passed the >>> Rampdown Phase 2, so the change can't get into 9 and hence can't get into jdk8u >>> by now. So it looks like jdk9 current status is blocking the path to jdk8u. >> >> I raised this recently in relation to a RFE that was being backported to 8u and thus could not now go into 9: >> >> http://mail.openjdk.java.net/pipermail/jdk8u-dev/2017-March/006486.html >> >> The "conclusion" was that this would be re-examined as the need arose. So if you have need ... >> >>> If the change (bug fix or enhancement) is PPC64-only (let's say a change in the >>> ppc.ad file), is it possible to get it into 8u before 18 April (next CPU release) >>> somehow? If not, what are the options? Is it necessary to wait 9 GA and so jdk9u? >> >> I don't know how you handle producing eg PPC64 builds to match a given CPU release. I would have presumed you would simply wait till the CPU changes are pushed to jdk8u-dev and then produce a build >> containing all CPU fixes along with any other fixes already accumulating in 8u-dev ? Yes, except that I believe distros will just add the fixes accumulated in 8u-dev under a request/justification (in addition to all security fixes). Thanks! Regards, Gustavo >> David >> ----- >> >>> Thank you very much. >>> >>> Best regards, >>> Gustavo >>> >>> [1] http://openjdk.java.net/projects/jdk8u/ >>> [2] http://openjdk.java.net/projects/jdk9/ >>> [3] http://mail.openjdk.java.net/pipermail/jdk10-dev/2017-January/000041.html >>> [4] http://www.oracle.com/technetwork/es/topics/security/alerts-086861.html >>> > From igor.ignatyev at oracle.com Wed Mar 29 02:50:22 2017 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 28 Mar 2017 19:50:22 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: Message-ID: <3754452A-699C-48CD-924A-6DC963B3AFFF@oracle.com> Hi David, thank you for review. please see my answers inlined. > On Mar 28, 2017, at 6:37 PM, David Holmes wrote: > > Hi Igor, > > This cleanup looks good to me! One query: > > test/com/sun/jdi/LineNumberOnBraceTest.java > > The old version seems to have the wrong line numbers - was it failing? it didn't run, because it had @test/nodynamiccopyright/ instead of @test > > Two nits: > - test/com/sun/jdi/FetchLocals.java > - test/com/sun/jdi/LambdaBreakpointTest.java > > Second copyright year should be 2017. sure, I'll update it before push. -- Igor > > Thanks, > David > ----- > > On 29/03/2017 10:42 AM, Igor Ignatyev wrote: >> Hi, >> >> still looking for a reviewer, anyone? >> >> -- Igor >>> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>> 295 lines changed: 176 ins; 15 del; 104 mod; >>> >>> Hi all, >>> >>> could you please review this fix for 8177507? >>> >>> due to their nature, some of jdi tests are line number sensitive. unfortunately different tests indicate that differently, so it's quite easy to overlook that and incidentally break tests, for example by changing module dependency declaration or license modification. this fix unifies the way line number sensitivity is indicated and also improves readability/maintainability of some tests by using constant fields instead of magic numbers. >>> >>> some of line number sensitive tests have been unexpectedly removed from execution because they had @test/nodynamiccopyright/ instead of @test tag. this changeset fixes and returns them to regular execution. >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>> testing: test/com/sun/jdi >>> >>> Thanks, >>> -- Igor >> From mikhailo.seledtsov at oracle.com Wed Mar 29 18:08:50 2017 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Wed, 29 Mar 2017 11:08:50 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: References: Message-ID: <58DBF832.2010902@oracle.com> Hi Igor, Looks good. One style nit: LineNumberOnBraceTarg: All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) for line number variables, but this test uses 'stopLine'. Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. The rest looks good to me (+ David's comments) Thank you, Misha On 3/28/17, 5:42 PM, Igor Ignatyev wrote: > Hi, > > still looking for a reviewer, anyone? > > -- Igor >> On Mar 24, 2017, at 1:56 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>> 295 lines changed: 176 ins; 15 del; 104 mod; >> Hi all, >> >> could you please review this fix for 8177507? >> >> due to their nature, some of jdi tests are line number sensitive. unfortunately different tests indicate that differently, so it's quite easy to overlook that and incidentally break tests, for example by changing module dependency declaration or license modification. this fix unifies the way line number sensitivity is indicated and also improves readability/maintainability of some tests by using constant fields instead of magic numbers. >> >> some of line number sensitive tests have been unexpectedly removed from execution because they had @test/nodynamiccopyright/ instead of @test tag. this changeset fixes and returns them to regular execution. >> >> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >> testing: test/com/sun/jdi >> >> Thanks, >> -- Igor From serguei.spitsyn at oracle.com Wed Mar 29 19:57:35 2017 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 29 Mar 2017 12:57:35 -0700 Subject: RFR(M) : 8177507 : line number sensitive tests for jdi should be unified In-Reply-To: <58DBF832.2010902@oracle.com> References: <58DBF832.2010902@oracle.com> Message-ID: Hi Igor, +1 to the Mikhailo's comment. This one also does not look unified: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00/test/com/sun/jdi/LambdaBreakpointTest.java.udiff.html But I do not have a strong opinion on it and leave it up to you. Thanks, Serguei On 3/29/17 11:08, Mikhailo Seledtsov wrote: > Hi Igor, > > Looks good. One style nit: > > LineNumberOnBraceTarg: > All other Java tests use style of CAP_UNDERSCORE (e.g. STOP_LINE) > for line number variables, but this test uses 'stopLine'. > Consider changing it to STOP_LINE (and STOP_LINE_2) to be uniform. > > The rest looks good to me (+ David's comments) > > > Thank you, > Misha > > > > On 3/28/17, 5:42 PM, Igor Ignatyev wrote: >> Hi, >> >> still looking for a reviewer, anyone? >> >> -- Igor >>> On Mar 24, 2017, at 1:56 PM, Igor >>> Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>>> 295 lines changed: 176 ins; 15 del; 104 mod; >>> Hi all, >>> >>> could you please review this fix for 8177507? >>> >>> due to their nature, some of jdi tests are line number sensitive. >>> unfortunately different tests indicate that differently, so it's >>> quite easy to overlook that and incidentally break tests, for >>> example by changing module dependency declaration or license >>> modification. this fix unifies the way line number sensitivity is >>> indicated and also improves readability/maintainability of some >>> tests by using constant fields instead of magic numbers. >>> >>> some of line number sensitive tests have been unexpectedly removed >>> from execution because they had @test/nodynamiccopyright/ instead of >>> @test tag. this changeset fixes and returns them to regular execution. >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev/8177507/webrev.00 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177507 >>> testing: test/com/sun/jdi >>> >>> Thanks, >>> -- Igor From neugens.limasoftware at gmail.com Wed Mar 22 17:56:44 2017 From: neugens.limasoftware at gmail.com (Mario Torre) Date: Wed, 22 Mar 2017 17:56:44 +0000 Subject: C/C++ IDE support for HotSpot In-Reply-To: <520E90B8-BCA9-415E-A198-E646F543C8F5@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> <520E90B8-BCA9-415E-A198-E646F543C8F5@oracle.com> Message-ID: For what is worth, I agree. Cheers, Mario On Wed, 22 Mar 2017 at 17:37, wrote: > Hi, > > I use the OpenJDK NetBeans project most of the time and am very happy with > it. I still think Mikael's initiative is a very good one since I don't > believe in forcing developers to use some specific tools, but rather think > that we should make the OpenJDK code available to as many developers as > possible regardless of their IDE preference. > > The technology should adapt to humans, humans should not adapt to > technology. > > Thanks, > /Jesper > > > On 22 Mar 2017, at 16:02, Stanislav Smirnov < > stanislav.smirnov at oracle.com> wrote: > > > > Hi Mikael, > > > > why do not you try NetBeans that has openjdk project support out of the > box? > > common/nb_native/nbproject > > > > Best regards, > > Stanislav Smirnov > > > > > >> On 22 Mar 2017, at 17:21, Mikael Gerdin > wrote: > >> > >> Hi all, > >> > >> I've finally grown tired of manually setting up a hand crafted Eclipse > CDT configuration for the JVM sources and decided to share my progress > towards improving the overall situation for JVM developers. > >> > >> To achieve better IDE support without having to add project generators > for all different kinds of past or future IDEs I've decided to try to > leverage CMake to do project generation. > >> The idea is to have the JDK build system generate a CMakeLists.txt > describing all the include paths and definitions required by an IDE to > interpret the sources correctly. > >> > >> Several modern IDEs natively support CMake but we can also rely on the > fact that the CMake build system has the ability to generate projects for a > number of different IDEs. For information about which generators CMake > supports see > >> https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html > >> for your CMake version. > >> > >> To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in > the jdk10/sandbox forest: > >> http://hg.openjdk.java.net/jdk10/sandbox/branches > >> So far I've only made changes in the toplevel and hotspot repositories. > >> I've written a short readme in the repo: > >> > http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html > >> > >> It would be great if people tried this out to see if it works on their > setup but I don't expect it to work on Windows without changing the > makefile to do path conversion. > >> If we can make this work properly then perhaps we can get rid of the > Visual Studio generator and rely on CMake to generate VS projects. > >> > >> It would also be great if someone from build-dev could give me some > hints about how to do the file writing and "vardeps" thing properly. > >> > >> Thanks > >> /Mikael > > > > From ch-hagedorn at hispeed.ch Thu Mar 30 00:13:34 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Thu, 30 Mar 2017 02:13:34 +0200 Subject: Hook for all newly created objects Message-ID: <000001d2a8ea$799ece70$6cdc6b50$@hispeed.ch> I have added a private field to the oopDesc class inside share/vm/oops/oop.hpp. Now I wanted to initialize it properly. How can I do that? I tried to do it inside the share/vm/runtime/InterpreterRuntime.cpp _new, newarray, anewarray and multianewarray. But then I miss some objects. So I looked further and found share/vm/gc_interface/collectedHeap.inline.hpp and additionally tried to initialize it inside post_allocation_notify. But I think I still miss some objects. What can I do to catch all new objects and initialize my field only once? - Christian From rkennke at redhat.com Thu Mar 30 06:37:46 2017 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 30 Mar 2017 08:37:46 +0200 Subject: Hook for all newly created objects In-Reply-To: <000001d2a8ea$799ece70$6cdc6b50$@hispeed.ch> References: <000001d2a8ea$799ece70$6cdc6b50$@hispeed.ch> Message-ID: Am 30. M?rz 2017 02:13:34 MESZ schrieb Christian Hagedorn : >I have added a private field to the oopDesc class inside >share/vm/oops/oop.hpp. Now I wanted to initialize it properly. How can >I do >that? > >I tried to do it inside the share/vm/runtime/InterpreterRuntime.cpp >_new, >newarray, anewarray and multianewarray. But then I miss some objects. > >So I looked further and found >share/vm/gc_interface/collectedHeap.inline.hpp >and additionally tried to initialize it inside post_allocation_notify. >But I >think I still miss some objects. > >What can I do to catch all new objects and initialize my field only >once? You need to implementiert the same hook in the interpreter, c1 and c2. There are fast inlined allocation routines ... Alternatively, disable TLABs -XX:-UseTLAB, but this will make allocations *much* slower. Roman > > > >- Christian -- Sent from my FairPhone From ch-hagedorn at hispeed.ch Thu Mar 30 15:30:26 2017 From: ch-hagedorn at hispeed.ch (Christian Hagedorn) Date: Thu, 30 Mar 2017 17:30:26 +0200 Subject: Hook for all newly created objects In-Reply-To: References: <000001d2a8ea$799ece70$6cdc6b50$@hispeed.ch> Message-ID: <000301d2a96a$8f497830$addc6890$@hispeed.ch> Alright, thank you. -- Christian Am 30. M?rz 2017 02:13:34 MESZ schrieb Christian Hagedorn : >I have added a private field to the oopDesc class inside >share/vm/oops/oop.hpp. Now I wanted to initialize it properly. How can >I do that? > >I tried to do it inside the share/vm/runtime/InterpreterRuntime.cpp >_new, >newarray, anewarray and multianewarray. But then I miss some objects. > >So I looked further and found >share/vm/gc_interface/collectedHeap.inline.hpp >and additionally tried to initialize it inside post_allocation_notify. >But I >think I still miss some objects. > >What can I do to catch all new objects and initialize my field only >once? You need to implementiert the same hook in the interpreter, c1 and c2. There are fast inlined allocation routines ... Alternatively, disable TLABs -XX:-UseTLAB, but this will make allocations *much* slower. Roman > > > >- Christian -- Sent from my FairPhone From aph at redhat.com Thu Mar 30 17:21:00 2017 From: aph at redhat.com (Andrew Haley) Date: Thu, 30 Mar 2017 18:21:00 +0100 Subject: JEP 270 concerns In-Reply-To: References: <9a9afaf8-c20e-2853-5d55-c56bf8894836@redhat.com> <3c814e52-878d-c1af-bf24-ec71348d884d@oracle.com> <676313e2-f83f-f06e-0cad-26cdd449c709@redhat.com> <33d05803-e972-9059-5acf-1d0fc150c20c@oracle.com> <6a0a9f3e-8c9f-0c95-b920-9bf09934768d@redhat.com> Message-ID: <19d07ed4-067b-72f1-c809-ca77847f3bec@redhat.com> On 13/01/17 14:11, Frederic Parain wrote: > It makes perfect sense. > > I've created bug JDK-8172791 to track these issues. Hi, It looks like this didn't get done. That's a shame, given that it wasn't difficult to fix the worst problems and I would have been quite happy to do it. I'd already written the code to walk up the stack finding scopes. Andrew. From cthalinger at twitter.com Thu Mar 30 18:22:31 2017 From: cthalinger at twitter.com (Christian Thalinger) Date: Thu, 30 Mar 2017 08:22:31 -1000 Subject: C/C++ IDE support for HotSpot In-Reply-To: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> References: <28993a4d-5213-7cda-57bd-faacb6005d8d@oracle.com> Message-ID: There is already support to generate IDE config files in 9: $ mx ideinit (this is part of JVMCI and came in via JEP 243) > On Mar 22, 2017, at 4:21 AM, Mikael Gerdin wrote: > > Hi all, > > I've finally grown tired of manually setting up a hand crafted Eclipse CDT configuration for the JVM sources and decided to share my progress towards improving the overall situation for JVM developers. > > To achieve better IDE support without having to add project generators for all different kinds of past or future IDEs I've decided to try to leverage CMake to do project generation. > The idea is to have the JDK build system generate a CMakeLists.txt describing all the include paths and definitions required by an IDE to interpret the sources correctly. > > Several modern IDEs natively support CMake but we can also rely on the fact that the CMake build system has the ability to generate projects for a number of different IDEs. For information about which generators CMake supports see > https://cmake.org/cmake/help/v3.5/manual/cmake-generators.7.html > for your CMake version. > > To try this out check out (heh) my branch "JDK-8177329-cmake-branch" in the jdk10/sandbox forest: > http://hg.openjdk.java.net/jdk10/sandbox/branches > So far I've only made changes in the toplevel and hotspot repositories. > I've written a short readme in the repo: > http://hg.openjdk.java.net/jdk10/sandbox/raw-file/JDK-8177329-cmake-branch/README-cmake.html > > It would be great if people tried this out to see if it works on their setup but I don't expect it to work on Windows without changing the makefile to do path conversion. > If we can make this work properly then perhaps we can get rid of the Visual Studio generator and rely on CMake to generate VS projects. > > It would also be great if someone from build-dev could give me some hints about how to do the file writing and "vardeps" thing properly. > > Thanks > /Mikael From dean.long at oracle.com Thu Mar 30 18:24:34 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 30 Mar 2017 11:24:34 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: <7b319f81-2a48-1533-efe1-6fd86a3d7244@oracle.com> References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> <6a3d62b5-fbea-e982-d7d5-087f9badb028@oracle.com> <3765035a-ebd6-c7ed-4c06-6d9e269bc87c@oracle.com> <7ade4c8d-891c-618b-0fa3-4d641da236ba@oracle.com> <7b319f81-2a48-1533-efe1-6fd86a3d7244@oracle.com> Message-ID: I would like to go with the webrev.2 version, with asserts removed. All the reviewers have said they are OK with that version, and it allows more complete testing than the minimal version. dl On 3/23/17 12:03 PM, dean.long at oracle.com wrote: > > On 3/23/17 11:25 AM, dean.long at oracle.com wrote: > >> On 3/22/17 1:49 PM, Vladimir Ivanov wrote: >> >>>> Also, it looks like the changes I made to ASB.appendChars(char[] s, >>>> int >>>> off, int end) are not needed. >>> >>> Agree. >>> >>>>> Vladimir, don't you need to replace checkIndex with checkOffset in >>>>> indexOf and lastIndexOf, so that we allow count == length? >>> >>> Yes, my bad. Good catch. Updated webrev in place. >>> >>> FTR I haven't done any extensive testing of the minimized fix. >>> >>> If we agree to proceed with it, the regression test should be >>> updated as well. I think the viable solution would be to construct >>> broken SBs (using reflection) and invoke affected methods on them. >>> >> >> We can construct broken SBs using the Helper class that gets patched >> into java.lang. I'll work on that. >> > > Nevermind. I forgot that some problems can only happen when the SB is > changed half-way through the method. For example, > in append(), we can't force an overflow unless we change the SB after > ensureCapacityInternal() is called. I could do something like: > > 760 public AbstractStringBuilder append(int i) { > 761 int count = this.count; > 762 int spaceNeeded = count + Integer.stringSize(i); > 763 ensureCapacityInternal(spaceNeeded); > 764 if (isLatin1()) { > >>>>>> Helper.fuzzValue(this); > 765 Integer.getChars(i, spaceNeeded, value); > 766 } else { > 767 byte[] val = this.value; > >>>>>> Helper.fuzzValue(this); > 768 checkBoundsBeginEnd(count, spaceNeeded, val.length >> 1); > 769 Integer.getCharsUTF16(i, spaceNeeded, val); > 770 } > 771 this.count = spaceNeeded; > 772 return this; > 773 } > > where the default Helper.fuzzValue() is an empty method, but the test > would patch in its own version of Helper that changes the ASB field > values. I like this less than refactoring the checks to StringUTF16. > > dl > >> dl >> >>> Best regards, >>> Vladimir Ivanov >>> >>>>> On 3/22/17 8:35 AM, Vladimir Ivanov wrote: >>>>>>>>> So are we convinced that the proposed changes will never lead >>>>>>>>> to a >>>>>>>>> crash due to a missing or incorrect bounds check, due to a racy >>>>>>>>> use of >>>>>>>>> an unsynchronized ASB instance e.g. StringBuilder? >>>>>>>> >>>>>>>> If only we had a static analysis tool that could tell us if the >>>>>>>> code is >>>>>>>> safe. Because we don't, in my initial changeset, we always take a >>>>>>>> snapshot of the ASB fields by passing those field values to >>>>>>>> StringUTF16 >>>>>>>> before doing checks on them. And I wrote a test to make sure that >>>>>>>> those >>>>>>>> StringUTF16 interfaces are catching all the underflows and >>>>>>>> overflows I >>>>>>>> could imagine, and I added verification code to detect when a >>>>>>>> check >>>>>>>> was >>>>>>>> missed. >>>>>>>> >>>>>>>> However, all the reviewers have requested to minimize the >>>>>>>> amount of >>>>>>>> changes. In Vladimir's version, if there is a missing check >>>>>>>> somewhere, >>>>>>>> then yes it could lead to a crash. >>>>>> >>>>>> I'd like to point out that asserts and verification code are >>>>>> disabled >>>>>> by default. They are invaluable during problem diagnosis, but don't >>>>>> help at all from defence-in-depth perspective. >>>>>> >>>>>> But I agree that it's easier to reason about and test the initial >>>>>> version of the fix. >>>>>> >>>>>>> I wonder if the reviewers have fully realized the potential impact >>>>>>> here? >>>>>>> This has exposed a flaw in the way intrinsics are used from core >>>>>>> classes. >>>>>> >>>>>> FTR here are the checks I omitted in the minimized version (modulo >>>>>> separation of indexOf/lastIndexOf for trusted/non-trusted callers): >>>>>> >>>>>> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ >>>>>> >>>>>> Other than that, the difference is mainly about undoing refactorings >>>>>> and removing verification logic (asserts + checks in the JVM). >>>>>> >>>>>> There are still unsafe accesses which are considered safe in both >>>>>> versions (see StringUTF16.Trusted usages in the initial version >>>>>> [1]). >>>>>> >>>>>> We used to provide safe wrappers for unsafe intrinsics which >>>>>> makes it >>>>>> much easier to reason about code correctness. I'd like to see >>>>>> compact >>>>>> string code refactored that way and IMO the initial version by Dean >>>>>> is a big step in the right direction. >>>>>> >>>>>> I still prefer to see a point fix in 9 and major refactoring >>>>>> happening in 10, but I'll leave the decision on how to proceed with >>>>>> the fix to core-libs folks. After finishing the exercise minimizing >>>>>> the fix, I'm much more comfortable with the initial fix [1] (though >>>>>> there are changes I consider excessive). >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>> [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 >>>>>> >>>>>>>>>>> Some clarifications: >>>>>>>>>>> >>>>>>>>>>> ============ >>>>>>>>>>> src/java.base/share/classes/java/lang/String.java: >>>>>>>>>>> >>>>>>>>>>> The bounds check is needed only in String.nonSyncContentEquals >>>>>>>>>>> when it >>>>>>>>>>> extracts info from AbstractStringBuilder. I don't see how >>>>>>>>>>> out of >>>>>>>>>>> bounds access can happen in String.contentEquals: >>>>>>>>>>> if (n != length()) { >>>>>>>>>>> return false; >>>>>>>>>>> } >>>>>>>>>>> ... >>>>>>>>>>> for (int i = 0; i < n; i++) { >>>>>>>>>>> if (StringUTF16.getChar(val, i) != >>>>>>>>>>> cs.charAt(i)) { >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK. >>>>>>>>>> >>>>>>>>>>> ============ >>>>>>>>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>>>>>>>> >>>>>>>>>>> I think bounds checks in StringConcatHelper.prepend() are >>>>>>>>>>> skipped >>>>>>>>>>> intentionally, since java.lang.invoke.StringConcatFactory >>>>>>>>>>> constructs >>>>>>>>>>> method handle chains which already contain bounds checks: array >>>>>>>>>>> length >>>>>>>>>>> is precomputed based on argument values and all accesses are >>>>>>>>>>> guaranteed to be in bounds. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This is calling the trusted version of getChars() with no bounds >>>>>>>>>> checks. It was a little more obvious when I had the Trusted >>>>>>>>>> inner >>>>>>>>>> class. >>>>>>>>>> >>>>>>>>>>> ============ >>>>>>>>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>> >>>>>>>>>>> + static void putChar(byte[] val, int index, int c) { >>>>>>>>>>> + assert index >= 0 && index < length(val) : "Trusted >>>>>>>>>>> caller >>>>>>>>>>> missed bounds check"; >>>>>>>>>>> >>>>>>>>>>> Unfortunately, asserts can affect inlining decisions (since >>>>>>>>>>> they >>>>>>>>>>> increase bytecode size). In order to minimize possible >>>>>>>>>>> performance >>>>>>>>>>> impact, I suggest to remove them from the fix targeting 9. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Sure. >>>>>>>>>> >>>>>>>>>>> ============ >>>>>>>>>>> private static int indexOfSupplementary(byte[] value, int >>>>>>>>>>> ch, int >>>>>>>>>>> fromIndex, int max) { >>>>>>>>>>> if (Character.isValidCodePoint(ch)) { >>>>>>>>>>> final char hi = Character.highSurrogate(ch); >>>>>>>>>>> final char lo = Character.lowSurrogate(ch); >>>>>>>>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>>>>>>>> >>>>>>>>>>> The check is redundant here. fromIndex & max are always >>>>>>>>>>> inbounds by >>>>>>>>>>> construction: >>>>>>>>>>> >>>>>>>>>>> public static int indexOf(byte[] value, int ch, int >>>>>>>>>>> fromIndex) { >>>>>>>>>>> int max = value.length >> 1; >>>>>>>>>>> if (fromIndex < 0) { >>>>>>>>>>> fromIndex = 0; >>>>>>>>>>> } else if (fromIndex >= max) { >>>>>>>>>>> // Note: fromIndex might be near -1>>>1. >>>>>>>>>>> return -1; >>>>>>>>>>> } >>>>>>>>>>> ... >>>>>>>>>>> return indexOfSupplementary(value, ch, >>>>>>>>>>> fromIndex, max); >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK. >>>>>>>>>> >>>>>>>>>>> ============ >>>>>>>>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>>>>>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range >>>>>>>>>>> check on >>>>>>>>>>> ABS.value & ABS.count. After that, all accesses should be >>>>>>>>>>> inbounds by >>>>>>>>>>> construction (in String.indexOf/lastIndexOf): >>>>>>>>>>> >>>>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>> static int lastIndexOf(byte[] src, byte srcCoder, int >>>>>>>>>>> srcCount, >>>>>>>>>>> String tgtStr, int fromIndex) { >>>>>>>>>>> >>>>>>>>>>> int rightIndex = srcCount - tgtCount; >>>>>>>>>>> if (fromIndex > rightIndex) { >>>>>>>>>>> fromIndex = rightIndex; >>>>>>>>>>> } >>>>>>>>>>> if (fromIndex < 0) { >>>>>>>>>>> return -1; >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>>>>>>>> byte[] tgt, int tgtCount, int >>>>>>>>>>> fromIndex) { >>>>>>>>>>> int min = tgtCount - 1; >>>>>>>>>>> int i = min + fromIndex; >>>>>>>>>>> int strLastIndex = tgtCount - 1; >>>>>>>>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>>>>>>>> >>>>>>>>>>> startSearchForLastChar: >>>>>>>>>>> while (true) { >>>>>>>>>>> while (i >= min && getChar(src, i) != >>>>>>>>>>> strLastChar) { >>>>>>>>>>> >>>>>>>>>>> There are 2 places: >>>>>>>>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - >>>>>>>>>>> inbound >>>>>>>>>>> >>>>>>>>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>>>>>>>> min = tgtCount - 1 >>>>>>>>>>> rightIndex = srcCount - tgtCount >>>>>>>>>>> fromIndex <= rightIndex >>>>>>>>>>> >>>>>>>>>>> 0 <= min + fromIndex <= min + rightIndex == >>>>>>>>>>> (tgtCount >>>>>>>>>>> - 1) >>>>>>>>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>>>>>>>> >>>>>>>>>>> Hence, should be covered by the check on count & value: >>>>>>>>>>> public int lastIndexOf(String str, int fromIndex) { >>>>>>>>>>> + byte[] value = this.value; >>>>>>>>>>> + int count = this.count; >>>>>>>>>>> + byte coder = this.coder; >>>>>>>>>>> + checkIndex(count, value.length >> coder); >>>>>>>>>>> return String.lastIndexOf(value, coder, count, str, >>>>>>>>>>> fromIndex); >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, I will go with your version if it's OK with Sherman. >>>>>>>>>> >>>>>>>>>> dl >>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Vladimir Ivanov >>>>>>>>>>> >>>>>>>>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>>> I have the same concern. Can we fix the immediate >>>>>>>>>>>>>>> problem in >>>>>>>>>>>>>>> 9 and >>>>>>>>>>>>>>> integrate verification logic in 10? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> OK, Tobias is suggesting having verification logic only >>>>>>>>>>>>>> inside the >>>>>>>>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>>>>>>>> >>>>>>>>>>>>> Yes and put them back in 10. >>>>>>>>>>>>> >>>>>>>>>>>>>> I'm OK with removing all the verification, but that won't >>>>>>>>>>>>>> reduce >>>>>>>>>>>>>> the >>>>>>>>>>>>>> library changes much. I could undo the renaming to >>>>>>>>>>>>>> Trusted.getChar, but >>>>>>>>>>>>>> we would still have the bounds checks moved into >>>>>>>>>>>>>> StringUTF16. >>>>>>>>>>>>> >>>>>>>>>>>>> I suggest to go with a point fix for 9: just add missing >>>>>>>>>>>>> range >>>>>>>>>>>>> checks. >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>> >>>> >> > From dean.long at oracle.com Thu Mar 30 19:27:40 2017 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 30 Mar 2017 12:27:40 -0700 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> <6a3d62b5-fbea-e982-d7d5-087f9badb028@oracle.com> <3765035a-ebd6-c7ed-4c06-6d9e269bc87c@oracle.com> <7ade4c8d-891c-618b-0fa3-4d641da236ba@oracle.com> <7b319f81-2a48-1533-efe1-6fd86a3d7244@oracle.com> Message-ID: Actually, the assert inside getChar/putChar shouldn't affect inlining, because C1 and C2 will use the intrinsic instead, so I'd like to use webrev.2 as is, so I don't have to re-run all the tests. dl On 3/30/17 11:24 AM, dean.long at oracle.com wrote: > > I would like to go with the webrev.2 version, with asserts removed. > All the reviewers have said they are OK with that version, and it > allows more complete testing than the minimal version. > > dl > > > On 3/23/17 12:03 PM, dean.long at oracle.com wrote: >> >> On 3/23/17 11:25 AM, dean.long at oracle.com wrote: >> >>> On 3/22/17 1:49 PM, Vladimir Ivanov wrote: >>> >>>>> Also, it looks like the changes I made to ASB.appendChars(char[] >>>>> s, int >>>>> off, int end) are not needed. >>>> >>>> Agree. >>>> >>>>>> Vladimir, don't you need to replace checkIndex with checkOffset in >>>>>> indexOf and lastIndexOf, so that we allow count == length? >>>> >>>> Yes, my bad. Good catch. Updated webrev in place. >>>> >>>> FTR I haven't done any extensive testing of the minimized fix. >>>> >>>> If we agree to proceed with it, the regression test should be >>>> updated as well. I think the viable solution would be to construct >>>> broken SBs (using reflection) and invoke affected methods on them. >>>> >>> >>> We can construct broken SBs using the Helper class that gets patched >>> into java.lang. I'll work on that. >>> >> >> Nevermind. I forgot that some problems can only happen when the SB >> is changed half-way through the method. For example, >> in append(), we can't force an overflow unless we change the SB after >> ensureCapacityInternal() is called. I could do something like: >> >> 760 public AbstractStringBuilder append(int i) { >> 761 int count = this.count; >> 762 int spaceNeeded = count + Integer.stringSize(i); >> 763 ensureCapacityInternal(spaceNeeded); >> 764 if (isLatin1()) { >> >>>>>> Helper.fuzzValue(this); >> 765 Integer.getChars(i, spaceNeeded, value); >> 766 } else { >> 767 byte[] val = this.value; >> >>>>>> Helper.fuzzValue(this); >> 768 checkBoundsBeginEnd(count, spaceNeeded, val.length >> 1); >> 769 Integer.getCharsUTF16(i, spaceNeeded, val); >> 770 } >> 771 this.count = spaceNeeded; >> 772 return this; >> 773 } >> >> where the default Helper.fuzzValue() is an empty method, but the test >> would patch in its own version of Helper that changes the ASB field >> values. I like this less than refactoring the checks to StringUTF16. >> >> dl >> >>> dl >>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>>>> On 3/22/17 8:35 AM, Vladimir Ivanov wrote: >>>>>>>>>> So are we convinced that the proposed changes will never lead >>>>>>>>>> to a >>>>>>>>>> crash due to a missing or incorrect bounds check, due to a racy >>>>>>>>>> use of >>>>>>>>>> an unsynchronized ASB instance e.g. StringBuilder? >>>>>>>>> >>>>>>>>> If only we had a static analysis tool that could tell us if the >>>>>>>>> code is >>>>>>>>> safe. Because we don't, in my initial changeset, we always >>>>>>>>> take a >>>>>>>>> snapshot of the ASB fields by passing those field values to >>>>>>>>> StringUTF16 >>>>>>>>> before doing checks on them. And I wrote a test to make sure >>>>>>>>> that >>>>>>>>> those >>>>>>>>> StringUTF16 interfaces are catching all the underflows and >>>>>>>>> overflows I >>>>>>>>> could imagine, and I added verification code to detect when a >>>>>>>>> check >>>>>>>>> was >>>>>>>>> missed. >>>>>>>>> >>>>>>>>> However, all the reviewers have requested to minimize the >>>>>>>>> amount of >>>>>>>>> changes. In Vladimir's version, if there is a missing check >>>>>>>>> somewhere, >>>>>>>>> then yes it could lead to a crash. >>>>>>> >>>>>>> I'd like to point out that asserts and verification code are >>>>>>> disabled >>>>>>> by default. They are invaluable during problem diagnosis, but don't >>>>>>> help at all from defence-in-depth perspective. >>>>>>> >>>>>>> But I agree that it's easier to reason about and test the initial >>>>>>> version of the fix. >>>>>>> >>>>>>>> I wonder if the reviewers have fully realized the potential impact >>>>>>>> here? >>>>>>>> This has exposed a flaw in the way intrinsics are used from core >>>>>>>> classes. >>>>>>> >>>>>>> FTR here are the checks I omitted in the minimized version (modulo >>>>>>> separation of indexOf/lastIndexOf for trusted/non-trusted callers): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ >>>>>>> >>>>>>> >>>>>>> Other than that, the difference is mainly about undoing >>>>>>> refactorings >>>>>>> and removing verification logic (asserts + checks in the JVM). >>>>>>> >>>>>>> There are still unsafe accesses which are considered safe in both >>>>>>> versions (see StringUTF16.Trusted usages in the initial version >>>>>>> [1]). >>>>>>> >>>>>>> We used to provide safe wrappers for unsafe intrinsics which >>>>>>> makes it >>>>>>> much easier to reason about code correctness. I'd like to see >>>>>>> compact >>>>>>> string code refactored that way and IMO the initial version by Dean >>>>>>> is a big step in the right direction. >>>>>>> >>>>>>> I still prefer to see a point fix in 9 and major refactoring >>>>>>> happening in 10, but I'll leave the decision on how to proceed with >>>>>>> the fix to core-libs folks. After finishing the exercise minimizing >>>>>>> the fix, I'm much more comfortable with the initial fix [1] (though >>>>>>> there are changes I consider excessive). >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>> [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 >>>>>>> >>>>>>>>>>>> Some clarifications: >>>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> src/java.base/share/classes/java/lang/String.java: >>>>>>>>>>>> >>>>>>>>>>>> The bounds check is needed only in String.nonSyncContentEquals >>>>>>>>>>>> when it >>>>>>>>>>>> extracts info from AbstractStringBuilder. I don't see how >>>>>>>>>>>> out of >>>>>>>>>>>> bounds access can happen in String.contentEquals: >>>>>>>>>>>> if (n != length()) { >>>>>>>>>>>> return false; >>>>>>>>>>>> } >>>>>>>>>>>> ... >>>>>>>>>>>> for (int i = 0; i < n; i++) { >>>>>>>>>>>> if (StringUTF16.getChar(val, i) != >>>>>>>>>>>> cs.charAt(i)) { >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK. >>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>>>>>>>>> >>>>>>>>>>>> I think bounds checks in StringConcatHelper.prepend() are >>>>>>>>>>>> skipped >>>>>>>>>>>> intentionally, since java.lang.invoke.StringConcatFactory >>>>>>>>>>>> constructs >>>>>>>>>>>> method handle chains which already contain bounds checks: >>>>>>>>>>>> array >>>>>>>>>>>> length >>>>>>>>>>>> is precomputed based on argument values and all accesses are >>>>>>>>>>>> guaranteed to be in bounds. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This is calling the trusted version of getChars() with no >>>>>>>>>>> bounds >>>>>>>>>>> checks. It was a little more obvious when I had the Trusted >>>>>>>>>>> inner >>>>>>>>>>> class. >>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>>> >>>>>>>>>>>> + static void putChar(byte[] val, int index, int c) { >>>>>>>>>>>> + assert index >= 0 && index < length(val) : >>>>>>>>>>>> "Trusted caller >>>>>>>>>>>> missed bounds check"; >>>>>>>>>>>> >>>>>>>>>>>> Unfortunately, asserts can affect inlining decisions (since >>>>>>>>>>>> they >>>>>>>>>>>> increase bytecode size). In order to minimize possible >>>>>>>>>>>> performance >>>>>>>>>>>> impact, I suggest to remove them from the fix targeting 9. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Sure. >>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> private static int indexOfSupplementary(byte[] value, int >>>>>>>>>>>> ch, int >>>>>>>>>>>> fromIndex, int max) { >>>>>>>>>>>> if (Character.isValidCodePoint(ch)) { >>>>>>>>>>>> final char hi = Character.highSurrogate(ch); >>>>>>>>>>>> final char lo = Character.lowSurrogate(ch); >>>>>>>>>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>>>>>>>>> >>>>>>>>>>>> The check is redundant here. fromIndex & max are always >>>>>>>>>>>> inbounds by >>>>>>>>>>>> construction: >>>>>>>>>>>> >>>>>>>>>>>> public static int indexOf(byte[] value, int ch, int >>>>>>>>>>>> fromIndex) { >>>>>>>>>>>> int max = value.length >> 1; >>>>>>>>>>>> if (fromIndex < 0) { >>>>>>>>>>>> fromIndex = 0; >>>>>>>>>>>> } else if (fromIndex >= max) { >>>>>>>>>>>> // Note: fromIndex might be near -1>>>1. >>>>>>>>>>>> return -1; >>>>>>>>>>>> } >>>>>>>>>>>> ... >>>>>>>>>>>> return indexOfSupplementary(value, ch, >>>>>>>>>>>> fromIndex, max); >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK. >>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>>>>>>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range >>>>>>>>>>>> check on >>>>>>>>>>>> ABS.value & ABS.count. After that, all accesses should be >>>>>>>>>>>> inbounds by >>>>>>>>>>>> construction (in String.indexOf/lastIndexOf): >>>>>>>>>>>> >>>>>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>>> static int lastIndexOf(byte[] src, byte srcCoder, int >>>>>>>>>>>> srcCount, >>>>>>>>>>>> String tgtStr, int fromIndex) { >>>>>>>>>>>> >>>>>>>>>>>> int rightIndex = srcCount - tgtCount; >>>>>>>>>>>> if (fromIndex > rightIndex) { >>>>>>>>>>>> fromIndex = rightIndex; >>>>>>>>>>>> } >>>>>>>>>>>> if (fromIndex < 0) { >>>>>>>>>>>> return -1; >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>>>>>>>>> byte[] tgt, int tgtCount, >>>>>>>>>>>> int >>>>>>>>>>>> fromIndex) { >>>>>>>>>>>> int min = tgtCount - 1; >>>>>>>>>>>> int i = min + fromIndex; >>>>>>>>>>>> int strLastIndex = tgtCount - 1; >>>>>>>>>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>>>>>>>>> >>>>>>>>>>>> startSearchForLastChar: >>>>>>>>>>>> while (true) { >>>>>>>>>>>> while (i >= min && getChar(src, i) != >>>>>>>>>>>> strLastChar) { >>>>>>>>>>>> >>>>>>>>>>>> There are 2 places: >>>>>>>>>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - >>>>>>>>>>>> inbound >>>>>>>>>>>> >>>>>>>>>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>>>>>>>>> min = tgtCount - 1 >>>>>>>>>>>> rightIndex = srcCount - tgtCount >>>>>>>>>>>> fromIndex <= rightIndex >>>>>>>>>>>> >>>>>>>>>>>> 0 <= min + fromIndex <= min + rightIndex == >>>>>>>>>>>> (tgtCount >>>>>>>>>>>> - 1) >>>>>>>>>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>>>>>>>>> >>>>>>>>>>>> Hence, should be covered by the check on count & value: >>>>>>>>>>>> public int lastIndexOf(String str, int fromIndex) { >>>>>>>>>>>> + byte[] value = this.value; >>>>>>>>>>>> + int count = this.count; >>>>>>>>>>>> + byte coder = this.coder; >>>>>>>>>>>> + checkIndex(count, value.length >> coder); >>>>>>>>>>>> return String.lastIndexOf(value, coder, count, str, >>>>>>>>>>>> fromIndex); >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, I will go with your version if it's OK with Sherman. >>>>>>>>>>> >>>>>>>>>>> dl >>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Vladimir Ivanov >>>>>>>>>>>> >>>>>>>>>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I have the same concern. Can we fix the immediate >>>>>>>>>>>>>>>> problem in >>>>>>>>>>>>>>>> 9 and >>>>>>>>>>>>>>>> integrate verification logic in 10? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> OK, Tobias is suggesting having verification logic only >>>>>>>>>>>>>>> inside the >>>>>>>>>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes and put them back in 10. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm OK with removing all the verification, but that >>>>>>>>>>>>>>> won't reduce >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> library changes much. I could undo the renaming to >>>>>>>>>>>>>>> Trusted.getChar, but >>>>>>>>>>>>>>> we would still have the bounds checks moved into >>>>>>>>>>>>>>> StringUTF16. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I suggest to go with a point fix for 9: just add missing >>>>>>>>>>>>>> range >>>>>>>>>>>>>> checks. >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>> >>>>> >>> >> > From tobias.hartmann at oracle.com Fri Mar 31 06:49:05 2017 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 31 Mar 2017 08:49:05 +0200 Subject: [9] RFR(L) 8158168: SIGSEGV: CollectedHeap::fill_with_objects(HeapWord*, unsigned long, bool)+0xa8 In-Reply-To: References: <50408e3b-ed4c-9d7c-d708-abf82a4bd51f@oracle.com> <97816c11-612b-32f1-b83f-9bbf381bafd0@oracle.com> <97612541-fb38-e4ef-bf0d-ec50c9cfdc1a@oracle.com> <3553280e-6955-a574-8154-19cb52409300@oracle.com> <4c5752dc-5f8c-a217-770a-175a168711ed@oracle.com> <8b8aa012-8890-8cbc-29bf-0382a6369fc2@oracle.com> <65c16119-c9a4-f94a-a7d4-f9e22cc61cb4@oracle.com> <6a3d62b5-fbea-e982-d7d5-087f9badb028@oracle.com> <3765035a-ebd6-c7ed-4c06-6d9e269bc87c@oracle.com> <7ade4c8d-891c-618b-0fa3-4d641da236ba@oracle.com> <7b319f81-2a48-1533-efe1-6fd86a3d7244@oracle.com> Message-ID: On 30.03.2017 20:24, dean.long at oracle.com wrote: > I would like to go with the webrev.2 version, with asserts removed. All the reviewers have said they are OK with that version, and it allows more complete testing than the minimal version. Okay, I'm fine with that! Best regards, Tobias > On 3/23/17 12:03 PM, dean.long at oracle.com wrote: >> >> On 3/23/17 11:25 AM, dean.long at oracle.com wrote: >> >>> On 3/22/17 1:49 PM, Vladimir Ivanov wrote: >>> >>>>> Also, it looks like the changes I made to ASB.appendChars(char[] s, int >>>>> off, int end) are not needed. >>>> >>>> Agree. >>>> >>>>>> Vladimir, don't you need to replace checkIndex with checkOffset in >>>>>> indexOf and lastIndexOf, so that we allow count == length? >>>> >>>> Yes, my bad. Good catch. Updated webrev in place. >>>> >>>> FTR I haven't done any extensive testing of the minimized fix. >>>> >>>> If we agree to proceed with it, the regression test should be updated as well. I think the viable solution would be to construct broken SBs (using reflection) and invoke affected methods on them. >>>> >>> >>> We can construct broken SBs using the Helper class that gets patched into java.lang. I'll work on that. >>> >> >> Nevermind. I forgot that some problems can only happen when the SB is changed half-way through the method. For example, >> in append(), we can't force an overflow unless we change the SB after ensureCapacityInternal() is called. I could do something like: >> >> 760 public AbstractStringBuilder append(int i) { >> 761 int count = this.count; >> 762 int spaceNeeded = count + Integer.stringSize(i); >> 763 ensureCapacityInternal(spaceNeeded); >> 764 if (isLatin1()) { >> >>>>>> Helper.fuzzValue(this); >> 765 Integer.getChars(i, spaceNeeded, value); >> 766 } else { >> 767 byte[] val = this.value; >> >>>>>> Helper.fuzzValue(this); >> 768 checkBoundsBeginEnd(count, spaceNeeded, val.length >> 1); >> 769 Integer.getCharsUTF16(i, spaceNeeded, val); >> 770 } >> 771 this.count = spaceNeeded; >> 772 return this; >> 773 } >> >> where the default Helper.fuzzValue() is an empty method, but the test would patch in its own version of Helper that changes the ASB field values. I like this less than refactoring the checks to StringUTF16. >> >> dl >> >>> dl >>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>>>> On 3/22/17 8:35 AM, Vladimir Ivanov wrote: >>>>>>>>>> So are we convinced that the proposed changes will never lead to a >>>>>>>>>> crash due to a missing or incorrect bounds check, due to a racy >>>>>>>>>> use of >>>>>>>>>> an unsynchronized ASB instance e.g. StringBuilder? >>>>>>>>> >>>>>>>>> If only we had a static analysis tool that could tell us if the >>>>>>>>> code is >>>>>>>>> safe. Because we don't, in my initial changeset, we always take a >>>>>>>>> snapshot of the ASB fields by passing those field values to >>>>>>>>> StringUTF16 >>>>>>>>> before doing checks on them. And I wrote a test to make sure that >>>>>>>>> those >>>>>>>>> StringUTF16 interfaces are catching all the underflows and overflows I >>>>>>>>> could imagine, and I added verification code to detect when a check >>>>>>>>> was >>>>>>>>> missed. >>>>>>>>> >>>>>>>>> However, all the reviewers have requested to minimize the amount of >>>>>>>>> changes. In Vladimir's version, if there is a missing check >>>>>>>>> somewhere, >>>>>>>>> then yes it could lead to a crash. >>>>>>> >>>>>>> I'd like to point out that asserts and verification code are disabled >>>>>>> by default. They are invaluable during problem diagnosis, but don't >>>>>>> help at all from defence-in-depth perspective. >>>>>>> >>>>>>> But I agree that it's easier to reason about and test the initial >>>>>>> version of the fix. >>>>>>> >>>>>>>> I wonder if the reviewers have fully realized the potential impact >>>>>>>> here? >>>>>>>> This has exposed a flaw in the way intrinsics are used from core >>>>>>>> classes. >>>>>>> >>>>>>> FTR here are the checks I omitted in the minimized version (modulo >>>>>>> separation of indexOf/lastIndexOf for trusted/non-trusted callers): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~vlivanov/dlong/8158168/redundant_checks/ >>>>>>> >>>>>>> Other than that, the difference is mainly about undoing refactorings >>>>>>> and removing verification logic (asserts + checks in the JVM). >>>>>>> >>>>>>> There are still unsafe accesses which are considered safe in both >>>>>>> versions (see StringUTF16.Trusted usages in the initial version [1]). >>>>>>> >>>>>>> We used to provide safe wrappers for unsafe intrinsics which makes it >>>>>>> much easier to reason about code correctness. I'd like to see compact >>>>>>> string code refactored that way and IMO the initial version by Dean >>>>>>> is a big step in the right direction. >>>>>>> >>>>>>> I still prefer to see a point fix in 9 and major refactoring >>>>>>> happening in 10, but I'll leave the decision on how to proceed with >>>>>>> the fix to core-libs folks. After finishing the exercise minimizing >>>>>>> the fix, I'm much more comfortable with the initial fix [1] (though >>>>>>> there are changes I consider excessive). >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>> [1] http://cr.openjdk.java.net/~dlong/8158168/webrev.0 >>>>>>> >>>>>>>>>>>> Some clarifications: >>>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> src/java.base/share/classes/java/lang/String.java: >>>>>>>>>>>> >>>>>>>>>>>> The bounds check is needed only in String.nonSyncContentEquals >>>>>>>>>>>> when it >>>>>>>>>>>> extracts info from AbstractStringBuilder. I don't see how out of >>>>>>>>>>>> bounds access can happen in String.contentEquals: >>>>>>>>>>>> if (n != length()) { >>>>>>>>>>>> return false; >>>>>>>>>>>> } >>>>>>>>>>>> ... >>>>>>>>>>>> for (int i = 0; i < n; i++) { >>>>>>>>>>>> if (StringUTF16.getChar(val, i) != cs.charAt(i)) { >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK. >>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> src/java.base/share/classes/java/lang/StringConcatHelper.java: >>>>>>>>>>>> >>>>>>>>>>>> I think bounds checks in StringConcatHelper.prepend() are skipped >>>>>>>>>>>> intentionally, since java.lang.invoke.StringConcatFactory >>>>>>>>>>>> constructs >>>>>>>>>>>> method handle chains which already contain bounds checks: array >>>>>>>>>>>> length >>>>>>>>>>>> is precomputed based on argument values and all accesses are >>>>>>>>>>>> guaranteed to be in bounds. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This is calling the trusted version of getChars() with no bounds >>>>>>>>>>> checks. It was a little more obvious when I had the Trusted inner >>>>>>>>>>> class. >>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>>> >>>>>>>>>>>> + static void putChar(byte[] val, int index, int c) { >>>>>>>>>>>> + assert index >= 0 && index < length(val) : "Trusted caller >>>>>>>>>>>> missed bounds check"; >>>>>>>>>>>> >>>>>>>>>>>> Unfortunately, asserts can affect inlining decisions (since they >>>>>>>>>>>> increase bytecode size). In order to minimize possible performance >>>>>>>>>>>> impact, I suggest to remove them from the fix targeting 9. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Sure. >>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> private static int indexOfSupplementary(byte[] value, int >>>>>>>>>>>> ch, int >>>>>>>>>>>> fromIndex, int max) { >>>>>>>>>>>> if (Character.isValidCodePoint(ch)) { >>>>>>>>>>>> final char hi = Character.highSurrogate(ch); >>>>>>>>>>>> final char lo = Character.lowSurrogate(ch); >>>>>>>>>>>> + checkBoundsBeginEnd(fromIndex, max, value); >>>>>>>>>>>> >>>>>>>>>>>> The check is redundant here. fromIndex & max are always inbounds by >>>>>>>>>>>> construction: >>>>>>>>>>>> >>>>>>>>>>>> public static int indexOf(byte[] value, int ch, int >>>>>>>>>>>> fromIndex) { >>>>>>>>>>>> int max = value.length >> 1; >>>>>>>>>>>> if (fromIndex < 0) { >>>>>>>>>>>> fromIndex = 0; >>>>>>>>>>>> } else if (fromIndex >= max) { >>>>>>>>>>>> // Note: fromIndex might be near -1>>>1. >>>>>>>>>>>> return -1; >>>>>>>>>>>> } >>>>>>>>>>>> ... >>>>>>>>>>>> return indexOfSupplementary(value, ch, fromIndex, max); >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK. >>>>>>>>>>> >>>>>>>>>>>> ============ >>>>>>>>>>>> I moved bounds checks from StringUTF16.lastIndexOf/indexOf to >>>>>>>>>>>> ABS.indexOf/lastIndexOf. I think it's enough to do range check on >>>>>>>>>>>> ABS.value & ABS.count. After that, all accesses should be >>>>>>>>>>>> inbounds by >>>>>>>>>>>> construction (in String.indexOf/lastIndexOf): >>>>>>>>>>>> >>>>>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>>> static int lastIndexOf(byte[] src, byte srcCoder, int srcCount, >>>>>>>>>>>> String tgtStr, int fromIndex) { >>>>>>>>>>>> >>>>>>>>>>>> int rightIndex = srcCount - tgtCount; >>>>>>>>>>>> if (fromIndex > rightIndex) { >>>>>>>>>>>> fromIndex = rightIndex; >>>>>>>>>>>> } >>>>>>>>>>>> if (fromIndex < 0) { >>>>>>>>>>>> return -1; >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> jdk/src/java.base/share/classes/java/lang/StringUTF16.java: >>>>>>>>>>>> public static int lastIndexOf(byte[] src, int srcCount, >>>>>>>>>>>> byte[] tgt, int tgtCount, int >>>>>>>>>>>> fromIndex) { >>>>>>>>>>>> int min = tgtCount - 1; >>>>>>>>>>>> int i = min + fromIndex; >>>>>>>>>>>> int strLastIndex = tgtCount - 1; >>>>>>>>>>>> char strLastChar = getChar(tgt, strLastIndex); >>>>>>>>>>>> >>>>>>>>>>>> startSearchForLastChar: >>>>>>>>>>>> while (true) { >>>>>>>>>>>> while (i >= min && getChar(src, i) != strLastChar) { >>>>>>>>>>>> >>>>>>>>>>>> There are 2 places: >>>>>>>>>>>> * getChar(tgt, strLastIndex) => getChar(tgt, tgtCount-1) - >>>>>>>>>>>> inbound >>>>>>>>>>>> >>>>>>>>>>>> * getChar(src, i); i in [ min; min+fromIndex ] >>>>>>>>>>>> min = tgtCount - 1 >>>>>>>>>>>> rightIndex = srcCount - tgtCount >>>>>>>>>>>> fromIndex <= rightIndex >>>>>>>>>>>> >>>>>>>>>>>> 0 <= min + fromIndex <= min + rightIndex == (tgtCount >>>>>>>>>>>> - 1) >>>>>>>>>>>> + (srcCount - tgtCount) == srcCount - 1 >>>>>>>>>>>> >>>>>>>>>>>> Hence, should be covered by the check on count & value: >>>>>>>>>>>> public int lastIndexOf(String str, int fromIndex) { >>>>>>>>>>>> + byte[] value = this.value; >>>>>>>>>>>> + int count = this.count; >>>>>>>>>>>> + byte coder = this.coder; >>>>>>>>>>>> + checkIndex(count, value.length >> coder); >>>>>>>>>>>> return String.lastIndexOf(value, coder, count, str, >>>>>>>>>>>> fromIndex); >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, I will go with your version if it's OK with Sherman. >>>>>>>>>>> >>>>>>>>>>> dl >>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Vladimir Ivanov >>>>>>>>>>>> >>>>>>>>>>>>> On 3/17/17 5:58 AM, Vladimir Ivanov wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I have the same concern. Can we fix the immediate problem in >>>>>>>>>>>>>>>> 9 and >>>>>>>>>>>>>>>> integrate verification logic in 10? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> OK, Tobias is suggesting having verification logic only >>>>>>>>>>>>>>> inside the >>>>>>>>>>>>>>> intrinsics. Are you suggesting removing that as well? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes and put them back in 10. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm OK with removing all the verification, but that won't reduce >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> library changes much. I could undo the renaming to >>>>>>>>>>>>>>> Trusted.getChar, but >>>>>>>>>>>>>>> we would still have the bounds checks moved into StringUTF16. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I suggest to go with a point fix for 9: just add missing range >>>>>>>>>>>>>> checks. >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>> >>>>> >>> >> >